Next edition of the workshop will be held in conjunction with ECCV 2022! -> [slrtp-2022.github.io]

The "Sign Language Recognition, Translation & Production" (SLRTP) Workshop brings together researchers working on different aspects of vision-based sign language research (including body posture, hands and face) and sign language linguists. The aims are to increase the linguistic understanding of sign languages within the computer vision community, and also to identify the strengths and limitations of current work and the problems that need solving. Finally, we hope that the workshop will cultivate future collaborations.

Recent developments in image captioning, visual question answering and visual dialogue have stimulated significant interest in approaches that fuse visual and linguistic modelling. As spatio-temporal linguistic constructs, sign languages represent a unique challenge where vision and language meet. Computer vision researchers have been studying sign languages in isolated recognition scenarios for the last three decades. However, now that large scale continuous corpora are beginning to become available, research has moved towards continuous sign language recognition. More recently, the new frontier has become sign language translation and production where new developments in generative models are enabling translation between spoken/written language and continuous sign language videos, and vice versa. In this workshop, we propose to bring together researchers to discuss the open challenges that lie at the intersection of sign language and computer vision.

Call for Papers

We are seeking submissions! If you would like the chance to present your work, please submit a paper to CMT at https://cmt3.research.microsoft.com/SLRTP2020/ by the end of July 6 (Anywhere on Earth). We are happy to receive submissions for both new work as well as work which has been accepted to other venues. In line with the Sign Language Linguistics Society (SLLS) Ethics Statement for Sign Language Research, we encourage submissions from Deaf researchers or from teams which include Deaf individuals, particularly as co-authors but also in other roles (advisor, research assistant, etc).

Suggested topics for contributions include, but are not limited to:

  • Continuous Sign Language Recognition and Analysis
  • Multi-modal Sign Language Recognition and Translation
  • Generative Models for Sign Language Production
  • Non-manual Features and Facial Expression Recognition for Sign Language
  • Hand Shape Recognition
  • Lip-reading/speechreading
  • Sign Language Recognition and Translation Corpora
  • Semi-automatic Corpora Annotation Tools
  • Human Pose Estimation

Paper Length and Format: Submissions should use the ECCV template and preserve anonymity. All the submissions will be subject to double-blind review process. A paper can be submitted in either long-format (full paper) or short-format (extended abstract):

  • Full papers should be no more than 14 pages (excluding references) and should contain new work that has not been admitted to other venues.
  • Extended abstracts should be no more than 4 pages (including references). A short paper can describe new, previously, or concurrently published research or work-in-progress.

Proceedings: Full papers will appear in the Springer ECCV workshop proceedings and on the workshop website. Extended abstracts will appear on the workshop website.

Workshop languages/accessibility: The languages of this workshop are English, British Sign Language (BSL) and American Sign Language (ASL). Interpretation between BSL/English and ASL/English will be provided, as will English subtitles, for all pre-recorded and live Q&A sessions. If you have questions about this, please contact dcal@ucl.ac.uk.

Download Latex Template CMT Submission Website

Dates

    Paper submission:
    July 19, 2020
    Notification of acceptance:
    July 24, 2020
    Pre-print and presentation submission*:
    July 31, 2020
    Workshop date:
    August 23, 2020
    Camera ready submission:
    September 14, 2020
*Presentation submission deadline is final and will not be extended, as they will be translated to ASL&BSL.

Keynotes

Lale Akarun

Lale
Akarun

Professor
Bogazici University

Matt Huenerfauth

Matt
Huenerfauth

Director of the School of Information
Rochester Institute of Technology

Oscar Koller

Oscar
Koller

Applied Scientist
Microsoft

Christian Vogler

Christian
Vogler

Professor, Director of Technology Access Program
Gallaudet University

Bencie Woll

Bencie
Woll

Professor
Deafness, Cognition and Language Research Centre (DCAL), UCL

Live Event Schedule

Date: 23 August

Time: 14:00-18:00 GMT+1 (UK Time)

  • 1400

    Opening Remarks

  • 1410
    Bencie Woll

    Invited talk by Bencie Woll:
    Processing Sign Languages: Linguistic, Technological, and Cultural Challenges

    Bio

    Professor Bencie Woll has been involved in research on sign language for nearly 40 years, starting with research at the University of Bristol where she was a co-founder of the Centre for Deaf Studies, pioneering research on the linguistics of BSL and on Deaf Studies. She moved to City University London in 2005 to take up the newly created Chair in Sign Language and Deaf Studies. From City she moved in 2005 to UCL, where she is Professor of Sign Language and Deaf Studies. In 2006, together with colleagues at UCL and City, she founded DCAL, and served as Diector until 2016. Her research and teaching interests embrace a wide range of topics related to sign language, including the linguistics of British Sign Language (BSL) and other sign languages, the history and sociolinguistics of BSL and the Deaf community, the development of BSL in young children, and sign language and the brain, including developmental and acquired sign language impairments.

    Presenters:
    Bencie Woll

    Bencie Woll

  • 1440
    Oscar Koller

    Invited talk by Oscar Koller:
    Sign Language Recognition - From Dispersed to Comparable Research

    Bio:

    Since January 2018, Oscar Koller is an applied scientist in Microsoft’s Speech and Language group led by Xuedong Huang. From 2011 till 2018, he was a doctoral student researcher in the Human Language Technology & Pattern Recognition Group led by Prof. Ney at RWTH Aachen University, Germany. He followed a dual supervision by Prof. Bowden and his Cognitive Vision group at University of Surrey, UK, where he spent 12 months as a visiting researcher. His main research interests include the areas of speech recognition, computer vision, sign language recognition, gesture recognition and lip reading.

    Abstract:

    In this talk we will look into the state of the art in sign language recognition to enable us sketch the requirements for future research that is needed. The talk is meant to be an overview of the field, focusing on the move from dispersed research to the current momentum it has gained. We will look into comparable research studies on the available benchmark data sets. We will analyse the statistics of popular sign language tasks to understand what is needed to continue on the field's accelerated journey to real accessibility. This will also include an investigation in how our research published at TPAMI helps dealing with the specific challenges in sign language recognition.

    Presenters:
    Oscar Koller

    Oscar Koller

  • 1510

    Coffee Break 1

  • 1520
    Christian Vogler

    Invited talk by Christian Vogler:
    Sign Language Technologies: What are We Hoping to Accomplish?

    Bio

    Dr. Christian Vogler is a professor in the Communication Studies program. He also is the director of the Technology Access Program research group. He leads and co-leads multiple research grants that focus on accessible technology for the deaf and hard of hearing. Topics include video relay services, captioned telephone services, closed captions for TV and streaming video, better consumer control over hearing aids and cochlear implants, next-generation smart home alerting systems, and accessibility of voice interfaces like Amazon Alexa and Google Assistant. He collaborates closely with Dr. Raja Kushlanagar at the IT program, Dr. Patrick Boudreault at Interpreting, and HSLS faculty.
    In his role, he also is involved in bringing consumers, industry, and policymakers together on accessibility issues, advocating for a deaf/hard of hearing perspective, as well as developing prototype technologies for improving the accessibility of such systems. Prior to joining TAP in 2011, Dr. Vogler has worked on various research projects related to sign language recognition and facial expression recognition from video at the University of Pennsylvania; the Gallaudet Research Institute; UNICAMP in Campinas, Brazil; and the Institute for Language and Speech Processing in Athens, Greece.
    Dr. Vogler passionately believes that deaf and hard of hearing people have only scratched the surface of what is possible with the internet and mobile communication technologies, and that the most exciting technological developments are still to come. He always is on the lookout for students who are interested in communication technologies and want to make a difference in how we use them.

    Abstract

    When we mention sign language recognition technologies to the deaf community, the most common response is a collective groan. In this talk we explain why this is the case. We further provide an analysis of the challenges with the current state of the field, and what can be done to improve matters. Collaboration with the deaf front and center is key, as is identifying realistic applications that people will want to use, based on inclusive principles that respect the community.

    Presenters:
    Christian Vogler

    Christian Vogler

  • 1550
    Matt Huenerfauth

    Invited talk by Matt Huenerfauth:
    Creating Useful Applications with Imperfect, Sign-Language Technologies

    Bio

    Matt Huenerfauth is a Professor at Rochester Institute of Technology (RIT) and director of RIT's iSchool (School of Information). He studies the design of technology to benefit people who are Deaf or Hard of Hearing or who have low written-language literacy, and his team of over 30 research students operates bilingually in English and American Sign Language (ASL). He has secured $5 million USD in external research funding since 2007, including a U.S. National Science Foundation CAREER Award in 2008. He has authored over 90 peer-reviewed scientific journal articles, book chapters, and conference papers, and he is a five-time recipient of the Best Paper Award at the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS), more than any other individual in the conference history. In 2019, he completed a maximum six-year term as editor-in-chief of the ACM Transactions on Accessible Computing (TACCESS) journal. In 2018, RIT awarded him the Trustees Scholarship Award, the university’s highest honor for faculty research.

    Abstract

    Creating sign-language recognition and synthesis technologies is difficult, and state-of-the-art systems are still imperfect. This limitation presents a challenge for researchers in seeking resources to support dataset creation, user requirements gathering, and other critical infrastructure for the field. This talk examines how it is possible to create useful applications in the near-term, to motivate research that would have long-term benefit to the field. Examples of funded projects that integrate imperfect sign-language technologies are discussed, including: providing automatic feedback for students learning American Sign Language (ASL) through analysis of videos of their signing, creating search-by-video interfaces for ASL dictionaries, generating understandable ASL animations to improve information access, and providing ASL content in reading-assistance software. The common thread is that the technologies at the core of each project (i.e. human animation synthesis or recognition of video of human motion) are all imperfect artificial-intelligence systems that occasionally fail in non-human-like ways. We investigate how to adapt these imperfect technologies for new domains, and we use human-computer interaction research methods to evaluate alternative system designs. Our goal is to enable users to cope with current limitations of these intelligent technologies so that they benefit from applications that employ them.

    Presenters:
    Matt Huenerfauth

    Matt Huenerfauth

  • 1620

    Q&A for Accepted Papers - Session 1

    1- Automatic Segmentation of Sign Language into Subtitle-Units
    Hannah Bull (LIMSI-CNRS), Michele Gouiffes (LIMSI-CNRS), Annelies Braffort (LIMSI-CNRS)
    2- Phonologically-meaningful Subunits for Deep Learning-based Sign Language Recognition
    Mark Borg (University of Malta), Kenneth P. Camilleri (University of Malta)
    3- Recognition of affective and grammatical facial expressions: a study for Brazilian sign language
    Emely Pujolli da Silva (University of Campinas), Paula Dornhofer Paro Costa (University of Campinas), Kate Mamhy Oliveira Kumada (Federal University of ABC), José Mario De Martino (University of Campinas), Gabriela Araujo Florentino (Seli Institute)
    4- Real-Time Sign Language Detection using Human Pose Estimation
    Amit Moryossef (Bar-Ilan University, Google), Ioannis Tsochantaridis (Google), Roee Aharoni (Google), Srini Narayanan (Google), Sarah Ebling (University Of Zurich)
    5- Exploiting 3D Hand Pose Estimation in Deep Learning-Based Sign Language Recognition from RGB Videos
    Maria Parelli (National Technical University of Athens), Katerina Papadimitriou (University of Thessaly), Gerasimos Potamianos (University of Thessaly), Georgios Pavlakos (University of Pennsylvania), Petros Maragos (National Technical University of Athens)
  • 1640

    Coffee Break 2

  • 1650
    Lale Akarun

    Invited talk by Lale Akarun:
    Turkish Sign Language Recognition at Boğaziçi University

    Bio

    Lale Akarun is a professor of Computer Engineering and the director of the Center for Telecommunications and Informatics (http://tetam.boun.edu.tr/en). She received the PhD degree in Electrical Engineering from the Polytechnic School of Engineering of NYU, in 1992. She has been a faculty member at Bogazici University, Istanbul since 1993. She has served as Department Head of Computer Engineering (2010-2012) and Vice Rector for Research (2012-2016). As Vice Rector, her responsibilities included Sponsored Research Projects, Technology Transfer, Incubation Centers and Technoparks of the University. Her research areas are in image processing and computer vision, and in particular, processing of faces and gestures. She has supervised over 50 graduate theses and published more than 200 scholarly papers in scientific journals and refereed conferences. She has conducted research projects in biometrics, face recognition, hand gesture recognition, human-computer interaction, and sign language recognition.

    Presenters:
    Lale Akarun

    Lale Akarun

  • 1720

    Q&A for Accepted Papers - Session 2

    6- A Plan for Developing an Auslan Communication Technologies Pipeline
    Jessica Korte (The University of Queensland), Axel Bender (Defense Science and Technology, Australia), Guy Gallasch (Defense Science and Technology, Australia), Janet Wiles (The University of Queensland), Andrew Back (The University of Queensland)
    7- A Multi-modal Machine Learning Approach and Toolkit to Automate Recognition of Early Stages of Dementia among British Sign Language Users
    Xing Liang (University of Greenwich), Anastassia Angelopoulou (University of Westminster), Epaminondas Kapetanios (University of Westminster), Bencie Woll (DCAL, University College of London), Reda Al-batat (University of Westminster), Tyron Woolfe (DCAL, University College London)
    8- Score-level Multi Cue Fusion for Sign Language Recognition
    Çağrı Gökçe (Bogazici University), Oğulcan Özdemir (Bogazici University), Ahmet Alp Kindiroglu (Bogazici University), Lale Akarun (Bogazici University)
    9- Unsupervised Discovery of Sign Terms by K-Nearest Neighbours Approach
    Korhan Polat (Bogazici University), Murat Saraçlar (Bogazici University)
    10- Improving Keyword Search Performance in Sign Language with Hand Shape Features
    Nazif Can Tamer (Bogazici University), Murat Saraçlar (Bogazici University)
    11- Towards Continuous Recognition of Illustrative and Spatial Structures in Sign Language
    Valentin Belissen (LIMSI-CNRS), Annelies Braffort (LIMSI-CNRS), Michele Gouiffes (LIMSI-CNRS)
    12- Attention is All You Sign: Sign Language Translation with Transformers
    Kayo Yin (École Polytechnique), Jesse Read (Ecole Polytechnique)
    13- How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language
    Amanda Duarte (Universitat Politecnica de Catalunya), Shruti Palaskar (Carnegie Mellon University), Deepti Ghadiyaram (Facebook AI), Kenneth DeHaan (Gallaudet University), Florian Metze (Carnegie Mellon University), Jordi Torres (Universitat Politecnica de Catalunya), Xavier Giro-i-Nieto (Universitat Politecnica de Catalunya)
    14- Can Everybody Sign Now? Exploring Sign Language Video Generation from 2D Poses
    Lucas Ventura (Universitat Politecnica de Catalunya), Amanda Duarte (Universitat Politecnica de Catalunya), Xavier Giro-i-Nieto (Universitat Politecnica de Catalunya)
    15- 3D Hands, Face and Body Extraction for Sign Language Recognition
    Agelos Kratimenos (National Technical University of Athens), Georgios Pavlakos (University of Pennsylvania), Petros Maragos (National Technical University of Athens)
    16- Effect of Ranking and Precision of Results on Users' Satisfaction with Search-by-Video Sign-Language Dictionaries
    Saad Hassan (Rochester Institute of Technology), Oliver Alonzo (Rochester Institute of Technology), Abraham Glasser (Rochester Institute of Technology), Matt Huenerfauth (Rochester Institute of Technology)
    17- Fingerspelling recognition in the wild with iterative visual attention
    Bowen Shi (Toyota Technological Institute at Chicago), Aurora Martinez Del Rio (University of Chicago), Jonathan Keane (University of Chicago), Diane Brentari (University of Chicago), Greg Shakhnarovich (Toyota Technological Institute at Chicago), Karen Livescu (Toyota Technological Institute at Chicago)
    18- On How Deaf and Hard of Hearing Users Might Use Sign Language Conversational User Interfaces
    Abraham Glasser (Rochester Institute of Technology), Vaishnavi Mande (Rochester Institute of Technology), Matt Huenerfauth (Rochester Institute of Technology)
  • 1755

    Closing Remarks

Important Notes

Live Session Date and Time : 23 August 14:00-18:00 GMT+1 (BST)

  • The presentation materials and the live interaction session will be accessible only to delegates registered to ECCV during the conference, the recordings will be made publicly available afterwards.

  • The morning session (06:00-08:00) is dedicated to playing pre-recorded, translated and captioned presentations. There wil be no live interaction in this time.

  • To access recordings: Look for the email from ECCV 2020 that you received after registration (if you registered before 19 August this would be “ECCV 2020 Launch"). Follow the instructions in that email to reset your ECCV password and then login to the ECCV site.
    Click on "Workshops" and then "Workshops and Tutorial Site", then choose Sign Language Recognition, Translation and Production (link here if you are already logged in). There will be a list of all recorded SLRTP presentations – click on each one and then click the Video tab to watch the presentation.

  • Please watch the pre-recorded presentations of the accepted papers before the live session. We will have their Q&A discussions during the live session. If you have questions for the authors, we encourage you to submit them here in advance, to save time.

  • As an atendee please use the Q&A functionality to ask your questions to the presenters during the live event. You can also use the Chat to raise technical issues.

  • During live Q&A session we suggest you to use Side-by-side Mode. You can activate it by clicking on Viewing Options (at the top) and selecting Side-by-side Mode.

Videos

Keynotes - Playlist

Full Papers - Playlist

Extended Abstracts - Playlist

Workshop - Recording Transcript No#1 Transcript No#2

Accepted Papers

Full Papers - Playlist

Extended Abstracts - Playlist

Organizers

Cihan Camgoz

Necati Cihan
Camgöz

Research Fellow
University of Surrey

Gul Varol

Gül
Varol

Post-doc
University of Oxford

Samuel Albanie

Samuel
Albanie

Post-doc
University of Oxford

Cihan Camgoz

Neil
Fox

Research Assistant
DCAL

Richard Bowden

Richard
Bowden

Professor
University of Surrey

Andrew Zisserman

Andrew
Zisserman

Professor
University of Oxford

Kearsy Cormier

Kearsy
Cormier

Professor
DCAL

Special thanks to Ben Saunders, Bencie Woll, Liliane Momeni, Oscar Koller, and Sajida Chaudhary for their help and advice.

We would also like to thank Dr. Robert Adam for ASL and BSL translations, Akbar Sikder and Esther Rose Bevan for BSL Interpretations, Anna Michaels and Brett Best from Arrow Interpreting for ASL Interpretations, and Katy Ryder and Tara Meyer from MyClearTxt for the live captioning.

Our Sponsors

We thank our sponsors for their support, making it possible to provide
American Sign Language (ASL) and British Sign Language (BSL) translations for this workshop.

Microsoft
Google