Supervisors
- Position
- Senior Lecturer in Computer Science (Software Development)
- Division / Faculty
- Faculty of Science
- Position
- Associate Lecturer in Animation
- Division / Faculty
- Faculty of CI, Education & Social Justice
Overview
The Australian Deaf community is a vibrant minority-language community who use Auslan (Australian Sign Language). Students joining this project will work on an aspect of sign language technologies, with potential directions including:
- Working with Motion Capture recordings of native signers to control the movements of an avatar
- Working with a sign language encoding system, such as SiGML, to automate avatar movements for translations
- Exploring how we can use AI technologies, such as ML/Deep learning models to recognise, translate, or generate (elements of) signs
- Developing technologies to support sign language learning, for children and/or adults
- We are also open to proposals from students for
This topic is interdisciplinary, and attached to 2 ongoing projects (Breaking the Communication Barrier for the Australian Deaf Community, Google Award for Inclusion Research; and Auslan Assist, Sydney Trains)
Research engagement
Students will have the opportunity to engage with different aspects of a research project depending on which branch of the project they are most interested in. All students will have the opportunity to undertake development work. Some students will undertake literature review work. Some students may have the opportunity to undertake supervised field work.
Research activities
Option 1: Motion Capture avatar work
Use Motion Capture data from native signers to control Epic Metahuman avatars in Unreal Engine, creating sequences of signs that can be used in translating English messages to Auslan.
Possibly: attend Motion Capture sessions with native signers.
Write research report.
This work will be primarily with Maria Zelenskaya
Option 2: Sign encoding
Comparison of sign encoding systems and tools.
Selection or development of a sign encoding tool.
Connect sign encoding tool with our existing signing avatar.
Possibly: develop automatic translation system from English to encoded sign.
Write research report.
This work will be with both Jessica Korte and Maria Zelenskaya
Option 3: Auslan AI
Literature review of relevant ML/Deep Learning/other AI models/approaches.
Comparison and selection of sign language dataset.
Development/adaptation and training of models on high performance computing infrastructure.
Write research report.
This work will be primarily with Jessica Korte
Option 4: Auslan learning technologies
Literature review of educational approaches.
Comparison of existing sign language learning tools and technologies.
Selection of educational and/or technology approach.
Development of prototype
Possibly: evaluation with users or proxies.
Write research report
Option 5: Your idea
Probably: Literature review, “market” comparison, prototype development
Possibly: data collection with potential users or proxies.
Write research report
Outcomes
Option 1: Develop a library of signs/phrases for the avatar.
Option 2: Develop a system for sign encoding (and possibly translation).
Option 3: Develop an AI model for some element of sign recognition, translation or generation.
Option 4: Develop a prototype of a sign language learning technology.
Skills and experience
Option 1: Motion Capture avatar work
Required: Experience with 3D animation and Unreal Engine
Option 2: Sign language encoding
Required: Experience with programming AND/OR Unreal Engine
Option 3: Auslan AI
Strongly recommended: Experience developing ML systems
Option 4: Auslan learning technologies
Strongly recommended: Programming
All: Sign language knowledge desired, not required.
Start date
22 November, 2024End date
14 February, 2025Location
This research can be undertaken at Gardens Point or Kelvin Grove. Virtual attendance may be available by arrangement.
Additional information
Appropriate computing resources will be available (e.g. remote access to QUT HPC, or computer suitable for animation work).
An existing avatar and sign language resources will be available for students’ use.
Keywords
- Auslan
- sign language
- AI
- ML
- Deep learning
- avatar
- animation
- education
- language acquisition
- programming
- development
- language
Contact
Jessica Korte
Jessica.korte@qut.edu.au