Enhancing 3D visual understanding through multimodal data fusion

Overview

The demand for 3D scene understanding through point clouds is rapidly growing in diverse applications, including augmented and virtual reality, autonomous driving, robotics, and environment monitoring. However, the field faces challenges due to limited data availability and predefined categories. Training deep 3D networks effectively for sparse LiDAR point clouds requires significant amounts of annotated data, which is both time-consuming and expensive. Building on the advancements in 2D models that leverage the power of image and language knowledge, our project aims to explore the potential of multimodal information extraction to enhance 3D understanding, especially in scenarios with limited annotated data.

Skills and experience

• Strong Python programming skills and machine learning experience.
• An appreciation of concepts in deep learning learning and computer vision.

Scholarships

You may be eligible to apply for a research scholarship.

Explore our research scholarships

Keywords

• deep learning
• machine learning
• computer vision
• large language models

Contact

Contact the supervisor for more information.