Thesis: GenAI-Enhanced Multimodal Sensor Fusion in Autonomous Driving

Company description:

Part of Volvo Group, Volvo Autonomous Solutions accelerates the development, commercialization and sales of autonomous transport solutions, focusing on defined segments for the on- and off-road space. The combination of strong tech expertise and skilled customer solutions creates innovative transport offers never seen before. We are constantly pushing our own skills and ability to drive change in a traditional industry to meet a growing customer demand. We are now looking for innovative, committed individuals to join us in our endeavor to create customer solutions that enhance safety, flexibility and productivity.

Job description:

 Thesis: GenAI-Enhanced Multimodal Sensor Fusion for Robust Object Detection in Autonomous Driving

Transport is at the core of modern society. Imagine using your expertise to shape sustainable transport and infrastructure solutions for the future. If you seek to make a difference on a global scale, working with next-gen technologies and the sharpest collaborative teams, then we could be a perfect match. 

What you will do

At Volvo Autonomous Solutions, we are at the forefront of innovation, designing the autonomous transport solutions of tomorrow. As part of our global and diverse team, you will have the opportunity to work on an industrial research challenge that contributes to the future of safe and reliable autonomous transport.

We are looking for a master’s thesis student to investigate how generative AI can improve multimodal sensor fusion and make 3D object detection more robust under adverse conditions and sensor failures.

The thesis project

Autonomous vehicles rely on multiple sensors, such as cameras, LiDAR and RADAR, to detect and understand their surroundings. Conventional sensor-fusion methods may not fully capture uncertainty or compensate effectively when sensor information is degraded or unavailable.

The goal of this thesis is to investigate whether a generative AI-based fusion module can improve the robustness of multimodal 3D object detection. The project will focus on developing a generative sensor-completion mechanism, for example by reconstructing degraded RADAR or LiDAR bird’s-eye-view features using camera information and other available modalities.

You will evaluate how the proposed approach affects 3D object-detection performance under adverse conditions. You will also investigate whether the system can degrade gracefully when one of the sensors becomes unavailable.

The work will include:

• Implementing a generative sensor-completion mechanism for degraded or missing sensor information.
• Integrating the completion mechanism with a pretrained bird’s-eye-view object detector.
• Training and evaluating the system using synchronized LiDAR, camera and RADAR data.
• Testing the approach under modality-dropout and simulated adverse-weather conditions.
• Measuring detection robustness and graceful degradation when sensor information is unavailable.
• Analysing the results and documenting your findings in a scientific report.

Resources and research environment

You will have access to a pretrained bird’s-eye-view 3D object detector, a generative completion model and synchronized LiDAR, camera and RADAR data. GPU compute resources, modality-dropout tools and adverse-weather simulation capabilities will also be available for developing and evaluating your approach.

You will work closely with experienced engineers and researchers in a collaborative AI and autonomy team, gaining insight into generative AI, multimodal perception and the development of robust autonomous-driving systems.

Who are you?

You are enrolled in a master’s programme in computer science, data science, machine learning, electrical engineering, robotics or a related field. You have solid Python programming skills and hands-on experience with a deep-learning framework such as PyTorch or TensorFlow.

We are looking for someone who is interested in autonomous systems, machine perception, generative AI and working with real-world sensor data.

You should also be:

• Independent, structured and curious.
• Analytical and motivated to solve challenging technical problems.
• Comfortable working with deep-learning models, multimodal data and experimental evaluation.
• Interested in sensor fusion and robust perception under challenging conditions.
• Proficient in written and spoken English.

What’s in it for you?

This thesis gives you the opportunity to work on a real industrial research problem with access to unique multimodal vehicle data and the tools needed to conduct advanced experiments. You will receive close technical supervision from experts in AI and autonomous systems, as well as access to GPU resources and relevant development tooling.

You will also become part of a dynamic and inclusive work environment where new ideas are encouraged and celebrated.

How to apply

To apply, send your CV, grade transcript and a short motivation explaining your interest in the project and relevant experience. Start and scope can be adjusted together with the supervisor.

We value your data privacy and therefore do not accept applications via mail. 

Who we are and what we believe in We are committed to shaping the future landscape of efficient, safe, and sustainable transport solutions. Fulfilling our mission creates countless career opportunities for talents across the group’s leading brands and entities.