Abstract
Swarms of unmanned aerial vehicles (UAVs) are used in search and rescue, infrastructure inspection, and environmental monitoring. An operator supervising such a swarm has to follow many vehicles at once, understand where they are relative to one another, and issue commands to groups rather than to single drones. Ground control stations present this three-dimensional situation on flat screens, which leaves the operator to reconstruct the spatial picture mentally. A Mixed Reality (MR) headset can instead place drone positions, planned trajectories, and status information in the space where the drones actually fly.
Supervision
- Prakash Aryan
- Sebastiano Panichella
Motivation
The design problem changes as the swarm grows. With five drones, every vehicle can be shown in full detail. With twenty, the same display becomes cluttered, and the operator can no longer attend to all of it. An interface for larger swarms therefore has to decide what to show, at which level of detail, and at what moment. This thesis studies how situational awareness changes as the swarm grows from five to twenty drones, and whether an MR interface that adapts its level of detail performs better than a fixed-detail MR interface and a conventional 2D ground control station.
Goal
The student will extend an existing MR dashboard for robot telemetry (Unity, MRTK3, Meta Quest 3) into an interaction layer for a simulated UAV swarm, connected through ROS 2 to PX4 SITL running in Gazebo. The thesis is expected to deliver:
- a level-of-detail scheme that simplifies or aggregates the representation of each drone according to the operator’s gaze, the drone’s distance, and its task state or anomalies
- spatial interaction for selecting groups, placing formations, and defining waypoints, comparing hand gestures, controllers, and gaze as input
- a within-subjects user study comparing adaptive-detail MR, fixed-detail MR, and a 2D baseline with swarms of 5, 10, and 20 simulated drones, measuring situational awareness, workload (NASA-TLX), and task performance
- the interface as open-source software, together with the study materials
Requirements
The student should be comfortable programming in C# with Unity and have a basic understanding of 3D transformations. Familiarity with ROS 2 and an interest in human-computer interaction and user studies are expected. Experience with PX4 or Gazebo is helpful but can be acquired during the thesis.
Pointers
- C. Zheng, A. Jarecki, and K. Lee, “Integrated system architecture with mixed-reality user interface for virtual-physical hybrid swarm simulations,” Scientific Reports, vol. 13, Art. no. 14761, 2023.
- P. Xu, J. Garcia, W. T. Ooi, and C. Jouffrais, “SafeSpect: Safety-First Augmented Reality Heads-up Display for Drone Inspections,” in Proc. CHI Conference on Human Factors in Computing Systems, 2025, pp. 1-17.
- P. Walker, J. Hamell, C. A. Miller, J. Ladwig, H. Wauck, and P. K. Keller, “Immersive Interaction Interface (I3): A Virtual Reality Swarm Control Interface,” IEEE Transactions on Field Robotics, vol. 1, pp. 424-445, 2024.
- PX4 Autopilot, “Multi-Vehicle Simulation with Gazebo,” PX4 User Guide. [Online]. Available: