Reducing Simulation Overhead in UAV/Drone Test Generation Using Surrogate Models

Seminar, MSc Thesis
Page content

Context

Unmanned aerial vehicles (UAVs), also known as drones, are acquiring increasing autonomy. With their commercial adoption, the problem of testing their safety requirements has become a critical concern. Simulation-based testing represents a fundamental practice, but the testing scenarios considered in software-in-the-loop testing may be different from the actual scenarios experienced in the field.

Motivation

Previous work [1] suggested SURREALIST (teSting UAVs in the neighboRhood of REAl flIghtS), a search-based approach that analyses the logs from real UAV flights and automatically generates simulation-based test cases in the neighborhood of such real flights, thereby improving the realism and representativeness of the simulation-based tests.

While SURREALIST effectively exposes unstable and potentially unsafe behavior which even leads to crashes in realistic scenarios, the search process requires numerous iterations, with each test case undergoing multiple simulations to evaluate its fitness, making the approach computationally expensive.

Goal

This project proposes an extension to SURREALIST by incorporating a surrogate model to estimate the fitness of potential test cases, thereby reducing the number of simulations required. The surrogate model, trained on data from previous iterations, predicts the fitness of new test cases based on observed patterns, allowing the search process to converge more quickly to critical test cases that challenge UAV safety. Preliminary experiments suggest that this approach can significantly reduce the computational cost while maintaining the effectiveness of the test case generation.

Requirements

Recommended:

  • Python program language knowledge,
  • Experience with testing (e.g., courses, and some experience in writing test cases),
  • Prior knowledge/curiosity of ROS and Gazebo is a plus.

Reference(s)

  • [1] S. Khatiri, S. Panichella and P. Tonella, “Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Neighborhood of Real Flights,” 2023 IEEE Conference on Software Testing, Verification and Validation (ICST), Dublin, Ireland, 2023, pp. 281-292, doi: 10.1109/ICST57152.2023.00034. https://ieeexplore.ieee.org/document/10132225/

Contact

Sebastiano Panichella, sebastiano.panichella@unibe.ch