GPU Performance and Energy Trade-offs in Simulation-based Testing of Autonomous Vehicles
Context
Autonomous vehicles (AVs) are complex cyberphysical systems that require extensive testing to ensure safety. Since field testing is costly and unsafe, simulation-based testing using platforms like CARLA and BeamNG.tech has become a cornerstone in AV software validation. These simulators rely heavily on GPU performance for rendering, physics, and sensor emulation, and are therefore both resource-intensive and energy-demanding. As the scale of simulation campaigns grows (thousands of tests per day in CI pipelines), understanding and optimizing GPU cost becomes critical for cost-effective and sustainable testing.
Motivation
Most research in AV testing focuses on failure detection and scenario generation, but few works address the resource costs of simulation. In practice, GPU performance is often the bottleneck: adding more agents, sensors, or realism reduces throughput (tests/hour) and increases energy consumption. At the same time, higher realism can increase the probability of exposing failures. This raises a key trade-off: How can we balance testing quality (e.g., failure detection) with GPU performance and energy efficiency? Addressing this trade-off is essential to make large-scale AV testing sustainable.
Goal
The goal of this project is to profile and analyze GPU performance vs. energy consumption in AV simulations under varying conditions. Students will:
- Set up a baseline scenario in CARLA or BeamNG.tech.
- Measure GPU utilization, CPU utilization, frequency, FPS, and energy usage during simulation runs.
- Vary factors such as:
- Map complexity
- Number of agents (vehicles, pedestrians)
- Sensors (number of cameras, LiDAR on/off)
- Weather and lighting
- Evaluate how these factors influence both GPU cost (performance and energy) and testing quality (failure/unsafe rate, e.g. collisions or lane departures).
- Produce plots and analyses of trade-offs between testing quality and GPU cost.
Deliverables
- Scripts for automated experiment runs and GPU/CPU/energy logging.
- A dataset of results.
- Analysis report (plots and discussion).
- Final presentation.
Requirements
- Programming skills in Python.
- Familiarity with Linux and basic system measurement.
- Interest in software testing and performance analysis.
- Prior experience with CARLA or BeamNG is useful but not mandatory.
Pointers
- CARLA Simulator: https://carla.org/
- BeamNG.tech Documentation: https://documentation.beamng.com/beamng_tech/
- Birchler et al., “Cost-effective Simulation-based Test Selection in Self-driving Cars Software with SDC-Scissor”, SANER 2022.
- Masoud Jamshidiyan Tehrani, Jinhan Kim, and Paolo Tonella. “PCLA: A Framework for Testing Autonomous Agents in the CARLA Simulator.” FSE 2025.