Abstract
Foundation models let a robots accept missions in natural language, but they also let a wrong or malicious instruction reach the propellers. PEACE [1] is a planner–executor agent for robots in which a large language model emits a complete, typed mission plan in a single call, and an independent safety service validates every motion command before actuation. That service currently enforces two geometric constraints: an altitude band and a horizontal geofence. It cannot express, let alone enforce, the rules that matter in the real world: keep distance from people, do not hover in front of windows, maintain a minimum altitude over roads, respect regulatory limits. This project replaces the geometric check with a two-tier guardrail: a plan-level check that grounds natural-language safety rules into formal specifications and verifies the whole mission plan before dispatch, and an execution-level check that grounds the same rules into spatial keep-out sets around detected objects.
Supervision
- Erdem Uysal
Motivation
Two lines of work show how to give LLM-enabled robots contextual safety. RoboGuard [2] uses a root-of-trust LLM, shielded from user input, to translate high-level rules into linear temporal logic over the robot’s current world model, and then checks or repairs the planner’s proposal; it reduces the success of jailbreaking attacks from over 92% to under 3%. CORE [3] uses a vision-language model to infer safety constraints directly from images, grounds them as spatial safe sets, and enforces them with control barrier functions. Both were developed for ground robots, both operate inside a step-by-step planning loop, and neither has been combined with the other.
PEACE’s architecture invites exactly that combination. Because the planner produces one typed plan per mission over a fixed tool set, the plan is a finite sequence over a known alphabet. A plan-level guardrail therefore needs one automaton check per mission instead of one per step, and it adds no latency to the control path, which is PEACE’s central design goal. PEACE’s separation of a static system prompt from the volatile user prompt already provides the isolation a root-of-trust model requires. And PEACE’s 3D world model provides the object positions needed to ground rules such as “stay 10 m away from any person” as geometric keep-out regions at execution time. Physical-safety benchmarks for LLM-controlled UAVs [4] and the jailbreaking literature [5] provide the evaluation.
Goal
The student will design, implement, and evaluate a guardrail service for PEACE with two tiers:
- Plan-level tier. A root-of-trust LLM grounds a small set of textual UAV safety rules (privacy, proximity to people, minimum altitude over roads, regulatory limits, return-to-launch on low battery) into temporal-logic specifications over propositions of the form tool(argument) combined with world-model objects. The emitted mission plan is checked against the specification with an automaton library such as Spot [6]; on rejection, the violated proposition is returned to PEACE’s bounded replanner as structured feedback.
- Execution-level tier. The same predicates are grounded spatially, in the spirit of CORE: keep-out cylinders around detected people, minimum-altitude surfaces over road segments. These are enforced by PEACE’s existing pre-actuation check; enforcement as a control barrier function on the velocity setpoint is an optional extension.
- Evaluation in PX4 software-in-the-loop simulation with Gazebo: utility on PEACE’s prompt set (must not degrade), attack success rate on a set of harmful UAV behaviors with and without each tier, an ablation of chain-of-thought reasoning in the root-of-trust model, and measured overhead in LLM calls and wall-clock time.
- A thesis and, if results allow, a workshop or conference submission.
Requirements
- Strong Python skills; working knowledge of ROS 2 and ideally PX4 or another autopilot.
- Background in formal methods or logic (temporal logic, automata) at the level of a bachelor’s course, or willingness to acquire it early in the thesis.
- Familiarity with LLM APIs and prompt engineering.
- Helpful: basic control theory (for the control barrier function extension) and experience with Gazebo.
Pointers
[1] E. Uysal, T. Kehrer, and S. Panichella, “PEACE: A planner–executor agent with constraint enforcement for UAVs,” arXiv preprint arXiv:2606.00104, 2026.
[2] Z. Ravichandran, A. Robey, V. Kumar, G. J. Pappas, and H. Hassani, “Safety guardrails for LLM-enabled robots,” arXiv preprint arXiv:2503.07885, 2025.
[3] Z. Ravichandran, D. Snyder, A. Robey, H. Hassani, V. Kumar, and G. J. Pappas, “Contextual safety reasoning and grounding for open-world robots,” arXiv preprint arXiv:2602.19983, 2026.
[4] Y.-C. Tang, P.-Y. Chen, and T.-Y. Ho, “Defining and evaluating physical safety for large language models,” arXiv preprint arXiv:2411.02317, 2024.
[5] A. Robey, Z. Ravichandran, V. Kumar, H. Hassani, and G. J. Pappas, “Jailbreaking LLM-controlled robots,” arXiv preprint arXiv:2410.13691, 2024.
[6] A. Duret-Lutz, A. Lewkowicz, A. Fauchille, T. Michaud, E. Renault, and L. Xu, “Spot 2.0 — A framework for LTL and ω-automata manipulation,” in Proc. Int. Symp. Automated Technology for Verification and Analysis (ATVA), 2016, pp. 122–129.
[7] A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada, “Control barrier functions: Theory and applications,” in Proc. European Control Conf. (ECC), 2019, pp. 3420–3431.
[8] Z. Ravichandran, V. Murali, M. Tzes, G. J. Pappas, and V. Kumar, “SPINE: Online semantic planning for missions with incomplete natural language specifications in unstructured environments,” arXiv preprint arXiv:2410.03035, 2024.
Contact
Erdem Uysal, Institute of Computer Science, University of Bern ()