A robot driving around a room often observes an object from an awkward position, where it may be partly hidden behind furniture, too distant, or lit from behind. A conventional vision pipeline runs an object detector on whatever image is available and accepts the result even when confidence is low. A robot on wheels, unlike a fixed camera, has a further option open to it: moving to a different position and observing again. Choosing where to look next, rather than only interpreting what is already in view, is known as active perception.
A robot moving through an unfamiliar building must build a picture of its surroundings and work out where it sits within them at the same time. Solving both problems together from camera images is known as SLAM, short for simultaneous localization and mapping. The cameras used here record a normal colour image together with the distance to everything in view; such devices are called depth cameras, or RGB-D cameras, and they suit indoor robots because they provide both appearance and scale without the cost of a laser scanner. Many open-source SLAM systems exist, and they differ considerably in how they store the map, how they recognize a place already visited, how much computing power they require, and how well they cope with dim light or rapid motion.
Indoor service robots carry medication, lab samples, and supplies through hospital corridors and move goods through warehouse aisles. They share this space with people who are not trained operators and who are often not paying attention to the robot. Navigation policies for such robots are increasingly learned through reinforcement learning or imitation. When a learned policy meets a situation outside its training data, it still produces an action, and nothing in that action tells the robot or its operator that the policy may be wrong.
Foundation models let a robots accept missions in natural language, but they also let a wrong or malicious instruction reach the propellers. PEACE [1] is a planner–executor agent for robots in which a large language model emits a complete, typed mission plan in a single call, and an independent safety service validates every motion command before actuation. That service currently enforces two geometric constraints: an altitude band and a horizontal geofence. It cannot express, let alone enforce, the rules that matter in the real world: keep distance from people, do not hover in front of windows, maintain a minimum altitude over roads, respect regulatory limits. This project replaces the geometric check with a two-tier guardrail: a plan-level check that grounds natural-language safety rules into formal specifications and verifies the whole mission plan before dispatch, and an execution-level check that grounds the same rules into spatial keep-out sets around detected objects.
Swarms of unmanned aerial vehicles (UAVs) are used in search and rescue, infrastructure inspection, and environmental monitoring. An operator supervising such a swarm has to follow many vehicles at once, understand where they are relative to one another, and issue commands to groups rather than to single drones. Ground control stations present this three-dimensional situation on flat screens, which leaves the operator to reconstruct the spatial picture mentally. A Mixed Reality (MR) headset can instead place drone positions, planned trajectories, and status information in the space where the drones actually fly.
PEACE [1] is a planner–executor agent for a single-robot systems: a large language model (LLM) turns a natural-language mission into one typed plan of tool calls, an executor dispatches it deterministically over ROS 2, and an independent constraint layer validates every motion command. Many missions that motivate language-tasked drones, such as area survey, search, or inspection of distributed infrastructure, are naturally multi-robot. This project extends the planner–executor paradigm from single-robot system to a multi-robot system: the planner must decompose a mission into per-robot plans with dependencies, the executor must coordinate several robots, the world model must be shared, and the constraint layer must enforce inter-robot constraints such as separation and operational environment partitioning in addition to per-robot geofences.
Large language models (LLMs) are increasingly used as high-level planners for robots: a natural-language mission is turned into a sequence of actions that a classical controller executes. Recent work has shown that such LLM-controlled robots can be jailbroken into performing physically harmful actions, often with 100% attack success, on commercial platforms [1]. Proposed defenses either wrap the planner in a guardrail [2] or restrict what the planner may emit. Planner–executor agents such as PEACE [3] take the second route: the LLM produces a single, fully typed plan over a fixed tool schema instead of free-form code or a step-by-step reasoning loop. Whether this architectural choice actually reduces the attack surface has never been measured. This project builds a reusable red-teaming harness for LLM-enabled robot planners in simulation and uses it to compare architectures under a common set of attacks.