Abstract
A robot driving around a room often observes an object from an awkward position, where it may be partly hidden behind furniture, too distant, or lit from behind. A conventional vision pipeline runs an object detector on whatever image is available and accepts the result even when confidence is low. A robot on wheels, unlike a fixed camera, has a further option open to it: moving to a different position and observing again. Choosing where to look next, rather than only interpreting what is already in view, is known as active perception.
Supervision
- Prakash Aryan
- Sebastiano Panichella
Motivation
Running the detector repeatedly on the same poor view does not resolve the underlying ambiguity, because the difficulty lies in the viewpoint rather than in the algorithm. Moving slightly closer or off to one side may bring the object into clear view and sharpen the estimate of its position in three dimensions, although repositioning also consumes time, drains battery, and introduces some risk of collision. The question worth studying is therefore when moving is worthwhile, and how much improvement it produces compared with remaining stationary.
Goal
The student will build a system that decides when a robot should move in order to obtain a better view. The platform is a small wheeled research robot called the TurtleBot3, fitted with a camera that records both colour and distance, taken from the Intel RealSense family. A ready-made object detector first identifies an object and estimates its position in three dimensions, and if either the confidence or the distance reading proves unstable, the robot selects one of a small number of preset moves such as left, right, or closer. It stops once the estimate settles or once the allowance of viewpoints is exhausted. Neither detector training nor reinforcement learning is required.
Project Scope
- Select three to five everyday objects that an existing detector already recognizes, and construct a small set of repeatable scenes on a table or at floor level.
- Implement a baseline that reports the first detection without moving the robot, so that the active version has something to be compared against.
- Implement the active controller, which considers at most three safe candidate positions and chooses among them using detector confidence and the consistency of the distance reading.
- Move the robot using the standard robot navigation software, Nav2, or simple short relative commands, logging every observation, confidence value, estimated position, and movement.
Evaluation
- Detection: how often the object is found, and whether confidence improves once the robot has moved.
- Position accuracy: how far the estimated three-dimensional position lies from a hand-measured reference.
- Cost: the total distance driven, the number of viewpoints used, and the time taken to complete the task.
- Failure cases: scenes in which moving does not improve the result, or the object remains hidden.
Requirements
The student should know some Python, have encountered ROS 2 before, and have an interest in robot perception. Object detection, depth cameras, and navigation software can all be learned during the project. The topic deliberately relies on preset movements and an existing detector so that the scope remains achievable within a seminar.
Pointers
- R. Bajcsy, “Active Perception,” Proceedings of the IEEE, vol. 76, no. 8, 1988. The paper that gave the idea its name.
- R. Zeng, Y. Wen, W. Zhao, and Y.-J. Liu, “View planning in robot active vision: A survey,” Computational Visual Media, vol. 6, no. 3, 2020. A broad overview of the field.
- P. Ammirato et al., “A Dataset for Developing and Benchmarking Active Vision,” in Proc. IEEE ICRA, 2017.
- Korbach et al., “Next-Best-View Estimation for Active Object Classification.”
- Ekvall et al., “Integrating Active Mobile Robot Object Recognition and Navigation.”