Robotics / 2023 / Validated on hardware

Assistive Perception System for a Quadruped Robot

Real-time ROS perception and audio-guidance system developed for a Unitree Go1 guide-dog concept, combining object detection, scene context, and synchronised spoken cues.

What this system solves

Existing assistive guidance is rigid, expensive, and rarely contextual. A quadruped platform with reliable scene perception and clear audio cues could expand assistive options — but only if perception is dependable and the human-facing feedback is intelligible in real environments.

System overview

I led the computer vision and perception integration on a team project around a Unitree Go1 quadruped. The system consumes camera frames, runs YOLO-based detection through darknet_ros, publishes perception events on ROS topics, and drives an audio feedback layer (sound_play) that warns the user about obstacles and nearby objects.

Architecture

• Camera frames feed a perception node running YOLO inference via darknet_ros.
• Detections are published on ROS topics with class labels and bounding boxes.
• An audio bridge subscribes to perception events and synthesises spoken cues through sound_play with debouncing so the user is not overwhelmed.
• The Unitree Go1 stack handles locomotion; perception and audio operate as parallel ROS subsystems.

Engineering decisions

What I built

• Led the computer-vision and perception integration for the quadruped guide-dog concept.
• Integrated darknet_ros YOLO detection into the ROS graph and exposed perception events on dedicated topics.
• Wrote the bridge between perception events and the sound_play audio layer so detections become intelligible cues.
• Participated in simulation and physical-robot validation runs and triaged sensor / perception failures.

Validation and results

• Verified perception behaviour in Gazebo across multiple simulated environments.
• Validated the perception + audio loop on the physical Unitree Go1.

Limitations

• Perception relies on pre-trained YOLO weights; long-tail obstacle categories were not specifically tuned.
• Audio cue grammar is intentionally simple — a richer affordance layer would benefit from user research.
• This was a team project — locomotion and lower-level control are not my contribution and the README credits the team and upstream libraries.

What would come next

• User study with visually impaired participants to evaluate cue intelligibility and trust.
• Switch to depth-aware perception for better obstacle distance estimates.
• Failsafe behaviours when perception confidence drops below a tunable threshold.

Media gallery

Next