Cozmo Robot Controller

A comprehensive Python controller for the Anki Cozmo robot (version 1), built on a patched version of pycozmo with an extended scripting language.

About Cozmo

Cozmo is a miniature educational robot originally created by Anki (2016-2019) and later continued by Digital Dream Labs. It's a charming, palm-sized robot with a distinctive personality powered by an "emotion engine" that allows it to express feelings through animations, sounds, and facial expressions on its OLED screen.

Hardware Specifications

Feature	Specification
Dimensions	~10 cm (4 inches) tall
Display	128×32 pixel OLED screen
Camera	320×240 VGA camera
Sensors	Cliff detection IR, gyroscope, accelerometer
Movement	2 tank treads, 1 lift arm, 1 head tilt
Connectivity	WiFi (creates own access point)
Lights	Backpack LEDs (5), IR head light
Audio	Built-in speaker
Battery	Li-ion, ~90 minutes runtime

Key Features

• Facial Recognition: Cozmo can recognize and remember faces
• Emotion Engine: Expresses happiness, sadness, surprise, and more
• 1000+ Animations: Pre-programmed movements and sounds
• Power Cubes: Interactive cubes for games and activities
• Programmable: Supports Python SDK and visual coding (Code Lab)

My story

I bought Cozmo ver 1, back at 2016 or 2017 because my younger son has seen it on Internet and he liked it. By the time I had the impression that this will be a great reason to introduce him to programming and robotics (he was 10 years old). But after playing with it for a while, he realized that - although it is intresting- his is not interested. Oh GOD!, I would loved to have something like this, plus all the tools and languages and platforms that are now available, back at start of 80's when I was a teenager. Anyway, fast forward 10 years after, the robot accumilated dust, and I decided to give it a try............

---

About pycozmo

This controller uses a patched version of pycozmo (roryt12/pycozmo), which is forked from the original zayfod/pycozmo. The patches fix several bugs that prevented proper robot operation.

pycozmo is a pure-Python communication library that provides an alternative SDK for Cozmo. Unlike the official Anki SDK, pycozmo:

• No mobile device required: Connects directly to Cozmo via WiFi
• No cloud dependency: Works completely offline
• Low-level access: Direct protocol communication
• Open source: Based on zayfod/pycozmo

The library is based on reverse-engineering of the Cozmo protocol and provides access to:

• Motor control (wheels, lift, head)
• Camera streaming
• Audio playback
• Display rendering
• Animation playback
• Sensor data

Note: The original project is described by its author as "unstable and heavily under development."

---

pycozmo Patches

This controller requires a patched version of pycozmo. The original library has several bugs that prevent proper robot operation. The patched version is available at:

https://github.com/roryt12/pycozmo

Required Patches

The following patches are included in the forked repository:

1. Protocol Buffer Size Fix (protocol_encoder.py) - Fixes AcknowledgeAction packet size mismatch
2. u-law Audio Encoding Fix (audio.py) - Prevents byte overflow in audio encoding
3. Procedural Face Rendering Fix (procedural_face.py) - Fixes "y1 must be greater than or equal to y0" error with animations that have invalid face keyframes
4. Client Initialization Fixes (client.py) - Various connection stability improvements

Installation

Clone and install the patched version:

git clone https://github.com/roryt12/pycozmo.git
cd pycozmo
pip install --user -e .

Compatibility

• Tested on: Cozmo version 1 (2016-2017, without button)
• Firmware: Compatible with firmware from Android app version 3.4
• Your mileage may vary: Different firmware versions may behave differently

---

Features

Robot Control

• ✅ Movement: Forward, backward, turning, precise navigation
• ✅ Head Control: Position control with angle aliases (up, down, middle)
• ✅ Lift Control: Speed-based control (position control not supported by firmware)
• ✅ Lights: Backpack LED control (individual and group)
• ✅ Screen: Text and image display on OLED

Audio

• ✅ Text-to-Speech: Via espeak with voice/language/speed/pitch options
• ✅ Sound Playback: From Cozmo's asset library (2214 WEM files) or external files
• ✅ Volume Control: 0-65535 range

Sensing

• ✅ Cliff Detection: Automatic stop on table edges (configurable reaction)
• ✅ Battery Monitoring: Voltage and percentage display
• ✅ Camera: Image capture

Animations

• ✅ Individual Animations: 993 available
• ✅ Animation Groups: 573 groups for random selection

Scripting

• ✅ Variables: Set and use with $var syntax
• ✅ Loops: for (range and list) and while loops
• ✅ Conditions: if/else/endif blocks
• ✅ Subroutines: def/call for reusable code
• ✅ Includes: Nested script files with spaces support
• ✅ Async Execution: Run animations and sounds in parallel

Safety

• ✅ Cliff Detection: Enabled by default with configurable reactions
• ✅ Connection Validation: Detects failed connections (serial=0, battery=0V)
• ✅ Loop Limits: Max 1000 iterations to prevent infinite loops
• ✅ Circular Include Protection: Prevents infinite recursion

---

Installation

Prerequisites

1. Python 3.8+ with pip
2. espeak for text-to-speech:

   sudo apt install espeak

3. sox for voice effects (optional):

   sudo apt install sox

4. ffmpeg for audio conversion:

   sudo apt install ffmpeg

5. vgmstream-cli for WEM audio files:

   # Build from source: https://github.com/vgmstream/vgmstream

6. Last but not least, pycozmo library, original or the patched I have in github (https://github.com/roryt12/pycozmo)

Download Assets

On first run, pycozmo will download Cozmo assets (~500MB) to ~/.pycozmo/assets/. The incuded script pycozmo_resources.py does this also.

---

Quick Start

Connect to Cozmo

1. Power on Cozmo
2. Lift the handles (or press the button, if it has one) to show WiFi credentials
3. Connect your computer to Cozmo's WiFi network
4. Run the controller:

python3 cozmo_controller.py status

Basic Commands

# Say hello
python3 cozmo_controller.py "say Hello World"

# Move forward 100mm
python3 cozmo_controller.py "move 100"

# Play an animation
python3 cozmo_controller.py "anim-group DanceMambo"

# Check battery
python3 cozmo_controller.py "battery mode=voltage"

Script File

Create myscript.txt:

# My first Cozmo script
say Hello! I am Cozmo.
head up
lights color=blue
anim-group CodeLabBored

Run it:

python3 cozmo_controller.py --script myscript.txt

---

Command Reference

Connection & Status

Command	Description
connect	Connect to robot
status	Show robot status
wait	Wait for robot to stabilize
sleep	<duration>

Movement

Command	Args	Options	Description
move	<distance>	speed=100, async=false	Move forward/backward (mm)
turn	<angle>	async=false	Turn by angle (degrees)
goto	<x> <y>	angle=0, async=false	Navigate to position
lift	<speed>	duration=1.0, async=false	Move lift (rad/s)

Head

Command	Args	Options	Description
head	-	angle=middle, async=false	Set head angle

Angle aliases: down, bottom, lower, middle, neutral, center, upper, up, top, or radians (-0.44 to 0.78)

Lights

Command	Args	Options	Description
lights	-	color=blue, colors=...	Set backpack LEDs
ir	-	enable=true	IR head light

Audio

Command	Args	Options	Description
say	<text>	voice=en-us, speed=150, pitch=50, amplitude=100, async=false, effect=normal	Text-to-speech
play-sound	-	name=..., file=..., async=false	Play sound
volume	<level>	-	Set volume (0-65535)

Voice Effects:

• effect=normal (default) - Standard TTS voice
• effect=cozmo - Robot-like voice (higher pitch, faster)

Requires sox: sudo apt install sox

Example:

python3 cozmo_controller.py "say Hello effect=cozmo"

Animation

Command	Args	Options	Description
animate	<name>	async=false, wait=auto	Play animation
anim-group	<group>	async=false, wait=auto	Play animation group
list-anims	-	search=..., duration=false	List animations (offline)
list-groups	-	search=..., duration=false	List groups (offline)
list-sounds	-	search=...	List sounds (offline)

Animation Wait Parameter:

• wait=auto (default) - Automatically compute duration from animation file
• wait=0 - Don't wait, continue immediately (use with sleep)
• wait=5 - Wait for specified seconds

The controller reads the animation file and calculates the exact duration from the keyframes. For example, DanceMambo automatically waits ~15.4 seconds.

Example for custom timing:

anim-group DanceMambo wait=0
sleep 5
lights color=blue

Listing with Durations:

# Show animation durations
python3 cozmo_controller.py "list-anims duration=true"

# Search and show durations
python3 cozmo_controller.py "list-anims search=dance duration=true"

# Show group durations with member counts
python3 cozmo_controller.py "list-groups duration=true"

Display & Camera

Command	Args	Options	Description
screen	<text>	duration=5.0, size=12, x=5, y=10	Display text
camera	-	output=camera_capture.jpg	Capture image
battery	-	mode=icon, duration=5.0	Show battery

Safety

Command	Args	Options	Description
cliff	-	enable=true, reaction=backup	Cliff detection
calibrate	-	head=true, lift=true	Calibrate motors

---

Scripting Language

Variables

set name Cozmo
set count 5
set count=10           # Alternative syntax
say Hello $name        # Use variable
say ${name} is here    # Braces for clarity

For Loops

# Range: 1 to 5
for i in 1..5
  say Count $i
endfor

# With step
for i in 0..10 step 2
  say Even number $i
endfor

# List of values
for color in red green blue
  lights color=$color
endfor

While Loops

set count 5
while $count > 0
  say Countdown $count
  set count $count-1
endwhile
say Blast off!

Conditions

if $count > 5
  say High count
else
  say Low count
endif

# Comparison operators: == != > < >= <=

Subroutines

def greet
  say Hello!
  lights color=blue
enddef

call greet           # Call subroutine

# With arguments (accessible as $1, $2, etc.)
def announce
  say Attention: $1
enddef

call announce Important message

Delays

say Starting
sleep 3              # Wait 3 seconds
say Three seconds later
sleep 0.5            # Wait half a second

Includes

include "common setup.txt"
include "scripts/dance moves.txt"
include "path with spaces/script.txt"

Full Example

# Dance party script
set rounds 3
set name Cozmo

for i in 1..$rounds
  say Round $i for $name
  
  # Animation auto-waits for its duration (~15s for DanceMambo)
  anim-group DanceMambo
  lights color=blue
endfor

say Thanks for watching!

---

Examples

Patrol Script

# patrol.txt
set corners 4
for i in 1..$corners
  move 200
  turn 90
endfor
say Patrol complete

Interactive Story

# story.txt
say Once upon a time
head down
anim-group CodeLabBored

head up
say there was a little robot
lights color=blue

anim-group DanceMambo
say who loved to dance!

Battery Check Routine

# battery_check.txt
battery mode=text
say Battery status displayed on screen

Listing Animations with Durations

# List all animations with their durations
python3 cozmo_controller.py "list-anims duration=true"

# Output:
# Available animations (993 total):
#   Name                                               Duration
#   -------------------------------------------------- ----------
#   anim_bored_01                                          2.1s
#   anim_dancing_mambo_01                                 15.4s
#   anim_greeting_01                                       3.5s

# List groups with average durations
python3 cozmo_controller.py "list-groups duration=true"

# Output:
# Available animation groups (573 total):
#   Name                                               Avg Duration  Members
#   -------------------------------------------------- ---------- --------
#   CodeLabBored                                            3.2s        5
#   DanceMambo                                             15.4s        1
#   FistBumpSuccess                                         2.8s        3

---

Troubleshooting

Connection Issues

Problem: "Connection failed: serial number is 0"

• Ensure Cozmo is powered on (blue light visible)
• Verify you're connected to Cozmo's WiFi network
• Check firewall allows UDP port 5551

Problem: "Connection failed: battery voltage is 0.0V"

• Robot state not received - wait longer or reconnect
• Try pressing Cozmo's button to re-broadcast WiFi

Audio Issues

Problem: "Error generating TTS"

• Install espeak: sudo apt install espeak

Problem: "Error converting with vgmstream-cli"

• Install vgmstream-cli for WEM file support
• Or use external audio files with file= option

Animation Issues

Problem: "Animation not found"

• Use list-anims to find correct names
• Animation groups: list-groups

Problem: Sound doesn't play during animation

• Cozmo has a single audio channel
• Animations with embedded audio will interrupt custom sounds
• Use async=true and ensure animations don't have sound

Movement Issues

Problem: Lift position control doesn't work

• This is a firmware limitation
• Use speed control with lift <speed> duration=X

Problem: Robot doesn't stop at table edge

• Cliff detection must be enabled: cliff enable=true
• Default reaction is to backup

---

Known Limitations

1. Lift Position Control: Firmware doesn't respond to position commands - use speed control
2. Audio Channel: Single channel - animations with sound will interrupt
3. Procedural Face: Must be disabled for custom screen content
4. Async Audio: Robot must stay connected until audio transmits
5. Cubes: I need to change batteries, so far I have not worked on cubes at all.

---

Credits

• Anki / Digital Dream Labs - Cozmo robot
• zayfod - Original pycozmo library
• vgmstream - WEM audio decoding
• ffmpeg - Audio conversion
• sox - Voice effects processing

---