feat: Implement voice-controlled movement generation for Reachy Mini with real-time audio processing, new tests, and documentation.

.gitignore

@@ -3,4 +3,6 @@ __pycache__/
 
 venv
 .env
-.DS_Store
+.DS_Store
+
+.pytest_cache

README.md

@@ -1,11 +1,62 @@
----
-title: Reachy Mini Danceml
-emoji: 👋
-colorFrom: red
-colorTo: blue
-sdk: static
-pinned: false
-short_description: Write your description here
-tags:
- - reachy_mini
----
+# Reachy Mini DanceML
+
+👋 Voice-controlled movement SDK for Reachy Mini robot.
+
+## Features
+
+- 🎤 **Voice Control**: Natural language commands via OpenAI Realtime API
+- 🧠 **Hybrid Architecture**: Intelligently switches between library retrieval and generative AI
+- 📚 **Move Library**: Zero-latency access to 100+ dances and emotional expressions
+- 🎯 **Keyframe Animations**: Smooth movements with cubic spline interpolation  
+- 🤖 **AI Agent Integration**: Function-calling schemas for LLM agents
+
+## Quick Start
+
+```bash
+# Install dependencies
+pip install -e .
+
+# Set your OpenAI API key
+export OPENAI_API_KEY="your-key-here"
+
+# Run the app
+python -m reachy_mini_danceml.main
+```
+
+## Documentation
+
+📖 See [SDK Documentation](docs/SDK_DOCUMENTATION.md) for:
+- Dataset format (HuggingFace dance library)
+- Core classes (KeyFrame, GeneratedMove, MovementGenerator)
+- Movement tool schemas for AI agents
+- Usage examples
+
+## HuggingFace Dataset
+
+Load pre-recorded dance moves:
+
+```python
+from datasets import load_dataset
+
+ds = load_dataset("pollen-robotics/reachy-mini-dances-library")
+print(ds['train'][0]['description'])  # "A sharp, forward, chicken-like pecking motion."
+```
+
+## Example: Create a Wave Animation
+
+```python
+from reachy_mini_danceml.movement_tools import KeyFrame
+from reachy_mini_danceml.movement_generator import GeneratedMove
+
+keyframes = [
+    KeyFrame(t=0.0, antennas=(0, 0)),
+    KeyFrame(t=0.3, antennas=(30, -30)),
+    KeyFrame(t=0.6, antennas=(-30, 30)),
+    KeyFrame(t=1.0, antennas=(0, 0)),
+]
+move = GeneratedMove(keyframes)
+```
+
+## Tags
+
+- reachy_mini

docs/ARCHITECTURE.md

@@ -0,0 +1,133 @@
+# Reachy Mini DanceML Architecture
+
+## System Architecture
+
+```mermaid
+flowchart TB
+    subgraph Input["🎤 Input Layer"]
+        USER["User Voice"]
+        MIC["Browser Microphone<br/>(Laptop/Mobile)"]
+    end
+
+    subgraph Streaming["⚡ Streaming Layer"]
+        GRADIO["Gradio UI<br/>:8042"]
+        FASTRTC["FastRTC<br/>Audio Stream"]
+    end
+
+    subgraph AI["🧠 AI Layer (OpenAI Realtime)"]
+        ASR["Speech-to-Text"]
+        REASON["gpt-realtime<br/>Reasoning"]
+        TTS["Text-to-Speech"]
+    end
+
+    subgraph ToolRouter["🔧 Tool Router"]
+        DISPATCH["Tool Dispatcher"]
+    end
+
+    subgraph CoreTools["Core Tools"]
+        GOTO["goto_pose"]
+        STOP["stop_movement"]
+    end
+
+    subgraph RetrievalTools["Retrieval Tools"]
+        SEARCH["search_moves"]
+        PLAY["play_move"]
+    end
+
+    subgraph GenerativeTools["Generative Tools"]
+        GUIDE["get_choreography_guide"]
+        CREATE["create_sequence"]
+    end
+
+    subgraph Backend["📦 Backend"]
+        LIBRARY["MoveLibrary<br/>(101 moves)"]
+        GENERATOR["MovementGenerator"]
+        DOCS["CHOREOGRAPHY_GUIDE.md"]
+    end
+
+    subgraph Robot["🤖 Reachy Mini"]
+        HEAD["Head<br/>roll/pitch/yaw"]
+        ANTENNAS["Antennas<br/>left/right"]
+    end
+
+    subgraph Output["🔊 Output Layer"]
+        SPEAKER["Speaker"]
+    end
+
+    %% Input flow
+    USER --> MIC --> GRADIO --> FASTRTC --> ASR
+
+    %% AI reasoning
+    ASR --> REASON
+    REASON --> TTS --> FASTRTC --> GRADIO --> SPEAKER
+
+    %% Tool calls
+    REASON -->|"function_call"| DISPATCH
+    DISPATCH --> CoreTools
+    DISPATCH --> RetrievalTools
+    DISPATCH --> GenerativeTools
+
+    %% Tool to backend
+    GOTO --> GENERATOR
+    STOP --> GENERATOR
+    SEARCH --> LIBRARY
+    PLAY --> LIBRARY
+    PLAY --> GENERATOR
+    GUIDE --> DOCS
+    CREATE --> GENERATOR
+
+    %% Backend to robot
+    GENERATOR --> HEAD
+    GENERATOR --> ANTENNAS
+
+    %% Results back
+    CoreTools -.->|"result"| DISPATCH
+    RetrievalTools -.->|"result"| DISPATCH
+    GenerativeTools -.->|"result"| DISPATCH
+    DISPATCH -.-> REASON
+```
+
+---
+
+## Tool Selection Flow
+
+```mermaid
+flowchart TD
+    START(("🎤 User<br/>Request")) --> INTENT{"Classify<br/>Intent"}
+
+    INTENT -->|"look left<br/>tilt head"| SIMPLE["🎯 SIMPLE"]
+    INTENT -->|"stop<br/>freeze"| EMERGENCY["🛑 STOP"]
+    INTENT -->|"show happy<br/>do a dance"| EMOTION["🎭 EMOTION"]
+    INTENT -->|"act like...<br/>create new"| CREATIVE["✨ CREATIVE"]
+
+    SIMPLE --> GOTO_POSE["goto_pose()"]
+    EMERGENCY --> STOP_MOVE["stop_movement()"]
+    
+    EMOTION --> SEARCH_LIB["search_moves()"]
+    SEARCH_LIB --> FOUND{"Results?"}
+    FOUND -->|"Yes"| PLAY_MOVE["play_move()"]
+    FOUND -->|"No"| LOAD_GUIDE
+    
+    CREATIVE --> LOAD_GUIDE["get_choreography_guide()"]
+    LOAD_GUIDE --> CREATE_SEQ["create_sequence()"]
+
+    GOTO_POSE --> EXECUTE["⚡ Execute"]
+    STOP_MOVE --> EXECUTE
+    PLAY_MOVE --> EXECUTE
+    CREATE_SEQ --> EXECUTE
+
+    EXECUTE --> ROBOT(("🤖 Robot<br/>Moves"))
+```
+
+---
+
+## Component Summary
+
+| Layer | Component | Purpose |
+|-------|-----------|---------|
+| **Input** | Gradio + FastRTC | Audio streaming |
+| **AI** | OpenAI Realtime | Speech, reasoning, TTS |
+| **Tools** | 6 functions | Intent execution |
+| **Backend** | MoveLibrary | 101 pre-built moves |
+| **Backend** | MovementGenerator | Keyframe interpolation |
+| **Output** | Reachy Mini SDK | Motor control |

docs/CHOREOGRAPHY_GUIDE.md

@@ -0,0 +1,64 @@
+# Reachy Mini Choreography Guide
+
+This guide provides the physics rules and artistic principles for creating custom movements for Reachy Mini. Use these rules when `create_sequence` is required.
+
+## 1. Physics Constraints
+
+### Head Movement
+*   **Roll** (Tilt L/R): -30° to +30°
+    *   *Positive* = Tilt Right
+    *   *Expressive*: Use roll for "curiosity" (tilt head) or "cuteness".
+*   **Pitch** (Up/Down): -30° to +30°
+    *   *Positive* = Look Up
+    *   *Expressive*: Pitch down for sadness/shame, up for pride/joy.
+*   **Yaw** (Turn L/R): -45° to +45°
+    *   *Positive* = Look Left
+    *   *Expressive*: Shake head for "no", scan room for "searching".
+
+### Antennas
+*   **Range**: -60° (Back) to +60° (Front)
+    *   *0°* = Vertical/Neutral
+*   **Expressiveness**:
+    *   *Forward (+)*: Alert, interested, angry (if rigid).
+    *   *Backward (-)*: Sad, scared, aerodynamic.
+    *   *Asymmetric*: Confused, playful (one up, one down).
+
+## 2. Timing & Smoothness
+
+*   **Minimum Duration**: 0.5s between keyframes for large moves (>20°).
+    *   *Fast*: 0.2s-0.3s (Small twitches, excitement).
+    *   *Normal*: 0.5s-1.0s (Looking around).
+    *   *Slow*: 1.5s+ (Breathing, sad movements).
+*   **Interpolation**: The system uses Cubic Spline.
+    *   *Avoid*: Two identical keyframes too close together (creates pauses).
+    *   *Do*: Use evenly spaced keyframes for smooth arcs.
+
+## 3. Choreography Patterns
+
+### "The Breath" (Idle/Calm)
+Gentle, slow pitch movement accompanied by slight antenna swaying.
+```python
+{"t": 0.0, "head": {"pitch": 0}, "antennas": [0, 0]}
+{"t": 2.0, "head": {"pitch": 5}, "antennas": [-10, -10]} # Inhale/Up
+{"t": 4.0, "head": {"pitch": 0}, "antennas": [0, 0]}     # Exhale
+```
+
+### "The Scan" (Searching)
+Head stays level (pitch 0), yaw sweeps, antennas alert (forward).
+```python
+{"t": 0.0, "head": {"yaw": -30}, "antennas": [30, 30]}
+{"t": 1.0, "head": {"yaw": 30}, "antennas": [30, 30]}
+```
+
+### "The Jiggle" (Excitement/Laugh)
+Rapid, small alternating rolls or antenna movements.
+```python
+{"t": 0.0, "head": {"roll": 0}, "antennas": [0, 0]}
+{"t": 0.1, "head": {"roll": 5}, "antennas": [20, -20]}
+{"t": 0.2, "head": {"roll": -5}, "antennas": [-20, 20]}
+{"t": 0.3, "head": {"roll": 0}, "antennas": [0, 0]}
+```
+
+## 4. Safety
+*   **Collision**: Antennas can hit the head if pitched back too far while antennas are forward.
+*   **Rule**: If Pitch < -20 (looking down), keep Antennas < 20.

docs/SDK_DOCUMENTATION.md

@@ -0,0 +1,485 @@
+# Reachy Mini DanceML SDK Documentation
+
+This documentation covers the SDK methods for controlling Reachy Mini movements and the data format for dance sequences.
+
+## Table of Contents
+- [Overview](#overview)
+- [Dataset Format](#dataset-format)
+- [Core Classes](#core-classes)
+- [Movement Tools](#movement-tools)
+- [Usage Examples](#usage-examples)
+
+---
+
+## Overview
+
+The Reachy Mini DanceML SDK enables:
+- **Voice-controlled movements** via OpenAI Realtime API
+- **Keyframe-based animations** with cubic spline interpolation
+- **Simple pose commands** for direct head positioning
+
+---
+
+## Available Datasets
+
+Pollen Robotics provides two HuggingFace datasets with pre-recorded movements:
+
+| Dataset | Records | Description |
+|---------|---------|-------------|
+| `pollen-robotics/reachy-mini-dances-library` | 20 | Dance moves (pecking, bobbing, swaying) |
+| `pollen-robotics/reachy-mini-emotions-library` | 81 | Emotional expressions (wonder, fear, joy, etc.) |
+
+Both datasets share the same schema and can be used interchangeably with this SDK.
+
+### Dataset Schema
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `description` | `string` | Human-readable description of the movement |
+| `time` | `List[float]` | Timestamps in seconds from animation start |
+| `set_target_data` | `List[TargetData]` | Array of pose targets at each timestamp |
+
+### TargetData Structure
+
+Each element in `set_target_data` contains:
+
+```python
+{
+    "head": [[4x4 homogeneous transformation matrix]],
+    "antennas": [left_angle, right_angle],  # in radians
+    "body_yaw": 0.0,                         # body rotation (typically 0)
+    "check_collision": false                 # collision check flag
+}
+```
+
+### Head Pose Matrix
+
+The `head` field is a 4x4 homogeneous transformation matrix representing the head orientation:
+
+```
+[[r11, r12, r13, tx],
+ [r21, r22, r23, ty],
+ [r31, r32, r33, tz],
+ [0,   0,   0,   1 ]]
+```
+
+Where:
+- The 3x3 upper-left submatrix encodes rotation (roll, pitch, yaw)
+- The last column `[tx, ty, tz, 1]` encodes translation
+
+### Loading the Datasets
+
+```python
+from datasets import load_dataset
+
+# Dance moves (requires HuggingFace login)
+dances = load_dataset("pollen-robotics/reachy-mini-dances-library")
+
+# Emotions library (requires HuggingFace login)
+emotions = load_dataset("pollen-robotics/reachy-mini-emotions-library")
+
+# Access a dance move
+dance = dances['train'][0]
+print(f"Description: {dance['description']}")
+# Output: "A sharp, forward, chicken-like pecking motion."
+
+# Access an emotion
+emotion = emotions['train'][0]
+print(f"Description: {emotion['description']}")
+# Output: "When you discover something extraordinary..."
+
+# Both have the same structure
+print(f"Duration: {emotion['time'][-1]} seconds")
+print(f"Frames: {len(emotion['time'])}")
+```
+
+### Example Emotion Descriptions
+
+The emotions library includes expressive movements such as:
+- **Wonder**: "When you discover something extraordinary"
+- **Fear**: "You look around without really knowing where to look"
+- **Joy**: Celebratory movements
+- **Surprise**: Reactive startle responses
+- **Curiosity**: Investigative head tilts
+
+---
+
+## Core Classes
+
+### KeyFrame
+
+A single keyframe in an animation sequence.
+
+```python
+from reachy_mini_danceml.movement_tools import KeyFrame
+
+@dataclass
+class KeyFrame:
+    t: float                      # Time in seconds from animation start
+    head: dict                    # {"roll": 0, "pitch": 0, "yaw": 0} in degrees
+    antennas: Tuple[float, float] # (left, right) antenna angles in degrees
+```
+
+#### Methods
+
+| Method | Description |
+|--------|-------------|
+| `KeyFrame.from_dict(data)` | Create KeyFrame from a dictionary |
+
+#### Example
+
+```python
+# Create keyframes for a nodding animation
+keyframes = [
+    KeyFrame(t=0.0, head={"roll": 0, "pitch": 0, "yaw": 0}, antennas=(0, 0)),
+    KeyFrame(t=0.3, head={"roll": 0, "pitch": -15, "yaw": 0}, antennas=(10, 10)),
+    KeyFrame(t=0.6, head={"roll": 0, "pitch": 10, "yaw": 0}, antennas=(-5, -5)),
+    KeyFrame(t=1.0, head={"roll": 0, "pitch": 0, "yaw": 0}, antennas=(0, 0)),
+]
+```
+
+---
+
+### GeneratedMove
+
+A Move generated from keyframes with cubic spline interpolation.
+
+```python
+from reachy_mini_danceml.movement_generator import GeneratedMove
+
+class GeneratedMove(Move):
+    def __init__(self, keyframes: List[KeyFrame])
+    
+    @property
+    def duration(self) -> float
+    
+    def evaluate(self, t: float) -> Tuple[np.ndarray, np.ndarray, float]
+```
+
+#### Properties
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `duration` | `float` | Total animation duration in seconds |
+
+#### Methods
+
+| Method | Parameters | Returns | Description |
+|--------|------------|---------|-------------|
+| `evaluate(t)` | `t: float` (time in seconds) | `(head_pose, antennas, body_yaw)` | Interpolate pose at time t |
+
+#### Return Values from `evaluate()`
+
+- `head_pose`: 4x4 numpy array (homogeneous transformation matrix)
+- `antennas`: numpy array `[left, right]` in radians
+- `body_yaw`: float (always 0.0)
+
+#### Example
+
+```python
+from reachy_mini_danceml.movement_generator import GeneratedMove
+from reachy_mini_danceml.movement_tools import KeyFrame
+
+keyframes = [
+    KeyFrame(t=0.0, head={"yaw": 0}),
+    KeyFrame(t=1.0, head={"yaw": 30}),
+    KeyFrame(t=2.0, head={"yaw": 0}),
+]
+
+move = GeneratedMove(keyframes)
+print(f"Duration: {move.duration} seconds")
+
+# Get pose at 0.5 seconds
+head, antennas, body_yaw = move.evaluate(0.5)
+```
+
+---
+
+### MoveLibrary
+
+Manages loading and indexing of dance and emotion datasets.
+
+```python
+from reachy_mini_danceml.dataset_loader import MoveLibrary
+
+library = MoveLibrary()
+library.load()
+
+# Search
+results = library.search_moves("happy")
+
+# Get Record
+record = library.get_move("joy_jump")
+```
+
+### MovementGenerator
+
+Generates and executes movements on Reachy Mini.
+
+```python
+from reachy_mini_danceml.movement_generator import MovementGenerator
+
+class MovementGenerator:
+    def __init__(self, reachy: ReachyMini)
+    
+    def create_from_keyframes(self, keyframes) -> GeneratedMove
+    
+    async def goto_pose(self, roll=0, pitch=0, yaw=0, duration=0.5) -> None
+    
+    async def play_move(self, move: Move) -> None
+    
+    async def stop(self) -> None
+```
+
+#### Methods
+
+| Method | Parameters | Description |
+|--------|------------|-------------|
+| `create_from_keyframes(keyframes)` | `List[KeyFrame]` or `List[dict]` | Create a GeneratedMove from keyframes |
+| `goto_pose(roll, pitch, yaw, duration)` | Angles in degrees, duration in seconds | Move head to specific pose |
+| `play_move(move)` | `Move` object | Play an animation asynchronously |
+| `stop()` | None | Stop current movement, return to neutral |
+
+#### Angle Limits
+
+| Parameter | Range | Direction |
+|-----------|-------|-----------|
+| `roll` | -30° to 30° | Positive = tilt right |
+| `pitch` | -30° to 30° | Positive = look up |
+| `yaw` | -45° to 45° | Positive = look left |
+| `antennas` | -60° to 60° | Each antenna independently |
+
+#### Example
+
+```python
+from reachy_mini import ReachyMini
+from reachy_mini_danceml.movement_generator import MovementGenerator
+
+async def demo(reachy: ReachyMini):
+    generator = MovementGenerator(reachy)
+    
+    # Simple pose
+    await generator.goto_pose(roll=0, pitch=10, yaw=-20, duration=0.5)
+    
+    # Keyframe animation
+    keyframes = [
+        {"t": 0.0, "head": {"yaw": 0}, "antennas": [0, 0]},
+        {"t": 0.5, "head": {"yaw": 30}, "antennas": [20, -20]},
+        {"t": 1.0, "head": {"yaw": 0}, "antennas": [0, 0]},
+    ]
+    move = generator.create_from_keyframes(keyframes)
+    await generator.play_move(move)
+```
+
+---
+
+## Movement Tools
+
+These are OpenAI function-calling tool schemas for voice control integration.
+
+### PLAY_MOVE_TOOL
+
+Play a pre-defined movement from the library by its name/ID.
+
+```python
+{
+    "type": "function",
+    "name": "play_move",
+    "description": "Play a pre-defined movement from the library by its name (e.g., 'joy', 'fear', 'chicken_dance'). Prefer this over creating sequences manually.",
+    "parameters": {
+        "properties": {
+            "name": {"type": "string", "description": "Name or ID of the movement"}
+        },
+        "required": ["name"]
+    }
+}
+```
+
+### SEARCH_MOVES_TOOL
+
+Search the library for available movements.
+
+```python
+{
+    "type": "function",
+    "name": "search_moves",
+    "description": "Search the movement library for available expressions or dances.",
+    "parameters": {
+        "properties": {
+            "query": {"type": "string", "description": "Keywords to search for"}
+        },
+        "required": ["query"]
+    }
+}
+```
+
+### GET_CHOREOGRAPHY_GUIDE_TOOL
+
+Retrieve physics rules and examples for custom generation.
+
+```python
+{
+    "type": "function",
+    "name": "get_choreography_guide",
+    "description": "Read the choreography guide to learn how to create safe and expressive custom movements. Call this BEFORE using create_sequence for new moves."
+}
+```
+
+### GOTO_POSE_TOOL
+
+Move the robot's head to a specific pose.
+
+```python
+{
+    "type": "function",
+    "name": "goto_pose",
+    "parameters": {
+        "properties": {
+            "roll": {"type": "number", "description": "Roll angle (-30 to 30°)"},
+            "pitch": {"type": "number", "description": "Pitch angle (-30 to 30°)"},
+            "yaw": {"type": "number", "description": "Yaw angle (-45 to 45°)"},
+            "duration": {"type": "number", "description": "Duration in seconds"}
+        }
+    }
+}
+```
+
+### CREATE_SEQUENCE_TOOL
+
+Create and play an animated movement sequence from keyframes.
+
+```python
+{
+    "type": "function",
+    "name": "create_sequence",
+    "parameters": {
+        "properties": {
+            "keyframes": {
+                "type": "array",
+                "items": {
+                    "type": "object",
+                    "properties": {
+                        "t": {"type": "number", "description": "Time in seconds"},
+                        "head": {
+                            "properties": {
+                                "roll": {"type": "number"},
+                                "pitch": {"type": "number"},
+                                "yaw": {"type": "number"}
+                            }
+                        },
+                        "antennas": {
+                            "type": "array",
+                            "items": {"type": "number"},
+                            "description": "[left, right] in degrees (-60 to 60)"
+                        }
+                    },
+                    "required": ["t"]
+                }
+            }
+        },
+        "required": ["keyframes"]
+    }
+}
+```
+
+### STOP_MOVEMENT_TOOL
+
+Stop any currently playing movement and return to neutral position.
+
+```python
+{
+    "type": "function",
+    "name": "stop_movement",
+    "description": "Stop current movement and return to neutral"
+}
+```
+
+---
+
+## Hybrid AI Workflow
+
+The SDK is designed for a **Hybrid Generative/Retrieval** architecture to optimize context usage.
+
+### Recommended Agent Logic
+
+1.  **Retrieval First**: Always try `search_moves(query)` first.
+2.  **Play by Name**: If a match is found, use `play_move(name)`. This uses 0 tokens for movement data.
+3.  **On-Demand Learning**: If no match is found, call `get_choreography_guide()` to load physics rules.
+4.  **Safe Generation**: Finally, use `create_sequence(keyframes)` to generate a custom move using the loaded rules.
+
+---
+
+## Usage Examples
+
+### Example 1: Wave Animation
+
+```python
+wave_keyframes = [
+    {"t": 0.0, "head": {"roll": 0, "yaw": 0}, "antennas": [0, 0]},
+    {"t": 0.3, "head": {"roll": 0, "yaw": 0}, "antennas": [30, -30]},
+    {"t": 0.6, "head": {"roll": 0, "yaw": 0}, "antennas": [-30, 30]},
+    {"t": 0.9, "head": {"roll": 0, "yaw": 0}, "antennas": [30, -30]},
+    {"t": 1.2, "head": {"roll": 0, "yaw": 0}, "antennas": [0, 0]},
+]
+```
+
+### Example 2: Curious Head Tilt
+
+```python
+curious_keyframes = [
+    {"t": 0.0, "head": {"roll": 0, "pitch": 0, "yaw": 0}},
+    {"t": 0.4, "head": {"roll": 15, "pitch": 5, "yaw": 10}},
+    {"t": 1.5, "head": {"roll": 15, "pitch": 5, "yaw": 10}},
+    {"t": 2.0, "head": {"roll": 0, "pitch": 0, "yaw": 0}},
+]
+```
+
+### Example 3: Excited Celebration
+
+```python
+excited_keyframes = [
+    {"t": 0.0, "head": {"pitch": 0}, "antennas": [0, 0]},
+    {"t": 0.2, "head": {"pitch": -10}, "antennas": [40, 40]},
+    {"t": 0.4, "head": {"pitch": 5}, "antennas": [-20, -20]},
+    {"t": 0.6, "head": {"pitch": -10}, "antennas": [40, 40]},
+    {"t": 0.8, "head": {"pitch": 5}, "antennas": [-20, -20]},
+    {"t": 1.0, "head": {"pitch": 0}, "antennas": [0, 0]},
+]
+```
+
+---
+
+## AI Agent Output Format
+
+When building an AI agent to generate movements for Reachy Mini, the output should match this format:
+
+### For Simple Poses
+
+```json
+{
+    "function": "goto_pose",
+    "arguments": {
+        "roll": 0,
+        "pitch": 10,
+        "yaw": -20,
+        "duration": 0.5
+    }
+}
+```
+
+### For Animated Sequences
+
+```json
+{
+    "function": "create_sequence",
+    "arguments": {
+        "keyframes": [
+            {"t": 0.0, "head": {"roll": 0, "pitch": 0, "yaw": 0}, "antennas": [0, 0]},
+            {"t": 0.5, "head": {"roll": 10, "pitch": -5, "yaw": 20}, "antennas": [15, -15]},
+            {"t": 1.0, "head": {"roll": 0, "pitch": 0, "yaw": 0}, "antennas": [0, 0]}
+        ]
+    }
+}
+```
+
+This format allows seamless integration with the OpenAI Realtime API for voice-controlled robot movements.

docs/plans/voice-controlled-movement.md

@@ -0,0 +1,692 @@
+# Voice-Controlled Movement App for Reachy Mini
+
+> [!IMPORTANT]
+> This plan has been implemented. For the latest architecture and tool definitions, please refer to [SDK_DOCUMENTATION.md](../SDK_DOCUMENTATION.md).
+
+
+
+A standalone app using **OpenAI Realtime API** and **fastrtc** for voice conversations with Reachy Mini. Speak naturally and Reachy responds with voice while executing movements.
+
+## Design Decisions (Approved)
+
+- **UI**: Web-based (Gradio) following project's existing CSS style
+- **Movement Complexity**: Advanced (full procedural generation with timing/interpolation)
+- **Voice API**: OpenAI Realtime API (bidirectional voice)
+- **Streaming**: fastrtc for low-latency audio
+- **Agent Framework**: LangGraph for stateful movement tool calling
+- **Integration**: Independent, can borrow patterns from conversation app
+
+---
+
+## Architecture
+
+```mermaid
+flowchart TB
+    subgraph User["User"]
+        MIC[🎤 Microphone]
+        SPK[🔊 Speaker]
+    end
+    
+    subgraph UI["Gradio Web UI"]
+        AUDIO[Audio Stream]
+        TRANS[Transcript Display]
+        STATUS[Status Display]
+    end
+    
+    subgraph FRTC["FastRTC Layer"]
+        STREAM[Stream Handler]
+        VAD[Voice Activity Detection]
+    end
+    
+    subgraph OPENAI["OpenAI Realtime API"]
+        WS[WebSocket Connection]
+        ASR[Speech Recognition]
+        LLM[GPT-4.1 Realtime]
+        TTS[Text-to-Speech]
+        TOOLS[Function Calling]
+    end
+    
+    subgraph Movement["Movement System"]
+        MG[Movement Generator]
+        SDK[Reachy Mini SDK]
+    end
+    
+    MIC --> AUDIO
+    AUDIO --> STREAM
+    STREAM --> VAD
+    VAD --> WS
+    WS --> ASR
+    ASR --> LLM
+    LLM --> TOOLS
+    TOOLS --> MG
+    MG --> SDK
+    LLM --> TTS
+    TTS --> WS
+    WS --> STREAM
+    STREAM --> AUDIO
+    AUDIO --> SPK
+    
+    SDK --> ROBOT[🤖 Reachy Mini]
+```
+
+---
+
+## OpenAI Realtime API Overview
+
+The Realtime API provides:
+- **WebSocket connection** for bidirectional streaming
+- **Built-in speech recognition** (input audio → text)
+- **Built-in TTS** (text → output audio)
+- **Function calling** during conversation
+- **~200ms latency** end-to-end
+
+### Event Flow
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant FastRTC
+    participant OpenAI
+    participant Tools
+    participant Reachy
+    
+    User->>FastRTC: Speaks "Wave hello"
+    FastRTC->>OpenAI: Audio chunks
+    OpenAI->>OpenAI: Transcribe
+    OpenAI->>OpenAI: Process with LLM
+    OpenAI->>Tools: function_call: create_sequence
+    Tools->>Reachy: Execute wave animation
+    Tools-->>OpenAI: function_result: success
+    OpenAI->>OpenAI: Generate response
+    OpenAI->>FastRTC: Audio response
+    FastRTC->>User: "Okay, waving hello!"
+```
+
+---
+
+## Proposed Changes
+
+### Component Structure
+
+```
+reachy_mini_danceml/
+├── reachy_mini_danceml/
+│   ├── main.py                  # [MODIFY] Add Gradio voice UI
+│   ├── realtime_handler.py      # [NEW] OpenAI Realtime API + fastrtc
+│   ├── movement_tools.py        # [NEW] Function calling tools
+│   ├── movement_generator.py    # [NEW] Keyframe interpolation
+│   └── static/                  # (existing static files)
+├── tests/
+│   └── test_movement_generator.py  # [NEW] Unit tests
+└── pyproject.toml               # [MODIFY] Add dependencies
+```
+
+---
+
+### [NEW] `reachy_mini_danceml/realtime_handler.py`
+
+OpenAI Realtime API integration with fastrtc.
+
+```python
+import asyncio
+import json
+from typing import Optional, Callable
+from fastrtc import Stream, ReplyOnPause
+from openai import AsyncOpenAI
+import numpy as np
+
+# Tool definitions for movement control
+MOVEMENT_TOOLS = [
+    {
+        "type": "function",
+        "name": "goto_pose",
+        "description": "Move the robot's head to a specific pose",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "roll": {"type": "number", "description": "Roll angle (-30 to 30 degrees)"},
+                "pitch": {"type": "number", "description": "Pitch angle (-30 to 30 degrees)"},
+                "yaw": {"type": "number", "description": "Yaw angle (-45 to 45 degrees)"},
+                "duration": {"type": "number", "description": "Duration in seconds", "default": 0.5}
+            }
+        }
+    },
+    {
+        "type": "function",
+        "name": "create_sequence",
+        "description": "Create and play an animated movement sequence",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "keyframes": {
+                    "type": "array",
+                    "items": {
+                        "type": "object",
+                        "properties": {
+                            "t": {"type": "number", "description": "Time in seconds"},
+                            "head": {"type": "object", "properties": {
+                                "roll": {"type": "number"},
+                                "pitch": {"type": "number"},
+                                "yaw": {"type": "number"}
+                            }},
+                            "antennas": {"type": "array", "items": {"type": "number"}}
+                        }
+                    }
+                }
+            },
+            "required": ["keyframes"]
+        }
+    },
+    {
+        "type": "function",
+        "name": "stop_movement",
+        "description": "Stop any currently playing movement"
+    }
+]
+
+SYSTEM_INSTRUCTIONS = """You are Reachy, a friendly robot companion. You control a robot with a head and antennas.
+
+Available movements:
+- Head: roll (-30 to 30°), pitch (-30 to 30°), yaw (-45 to 45°)
+- Antennas: left/right (-60 to 60°)
+
+Use goto_pose for simple movements ("look left", "look up").
+Use create_sequence for animations ("wave hello", "nod excitedly", "dance").
+
+Be conversational and friendly. Confirm what you're doing.
+Examples:
+- "Look left" → goto_pose with yaw=-30
+- "Wave hello" → create_sequence with antenna keyframes
+- "Nod yes" → create_sequence with pitch keyframes"""
+
+class RealtimeHandler:
+    def __init__(self, openai_key: str, movement_generator):
+        self.client = AsyncOpenAI(api_key=openai_key)
+        self.generator = movement_generator
+        self.connection = None
+        
+    async def handle_tool_call(self, name: str, arguments: dict) -> str:
+        """Execute a tool call and return the result."""
+        if name == "goto_pose":
+            await self.generator.goto_pose(**arguments)
+            return f"Moved to pose: roll={arguments.get('roll', 0)}°, pitch={arguments.get('pitch', 0)}°, yaw={arguments.get('yaw', 0)}°"
+        elif name == "create_sequence":
+            keyframes = arguments.get("keyframes", [])
+            move = self.generator.create_from_keyframes(keyframes)
+            await self.generator.play_move(move)
+            return f"Played sequence with {len(keyframes)} keyframes"
+        elif name == "stop_movement":
+            await self.generator.stop()
+            return "Movement stopped"
+        return "Unknown tool"
+        
+    def create_stream(self):
+        """Create a fastrtc Stream for audio processing."""
+        async def audio_handler(audio_input):
+            """Process incoming audio through OpenAI Realtime API."""
+            async with self.client.beta.realtime.connect(
+                model="gpt-realtime"
+            ) as conn:
+                # Configure session with tools
+                await conn.session.update(session={
+                    "modalities": ["text", "audio"],
+                    "instructions": SYSTEM_INSTRUCTIONS,
+                    "tools": MOVEMENT_TOOLS,
+                    "input_audio_format": "pcm16",
+                    "output_audio_format": "pcm16",
+                })
+                
+                # Send audio input
+                await conn.input_audio_buffer.append(audio=audio_input)
+                await conn.input_audio_buffer.commit()
+                await conn.response.create()
+                
+                # Process response events
+                async for event in conn:
+                    if event.type == "response.audio.delta":
+                        yield event.delta  # Audio output
+                    elif event.type == "response.function_call_arguments.done":
+                        # Execute tool call
+                        result = await self.handle_tool_call(
+                            event.name, 
+                            json.loads(event.arguments)
+                        )
+                        # Send result back
+                        await conn.conversation.item.create(item={
+                            "type": "function_call_output",
+                            "call_id": event.call_id,
+                            "output": result
+                        })
+                        await conn.response.create()
+        
+        return Stream(
+            handler=ReplyOnPause(audio_handler),
+            modality="audio",
+            mode="send-receive"
+        )
+```
+
+---
+
+### [NEW] `reachy_mini_danceml/movement_tools.py`
+
+Movement tool definitions (shared between Realtime API and any non-realtime fallback).
+
+```python
+from dataclasses import dataclass
+from typing import List, Tuple, Optional
+
+@dataclass
+class KeyFrame:
+    t: float
+    head: dict  # {"roll": 0, "pitch": 0, "yaw": 0}
+    antennas: Tuple[float, float] = (0.0, 0.0)
+    
+    @classmethod
+    def from_dict(cls, data: dict) -> "KeyFrame":
+        return cls(
+            t=data.get("t", 0),
+            head=data.get("head", {}),
+            antennas=tuple(data.get("antennas", [0, 0]))
+        )
+
+# Tool schemas for OpenAI function calling
+GOTO_POSE_SCHEMA = {
+    "type": "function",
+    "name": "goto_pose",
+    "description": "Move the robot's head to a specific pose",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "roll": {"type": "number", "description": "Roll angle (-30 to 30 degrees)", "default": 0},
+            "pitch": {"type": "number", "description": "Pitch angle (-30 to 30 degrees)", "default": 0},
+            "yaw": {"type": "number", "description": "Yaw angle (-45 to 45 degrees)", "default": 0},
+            "duration": {"type": "number", "description": "Duration in seconds", "default": 0.5}
+        }
+    }
+}
+
+CREATE_SEQUENCE_SCHEMA = {
+    "type": "function",
+    "name": "create_sequence",
+    "description": "Create and play an animated movement sequence from keyframes",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "keyframes": {
+                "type": "array",
+                "description": "List of keyframes defining the animation",
+                "items": {
+                    "type": "object",
+                    "properties": {
+                        "t": {"type": "number", "description": "Time in seconds"},
+                        "head": {
+                            "type": "object",
+                            "properties": {
+                                "roll": {"type": "number"},
+                                "pitch": {"type": "number"},
+                                "yaw": {"type": "number"}
+                            }
+                        },
+                        "antennas": {
+                            "type": "array",
+                            "items": {"type": "number"},
+                            "description": "[left, right] degrees"
+                        }
+                    }
+                }
+            }
+        },
+        "required": ["keyframes"]
+    }
+}
+
+STOP_MOVEMENT_SCHEMA = {
+    "type": "function",
+    "name": "stop_movement",
+    "description": "Stop any currently playing movement"
+}
+
+ALL_TOOLS = [GOTO_POSE_SCHEMA, CREATE_SEQUENCE_SCHEMA, STOP_MOVEMENT_SCHEMA]
+```
+
+---
+
+### [NEW] `reachy_mini_danceml/movement_generator.py`
+
+Keyframe-to-Move conversion with interpolation (same as before).
+
+```python
+from dataclasses import dataclass
+from typing import List, Tuple, Optional, Union
+import numpy as np
+from scipy.interpolate import CubicSpline
+from reachy_mini import ReachyMini
+from reachy_mini.motion.move import Move
+from reachy_mini.utils import create_head_pose
+from .movement_tools import KeyFrame
+
+class GeneratedMove(Move):
+    """A Move generated from keyframes with cubic spline interpolation."""
+    
+    def __init__(self, keyframes: List[KeyFrame]):
+        if len(keyframes) < 2:
+            raise ValueError("Need at least 2 keyframes")
+            
+        times = [kf.t for kf in keyframes]
+        self.roll_spline = CubicSpline(times, [kf.head.get("roll", 0) for kf in keyframes])
+        self.pitch_spline = CubicSpline(times, [kf.head.get("pitch", 0) for kf in keyframes])
+        self.yaw_spline = CubicSpline(times, [kf.head.get("yaw", 0) for kf in keyframes])
+        self.left_ant_spline = CubicSpline(times, [kf.antennas[0] for kf in keyframes])
+        self.right_ant_spline = CubicSpline(times, [kf.antennas[1] for kf in keyframes])
+        self._duration = max(times)
+        self._keyframes = keyframes
+        
+    @property
+    def duration(self) -> float:
+        return self._duration
+        
+    def evaluate(self, t: float):
+        """Evaluate the move at time t. Returns (head_pose, antennas, body_yaw)."""
+        t = min(t, self._duration)  # Clamp to duration
+        
+        roll = float(self.roll_spline(t))
+        pitch = float(self.pitch_spline(t))
+        yaw = float(self.yaw_spline(t))
+        head = create_head_pose(roll=roll, pitch=pitch, yaw=yaw, degrees=True)
+        
+        left_ant = np.deg2rad(float(self.left_ant_spline(t)))
+        right_ant = np.deg2rad(float(self.right_ant_spline(t)))
+        antennas = np.array([left_ant, right_ant])
+        
+        return head, antennas, 0.0
+
+class MovementGenerator:
+    def __init__(self, reachy: ReachyMini):
+        self.reachy = reachy
+        self.last_movement: Optional[GeneratedMove] = None
+        
+    def create_from_keyframes(self, keyframes: Union[List[KeyFrame], List[dict]]) -> GeneratedMove:
+        """Create a Move from keyframes."""
+        # Convert dicts to KeyFrame objects if needed
+        if keyframes and isinstance(keyframes[0], dict):
+            keyframes = [KeyFrame.from_dict(kf) for kf in keyframes]
+        move = GeneratedMove(keyframes)
+        self.last_movement = move
+        return move
+        
+    async def goto_pose(self, roll: float = 0, pitch: float = 0, yaw: float = 0, duration: float = 0.5):
+        """Go to a specific head pose."""
+        head = create_head_pose(roll=roll, pitch=pitch, yaw=yaw, degrees=True)
+        self.reachy.goto_target(head=head, duration=duration)
+        
+    async def play_move(self, move: Move):
+        """Play a move asynchronously."""
+        await self.reachy.async_play_move(move)
+        
+    async def stop(self):
+        """Stop current movement (return to neutral)."""
+        head = create_head_pose(roll=0, pitch=0, yaw=0, degrees=True)
+        self.reachy.goto_target(head=head, duration=0.3)
+```
+
+---
+
+### [MODIFY] `reachy_mini_danceml/main.py`
+
+Add Gradio UI with fastrtc audio stream.
+
+```python
+import threading
+import os
+from reachy_mini import ReachyMini, ReachyMiniApp
+import gradio as gr
+
+from .realtime_handler import RealtimeHandler
+from .movement_generator import MovementGenerator
+
+
+class ReachyMiniDanceml(ReachyMiniApp):
+    custom_app_url: str | None = "http://0.0.0.0:8042"
+
+    def run(self, reachy_mini: ReachyMini, stop_event: threading.Event):
+        # Initialize movement generator
+        generator = MovementGenerator(reachy_mini)
+        
+        # Check for OpenAI API key
+        openai_key = os.environ.get("OPENAI_API_KEY")
+        if not openai_key:
+            print("WARNING: OPENAI_API_KEY not set. Voice control disabled.")
+            return self._run_basic_mode(reachy_mini, stop_event)
+        
+        # Initialize realtime handler
+        handler = RealtimeHandler(openai_key, generator)
+        stream = handler.create_stream()
+        
+        # Create Gradio UI
+        with gr.Blocks(
+            title="Reachy Voice Control",
+            css=self._get_custom_css()
+        ) as demo:
+            gr.Markdown("# 🎤 Reachy Voice Control")
+            gr.Markdown("Speak to control Reachy's movements!")
+            
+            with gr.Row():
+                audio = gr.Audio(
+                    sources=["microphone"],
+                    streaming=True,
+                    label="Speak to Reachy"
+                )
+            
+            with gr.Row():
+                status = gr.Textbox(
+                    label="Status",
+                    value="Ready to listen...",
+                    interactive=False
+                )
+            
+            # Connect audio stream
+            stream.ui.render()
+        
+        # Launch Gradio
+        demo.launch(
+            server_name="0.0.0.0",
+            server_port=8042,
+            share=False,
+            prevent_thread_lock=True
+        )
+        
+        # Keep running until stop event
+        while not stop_event.is_set():
+            stop_event.wait(1)
+            
+    def _get_custom_css(self):
+        """Custom CSS matching project theme."""
+        return """
+        .gradio-container {
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%) !important;
+            min-height: 100vh;
+        }
+        .main {
+            background: white;
+            border-radius: 20px;
+            box-shadow: 0 20px 40px rgba(0, 0, 0, 0.1);
+            padding: 2rem;
+            margin: 2rem auto;
+            max-width: 800px;
+        }
+        h1 {
+            color: #1e293b;
+            text-align: center;
+        }
+        .audio-container {
+            background: #f0f9ff;
+            border-radius: 12px;
+            padding: 1.5rem;
+            margin: 1rem 0;
+        }
+        button.primary {
+            background: linear-gradient(135deg, #667eea, #764ba2) !important;
+            border: none !important;
+            border-radius: 50px !important;
+        }
+        """
+        
+    def _run_basic_mode(self, reachy_mini, stop_event):
+        """Fallback mode without voice control."""
+        while not stop_event.is_set():
+            stop_event.wait(1)
+
+
+if __name__ == "__main__":
+    app = ReachyMiniDanceml()
+    app.wrapped_run()
+```
+
+---
+
+### [MODIFY] `pyproject.toml`
+
+```diff
+  dependencies = [
+      "reachy-mini",
++     "fastrtc[vad]>=0.0.16",
++     "openai>=1.0.0",
++     "gradio>=4.0.0",
++     "scipy>=1.10.0",
+  ]
+```
+
+---
+
+## Verification Plan
+
+### Unit Tests
+
+Since the movement generator is the only pure-logic component, we focus tests there.
+
+#### [NEW] `tests/test_movement_generator.py`
+
+```python
+import pytest
+import numpy as np
+from reachy_mini_danceml.movement_generator import GeneratedMove
+from reachy_mini_danceml.movement_tools import KeyFrame
+
+def test_generate_move_from_keyframes():
+    """Test that keyframes produce valid Move object."""
+    keyframes = [
+        KeyFrame(t=0.0, head={"yaw": 0}, antennas=(0, 0)),
+        KeyFrame(t=1.0, head={"yaw": 30}, antennas=(20, -20)),
+    ]
+    move = GeneratedMove(keyframes)
+    assert move.duration == 1.0
+    
+def test_evaluate_returns_correct_shape():
+    """Test that evaluate returns proper head pose and antennas."""
+    keyframes = [
+        KeyFrame(t=0.0, head={"roll": 0, "pitch": 0, "yaw": 0}, antennas=(0, 0)),
+        KeyFrame(t=1.0, head={"roll": 10, "pitch": 10, "yaw": 30}, antennas=(20, -20)),
+    ]
+    move = GeneratedMove(keyframes)
+    head, antennas, body_yaw = move.evaluate(0.5)
+    
+    assert head.shape == (4, 4)  # 4x4 homogeneous matrix
+    assert len(antennas) == 2
+    assert body_yaw == 0.0
+
+def test_requires_minimum_keyframes():
+    """Test that we need at least 2 keyframes."""
+    with pytest.raises(ValueError):
+        GeneratedMove([KeyFrame(t=0, head={})])
+        
+def test_keyframe_from_dict():
+    """Test KeyFrame.from_dict parsing."""
+    data = {"t": 1.5, "head": {"yaw": 30}, "antennas": [10, -10]}
+    kf = KeyFrame.from_dict(data)
+    assert kf.t == 1.5
+    assert kf.head["yaw"] == 30
+    assert kf.antennas == (10, -10)
+```
+
+**Run tests:**
+```bash
+source venv/bin/activate
+pip install pytest
+python -m pytest tests/test_movement_generator.py -v
+```
+
+### Manual Testing (requires robot + microphone)
+
+Since Realtime API is primarily interactive, manual testing is essential.
+
+**Prerequisites:**
+1. Set `OPENAI_API_KEY=sk-...` in environment or `.env` file
+2. Have Reachy Mini connected (or simulator)
+3. Have microphone connected to computer
+
+**Test 1: Basic Startup**
+1. Run: `source venv/bin/activate && python -m reachy_mini_danceml.main`
+2. Open browser to `http://localhost:8042`
+3. ✅ Verify: Gradio UI loads with purple gradient theme
+
+**Test 2: Voice Recognition**
+1. Click microphone button in Gradio UI
+2. Say: "Hello Reachy"
+3. ✅ Verify: You hear Reachy respond with voice
+
+**Test 3: Simple Movement**
+1. Say: "Look to the left"
+2. ✅ Verify: Reachy's head rotates left
+3. ✅ Verify: Reachy says something like "Looking left!"
+
+**Test 4: Animated Movement**
+1. Say: "Wave your antennas excitedly"
+2. ✅ Verify: Antennas move in an animated pattern
+3. ✅ Verify: Reachy confirms verbally
+
+**Test 5: Complex Request**
+1. Say: "Nod your head up and down twice, then look at me"
+2. ✅ Verify: Reachy performs multi-part animation
+
+---
+
+## File Summary
+
+| File | Action | Description |
+|------|--------|-------------|
+| `realtime_handler.py` | NEW | OpenAI Realtime API + fastrtc integration |
+| `movement_tools.py` | NEW | Tool schemas and KeyFrame dataclass |
+| `movement_generator.py` | NEW | Keyframe interpolation and movement execution |
+| `main.py` | MODIFY | Add Gradio UI with audio stream |
+| `pyproject.toml` | MODIFY | Add fastrtc, openai, gradio, scipy |
+| `tests/test_movement_generator.py` | NEW | Unit tests |
+
+---
+
+## Comparison: Old vs New Architecture
+
+| Aspect | Previous (LangGraph + Web Speech) | New (Realtime API + fastrtc) |
+|--------|-----------------------------------|------------------------------|
+| Speech-to-text | Browser Web Speech API | OpenAI Realtime (built-in) |
+| LLM | Separate GPT-4.1 calls | Realtime API (streaming) |
+| Text-to-speech | None | OpenAI Realtime (built-in) |
+| Latency | ~500-800ms | ~200ms |
+| Reachy speaks? | No (text only) | Yes (voice responses) |
+| State management | LangGraph | OpenAI maintains conversation |
+| UI Framework | Custom HTML/JS | Gradio |
+| Complexity | More code | Less code (fastrtc handles details) |
+
+---
+
+## Next Steps (After Approval)
+
+1. ✅ Plan approved (waiting)
+2. [ ] Update `pyproject.toml` with dependencies
+3. [ ] Install dependencies
+4. [ ] Create `movement_tools.py`
+5. [ ] Create `movement_generator.py`
+6. [ ] Create `realtime_handler.py`
+7. [ ] Update `main.py` with Gradio UI
+8. [ ] Create unit tests
+9. [ ] Manual testing with robot

pyproject.toml

@@ -10,7 +10,12 @@ description = "Add your description here"
 readme = "README.md"
 requires-python = ">=3.10"
 dependencies = [
-    "reachy-mini"
+    "reachy-mini",
+    "fastrtc[vad]>=0.0.16",
+    "openai>=1.0.0",
+    "gradio>=4.0.0",
+    "scipy>=1.10.0",
+    "sounddevice>=0.4.6",
 ]
 keywords = ["reachy-mini-app"]
 

reachy_mini_danceml/audio_capture.py

@@ -0,0 +1,97 @@
+import asyncio
+import numpy as np
+import sounddevice as sd
+import threading
+import queue
+from typing import Optional, Callable, AsyncGenerator
+
+class LocalAudioCapture:
+    """Captures audio from the default local microphone using sounddevice."""
+    
+    def __init__(self, sample_rate: int = 24000, channels: int = 1, device_name: Optional[str] = None):
+        self.sample_rate = sample_rate
+        self.channels = channels
+        self.device_name = device_name
+        self._input_stream: Optional[sd.InputStream] = None
+        self._audio_queue = queue.Queue()  # Thread-safe queue
+        self._running = False
+        self._device_index: Optional[int] = None
+
+        if self.device_name:
+             self._device_index = self._find_device_index(self.device_name)
+             if self._device_index is not None:
+                 print(f"Selected audio device '{self.device_name}' (Index: {self._device_index})")
+             else:
+                 print(f"Warning: Audio device '{self.device_name}' not found. Using default.")
+
+    def _find_device_index(self, name_substring: str) -> Optional[int]:
+        """Find device index by name substring."""
+        try:
+            devices = sd.query_devices()
+            for i, dev in enumerate(devices):
+                if name_substring in dev['name'] and dev['max_input_channels'] > 0:
+                    return i
+        except Exception as e:
+            print(f"Error querying devices: {e}")
+        return None
+
+    def start(self):
+        """Start capturing audio."""
+        if self._running:
+            return
+            
+        self._running = True
+        
+        # Start sounddevice stream (uses callback in separate thread)
+        self._input_stream = sd.InputStream(
+            samplerate=self.sample_rate,
+            channels=self.channels,
+            dtype=np.int16,
+            callback=self._audio_callback,
+            device=self._device_index
+        )
+        self._input_stream.start()
+        print(f"Local audio capture started at {self.sample_rate}Hz using device {self._device_index if self._device_index is not None else 'Default'}")
+
+    def stop(self):
+        """Stop capturing audio."""
+        self._running = False
+        if self._input_stream:
+            self._input_stream.stop()
+            self._input_stream.close()
+            self._input_stream = None
+        print("Local audio capture stopped")
+
+    def _audio_callback(self, indata, frames, time, status):
+        """Callback from sounddevice (runs in a separate thread)."""
+        if status:
+            print(f"Audio status: {status}")
+        if self._running:
+            # Copy data effectively
+            audio_bytes = indata.tobytes()
+            # Thread-safe queue put
+            self._audio_queue.put_nowait(audio_bytes)
+
+    async def stream_audio(self) -> AsyncGenerator[bytes, None]:
+        """Yields audio chunks from the queue."""
+        while self._running:
+            try:
+                # Non-blocking poll of sync queue
+                try:
+                    chunk = self._audio_queue.get_nowait()
+                    yield chunk
+                except queue.Empty:
+                    # No data yet, sleep briefly
+                    await asyncio.sleep(0.01)
+            except asyncio.CancelledError:
+                break
+            except Exception as e:
+                print(f"Error in audio stream: {e}")
+                break
+    
+    def get_chunk(self, timeout: float = 0.1) -> Optional[bytes]:
+        """Get next audio chunk synchronously (with timeout)."""
+        try:
+            return self._audio_queue.get(timeout=timeout)
+        except queue.Empty:
+            return None

reachy_mini_danceml/dataset_loader.py

@@ -0,0 +1,189 @@
+import os
+from dataclasses import dataclass
+from typing import List, Dict, Optional, Tuple, Any
+import numpy as np
+from datasets import load_dataset, concatenate_datasets
+from scipy.spatial.transform import Rotation
+
+from reachy_mini.motion.recorded_move import RecordedMove
+
+from .movement_tools import KeyFrame
+
+@dataclass
+class MoveRecord:
+    name: str
+    description: str
+    dataset_source: str
+    raw_move_data: Dict[str, Any]  # Store raw data for RecordedMove
+    keyframes: List[KeyFrame]  # Keep for backward compatibility
+    
+    def to_recorded_move(self) -> RecordedMove:
+        """Create a RecordedMove from the raw data for smooth playback."""
+        return RecordedMove(self.raw_move_data)
+
+class MoveLibrary:
+    """
+    Manages loading, caching, and serving movement data from HuggingFace datasets.
+    """
+    DANCE_DATASET = "pollen-robotics/reachy-mini-dances-library"
+    EMOTION_DATASET = "pollen-robotics/reachy-mini-emotions-library"
+
+    def __init__(self):
+        self._moves: Dict[str, MoveRecord] = {}
+        self._is_loaded = False
+
+    def load(self):
+        """Loads and indexes datasets from HuggingFace."""
+        if self._is_loaded:
+            return
+
+        print("Loading movement library...")
+        try:
+            # Load datasets
+            dances = load_dataset(self.DANCE_DATASET, split="train")
+            emotions = load_dataset(self.EMOTION_DATASET, split="train")
+            
+            self._process_dataset(dances, "dance")
+            self._process_dataset(emotions, "emotion")
+            
+            self._is_loaded = True
+            print(f"Library loaded with {len(self._moves)} moves.")
+            
+        except Exception as e:
+            print(f"Error loading datasets: {e}")
+            # Fallback or re-raise depending on strictness requirements
+            raise e
+
+    def _process_dataset(self, dataset, source_type: str):
+        """Converts raw dataset rows into MoveRecord objects."""
+        for row in dataset:
+            try:
+                name = self._generate_id(row['description'])
+                description = row['description']
+                times = row['time']
+                targets = row['set_target_data']
+                
+                # Store raw data for RecordedMove (smooth playback)
+                raw_move_data = {
+                    'description': description,
+                    'time': times,
+                    'set_target_data': targets
+                }
+                
+                # Also create KeyFrames for backward compatibility
+                keyframes = []
+                for t, target in zip(times, targets):
+                    kf = self._convert_target_to_keyframe(t, target)
+                    keyframes.append(kf)
+                
+                # Deduplicate names if necessary (simple increment for now)
+                original_name = name
+                counter = 1
+                while name in self._moves:
+                    name = f"{original_name}_{counter}"
+                    counter += 1
+
+                self._moves[name] = MoveRecord(
+                    name=name,
+                    description=description,
+                    dataset_source=source_type,
+                    raw_move_data=raw_move_data,
+                    keyframes=keyframes
+                )
+            except Exception as e:
+                print(f"Skipping malformed row: {e}")
+
+    def _convert_target_to_keyframe(self, t: float, target: Dict[str, Any]) -> KeyFrame:
+        """
+        Converts the dataset's target structure (matrix, radians) 
+        to our SDK's KeyFrame structure (euler degrees).
+        """
+        # 1. Extract Head Pose (4x4 Matrix)
+        # Note: Dataset provides list of lists, need numpy
+        matrix = np.array(target['head'])
+        
+        # Extract rotation (top-left 3x3)
+        rot_matrix = matrix[:3, :3]
+        r = Rotation.from_matrix(rot_matrix)
+        
+        # Convert to Euler angles (degrees) - standard order for Reachy/SDK usually zyx or xyz?
+        # SDK create_head_pose uses: R.from_euler('xyz', [roll, pitch, yaw], degrees=True)
+        # So we should extract 'xyz'
+        roll, pitch, yaw = r.as_euler('xyz', degrees=True)
+        
+        # 2. Extract Antennas (Radians -> Degrees)
+        # Dataset: [left, right] in radians
+        ant_rad = target['antennas']
+        ant_deg = [np.rad2deg(a) for a in ant_rad]
+        
+        return KeyFrame(
+            t=t,
+            head={
+                "roll": float(roll),
+                "pitch": float(pitch),
+                "yaw": float(yaw)
+            },
+            antennas=tuple(ant_deg)
+        )
+
+    def _generate_id(self, description: str) -> str:
+        """Generates a slug-like ID from the description."""
+        # Simple heuristic: take first few words or key concept
+        # Actual descriptions are like "A sharp, forward..."
+        # We might want to use specific names if the dataset had them, but it doesn't really.
+        # Let's clean up the string to make a valid python-ish name
+        slug = description.lower()
+        slug = "".join(c if c.isalnum() else "_" for c in slug)
+        words = list(filter(None, slug.split("_")))
+        return "_".join(words[:4])  # Limit length
+
+    def get_move(self, name: str) -> Optional[MoveRecord]:
+        """Retrieves a move by its generated ID/name."""
+        if not self._is_loaded:
+            self.load()
+        return self._moves.get(name)
+
+    def search_moves(self, query: str) -> List[Dict[str, str]]:
+        """
+        Searches move descriptions using OR matching with relevance scoring.
+        Returns list of {"name": name, "description": description, "source": source}.
+        """
+        if not self._is_loaded:
+            self.load()
+            
+        params = query.lower().split()
+        scored_results = []
+        
+        for name, record in self._moves.items():
+            desc_lower = record.description.lower()
+            name_lower = name.lower()
+            
+            # Score based on how many keywords match (OR logic)
+            score = 0
+            for p in params:
+                if p in desc_lower:
+                    score += 2  # Description match worth more
+                if p in name_lower:
+                    score += 1  # Name match
+            
+            if score > 0:
+                scored_results.append({
+                    "name": name, 
+                    "description": record.description,
+                    "source": record.dataset_source,
+                    "_score": score
+                })
+        
+        # Sort by score (highest first)
+        scored_results.sort(key=lambda x: x["_score"], reverse=True)
+        
+        # Remove internal score field
+        for r in scored_results:
+            del r["_score"]
+        
+        return scored_results[:15]  # Return more results for better options
+
+    def list_moves(self) -> List[str]:
+        if not self._is_loaded:
+            self.load()
+        return list(self._moves.keys())

reachy_mini_danceml/main.py

@@ -1,67 +1,321 @@
+"""Voice-controlled Reachy Mini app using OpenAI Realtime API.
+
+This app allows users to speak to Reachy Mini and control its movements
+through natural language commands. Uses OpenAI Realtime API for low-latency
+bidirectional voice conversations.
+"""
+
 import threading
+import os
+import asyncio
+from pathlib import Path
+
+# Load .env file from the project directory
+from dotenv import load_dotenv
+# Try loading from multiple possible locations
+env_paths = [
+    Path(__file__).parent.parent / ".env",  # Project root
+    Path.cwd() / ".env",  # Current working directory
+]
+for env_path in env_paths:
+    if env_path.exists():
+        load_dotenv(env_path)
+        print(f"Loaded environment from: {env_path}")
+        break
+
 from reachy_mini import ReachyMini, ReachyMiniApp
 from reachy_mini.utils import create_head_pose
 import numpy as np
-import time
-from pydantic import BaseModel
+
+from .movement_generator import MovementGenerator
+from .realtime_handler import RealtimeHandler
 
 
 class ReachyMiniDanceml(ReachyMiniApp):
-    # Optional: URL to a custom configuration page for the app
-    # eg. "http://localhost:8042"
+    """Voice-controlled Reachy Mini app with Gradio UI."""
+    
+    # URL for the custom app page (Gradio will serve here)
     custom_app_url: str | None = "http://0.0.0.0:8042"
+    
+    def __init__(self):
+        super().__init__()
+        self._stop_event = None
 
-    def run(self, reachy_mini: ReachyMini, stop_event: threading.Event):
-        t0 = time.time()
-
-        antennas_enabled = True
-        sound_play_requested = False
-
-        # You can ignore this part if you don't want to add settings to your app. If you set custom_app_url to None, you have to remove this part as well.
-        # === vvv ===
-        class AntennaState(BaseModel):
-            enabled: bool
-
-        @self.settings_app.post("/antennas")
-        def update_antennas_state(state: AntennaState):
-            nonlocal antennas_enabled
-            antennas_enabled = state.enabled
-            return {"antennas_enabled": antennas_enabled}
-
-        @self.settings_app.post("/play_sound")
-        def request_sound_play():
-            nonlocal sound_play_requested
-            sound_play_requested = True
+    def stop(self):
+        """Stop the application."""
+        print("Stopping app...")
+        if self._stop_event:
+            self._stop_event.set()
+    
+    def wrapped_run(self, *args, **kwargs):
+        """Override to connect without requiring camera.
+        
+        Our app only uses voice control, so we don't need the camera.
+        This prevents the SDK from failing if camera access is denied.
+        """
+        import threading
+        
+        # Try to connect without camera first
+        try:
+            # Set environment variable to skip camera auth dialog
+            os.environ['OPENCV_AVFOUNDATION_SKIP_AUTH'] = '1'
             
-        # === ^^^ ===
-
-        # Main control loop
-        while not stop_event.is_set():
-            t = time.time() - t0
-
-            yaw_deg = 30.0 * np.sin(2.0 * np.pi * 0.2 * t)
-            head_pose = create_head_pose(yaw=yaw_deg, degrees=True)
-
-            if antennas_enabled:
-                amp_deg = 25.0
-                a = amp_deg * np.sin(2.0 * np.pi * 0.5 * t)
-                antennas_deg = np.array([a, -a])
+            with ReachyMini(*args, **kwargs) as reachy_mini:
+                self._stop_event = threading.Event()
+                stop_event = self._stop_event
+                
+                # Run the app
+                try:
+                    self.run(reachy_mini, stop_event)
+                except KeyboardInterrupt:
+                    stop_event.set()
+                    
+        except RuntimeError as e:
+            if "Camera not found" in str(e):
+                print("=" * 60)
+                print("Note: Camera not available, but that's OK!")
+                print("Voice control doesn't require a camera.")
+                print("Proceeding without camera...")
+                print("=" * 60)
+                
+                # Create a minimal connection without media manager
+                # by catching and ignoring the camera error
+                self._run_without_camera(*args, **kwargs)
             else:
-                antennas_deg = np.array([0.0, 0.0])
+                raise
+                
+    def _run_without_camera(self, *args, **kwargs):
+        """Run the app with a manual ReachyMini connection that tolerates camera failure."""
+        import threading
+        from reachy_mini.io.zenoh_client import ZenohClient
+        
+        # Create a simple client connection without full media manager
+        client = ZenohClient(localhost_only=True)
+        client.wait_for_connection(timeout=5.0)
+        
+        # Create a minimal ReachyMini-like object
+        class MinimalReachy:
+            def __init__(self, client):
+                self._client = client
+                
+            def set_target(self, head=None, antennas=None, body_yaw=None):
+                # Send target via client
+                pass
+                
+            def goto_target(self, head=None, antennas=None, body_yaw=None, duration=0.5):
+                # Send goto via client
+                pass
+                
+            async def async_play_move(self, move):
+                # Play move - simplified
+                pass
+        
+        # For now, just inform user this mode isn't fully implemented
+        print("=" * 60)
+        print("Camera-less mode not fully implemented yet.")
+        print("Please grant camera access to Terminal in System Preferences")
+        print("or contact support for help.")
+        print("=" * 60)
 
-            if sound_play_requested:
-                print("Playing sound...")
-                reachy_mini.media.play_sound("wake_up.wav")
-                sound_play_requested = False
-
-            antennas_rad = np.deg2rad(antennas_deg)
-
-            reachy_mini.set_target(
-                head=head_pose,
-                antennas=antennas_rad,
+    def run(self, reachy_mini: ReachyMini, stop_event: threading.Event):
+        """Run the voice control app.
+        
+        Args:
+            reachy_mini: Connected ReachyMini robot instance.
+            stop_event: Event to signal when to stop.
+        """
+        # Check for OpenAI API key
+        openai_key = os.environ.get("OPENAI_API_KEY")
+        if not openai_key:
+            print("=" * 60)
+            print("WARNING: OPENAI_API_KEY not set!")
+            print("Voice control is disabled.")
+            print("Set OPENAI_API_KEY environment variable to enable.")
+            print("=" * 60)
+            self._run_fallback_mode(reachy_mini, stop_event)
+            return
+            
+        # Initialize components
+        generator = MovementGenerator(reachy_mini)
+        handler = RealtimeHandler(openai_key, generator, audio_device_name="Reachy Mini Audio")
+        
+        # Create and launch Gradio UI
+        self._launch_gradio(handler, stop_event)
+        
+    def _launch_gradio(self, handler: RealtimeHandler, stop_event: threading.Event):
+        """Launch the Gradio web UI.
+        
+        Args:
+            handler: Configured RealtimeHandler.
+            stop_event: Event to signal when to stop.
+        """
+        import gradio as gr
+        
+        # Custom CSS matching project theme
+        custom_css = """
+        .gradio-container {
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%) !important;
+            min-height: 100vh;
+        }
+        .main {
+            background: white;
+            border-radius: 20px;
+            box-shadow: 0 20px 40px rgba(0, 0, 0, 0.1);
+            padding: 2rem;
+            margin: 2rem auto;
+            max-width: 800px;
+        }
+        h1, h2 {
+            color: #1e293b;
+            text-align: center;
+        }
+        .voice-status {
+            background: #f0f9ff;
+            border: 2px solid #e0f2fe;
+            border-radius: 12px;
+            padding: 1rem;
+            text-align: center;
+            color: #0369a1;
+            font-weight: 500;
+        }
+        .instruction-box {
+            background: #fef3c7;
+            border: 2px solid #fde047;
+            border-radius: 12px;
+            padding: 1rem;
+            margin: 1rem 0;
+        }
+        button.primary {
+            background: linear-gradient(135deg, #667eea, #764ba2) !important;
+            border: none !important;
+            border-radius: 50px !important;
+            padding: 1rem 2rem !important;
+        }
+        """
+        
+        # Build Gradio interface
+        with gr.Blocks(
+            title="Reachy Voice Control",
+            css=custom_css,
+            theme=gr.themes.Soft()
+        ) as demo:
+            gr.Markdown("# 🎤 Reachy Voice Control")
+            gr.Markdown("Speak to control Reachy's movements!")
+            
+            with gr.Row():
+                with gr.Column():
+                    # Instructions
+                    gr.Markdown("""
+                    ### How to use:
+                    1. Click the microphone button to start
+                    2. Speak your command (e.g., "Look to the left" or "Wave hello")
+                    3. Reachy will respond and move!
+                    
+                    ### Example commands:
+                    - "Look up" / "Look down" / "Look left" / "Look right"
+                    - "Nod your head yes"
+                    - "Wave your antennas excitedly"
+                    - "Do a happy dance"
+                    - "Tilt your head curiously"
+                    """)
+            
+            # Microphone toggle button
+            with gr.Row():
+                mic_button = gr.Button(
+                    "🎤 Start Listening",
+                    variant="primary",
+                    size="lg",
+                    elem_id="mic-toggle"
+                )
+            
+            with gr.Row():
+                status = gr.Textbox(
+                    label="Status",
+                    value="🤖 Ready - Click the button to start listening",
+                    interactive=False,
+                    elem_classes=["voice-status"]
+                )
+            
+            # Button click handler
+            def toggle_mic():
+                handler.toggle_listening()
+                if handler.is_listening:
+                    return gr.update(value="🔴 Stop Listening", variant="stop"), "🎤 Listening... Speak now!"
+                else:
+                    return gr.update(value="🎤 Start Listening", variant="primary"), "🤖 Ready - Click the button to start listening"
+            
+            mic_button.click(
+                fn=toggle_mic,
+                outputs=[mic_button, status]
             )
+            
+            gr.Markdown("### 💡 Tip: Click the button above to enable voice control")
 
-            time.sleep(0.02)
+        # Launch Gradio server
+        print("=" * 60)
+        print("🎤 Reachy Voice Control Starting!")
+        print("Open http://localhost:8042 in your browser")
+        print("=" * 60)
+        
+        demo.launch(
+            server_name="0.0.0.0",
+            server_port=8042,
+            share=False,
+            prevent_thread_lock=True,
+            show_error=True
+        )
+        
+        # Start robot mic capture
+        from .audio_capture import LocalAudioCapture
+        
+        print("Starting robot microphone capture...")
+        local_capture = LocalAudioCapture(device_name="Reachy Mini Audio")
+        local_capture.start()
+        
+        # Run the OpenAI session in the background
+        async def run_session():
+            try:
+                await handler.run_local_session(local_capture._audio_queue, stop_event)
+            except Exception as e:
+                print(f"Session error: {e}")
+        
+        # Start the async session
+        import threading
+        def run_async_session():
+            loop = asyncio.new_event_loop()
+            asyncio.set_event_loop(loop)
+            loop.run_until_complete(run_session())
+        
+        session_thread = threading.Thread(target=run_async_session, daemon=True)
+        session_thread.start()
+        
+        # Keep running until stop event
+        while not stop_event.is_set():
+            stop_event.wait(1)
+            
+        print("Shutting down...")
+        local_capture.stop()
+        demo.close()
+            
+    def _run_fallback_mode(self, reachy_mini: ReachyMini, stop_event: threading.Event):
+        """Run a simple fallback mode when API key is not available.
+        
+        Just keeps robot alive without voice control.
+        
+        Args:
+            reachy_mini: Connected ReachyMini robot instance.
+            stop_event: Event to signal when to stop.
+        """
+        print("Running in fallback mode (no voice control)...")
+        
+        # Return to neutral position
+        head_pose = create_head_pose(roll=0, pitch=0, yaw=0, degrees=True)
+        antennas = np.array([0.0, 0.0])
+        reachy_mini.set_target(head=head_pose, antennas=antennas)
+        
+        # Wait for stop
+        while not stop_event.is_set():
+            stop_event.wait(1)
 
 
 if __name__ == "__main__":

reachy_mini_danceml/movement_generator.py

@@ -0,0 +1,375 @@
+"""Movement generator for voice-controlled Reachy Mini.
+
+Creates smooth movements from keyframes using cubic spline interpolation.
+"""
+
+import asyncio
+from dataclasses import dataclass
+from typing import List, Optional, Union
+import numpy as np
+from scipy.interpolate import CubicSpline
+from reachy_mini import ReachyMini
+from reachy_mini.motion.move import Move
+from reachy_mini.utils import create_head_pose
+
+from .movement_tools import KeyFrame
+from .dataset_loader import MoveLibrary
+
+
+def lerp(v0: float, v1: float, alpha: float) -> float:
+    """Linear interpolation between two values."""
+    return v0 + alpha * (v1 - v0)
+
+
+class GeneratedMove(Move):
+    """A Move generated from keyframes using linear pose interpolation.
+    
+    Uses the same interpolation approach as SDK's RecordedMove for
+    smooth, consistent motion.
+    """
+    
+    def __init__(self, keyframes: List[KeyFrame]):
+        """Initialize from a list of keyframes.
+        
+        Args:
+            keyframes: At least 2 keyframes defining the animation.
+            
+        Raises:
+            ValueError: If fewer than 2 keyframes provided.
+        """
+        if len(keyframes) < 2:
+            raise ValueError("Need at least 2 keyframes for interpolation")
+            
+        self._keyframes = keyframes
+        self._times = [kf.t for kf in keyframes]
+        
+        # Pre-compute head poses as 4x4 matrices
+        self._head_poses = []
+        for kf in keyframes:
+            roll = kf.head.get("roll", 0)
+            pitch = kf.head.get("pitch", 0)
+            yaw = kf.head.get("yaw", 0)
+            head = create_head_pose(roll=roll, pitch=pitch, yaw=yaw, degrees=True)
+            self._head_poses.append(head)
+        
+        # Pre-compute antenna positions in radians
+        self._antennas = []
+        for kf in keyframes:
+            left = np.deg2rad(kf.antennas[0])
+            right = np.deg2rad(kf.antennas[1])
+            self._antennas.append([left, right])
+        
+        self._duration = max(self._times)
+        
+    @property
+    def duration(self) -> float:
+        """Get the total duration of this move in seconds."""
+        return self._duration
+        
+    def evaluate(self, t: float):
+        """Evaluate the move at time t using linear pose interpolation.
+        
+        Args:
+            t: Time in seconds from start of animation.
+            
+        Returns:
+            Tuple of (head_pose, antennas, body_yaw):
+                - head_pose: 4x4 homogeneous transformation matrix
+                - antennas: numpy array of [left, right] in radians
+                - body_yaw: body rotation in radians (always 0)
+        """
+        from reachy_mini.utils.interpolation import linear_pose_interpolation
+        import bisect
+        
+        # Clamp time to valid range
+        t = max(0, min(t, self._duration - 1e-3))
+        
+        # Find the right interval using binary search (like RecordedMove)
+        index = bisect.bisect_right(self._times, t)
+        idx_prev = index - 1 if index > 0 else 0
+        idx_next = index if index < len(self._times) else idx_prev
+        
+        t_prev = self._times[idx_prev]
+        t_next = self._times[idx_next]
+        
+        # Calculate interpolation alpha
+        if t_next == t_prev:
+            alpha = 0.0
+        else:
+            alpha = (t - t_prev) / (t_next - t_prev)
+        
+        # Linear pose interpolation for head (like RecordedMove)
+        head_prev = self._head_poses[idx_prev]
+        head_next = self._head_poses[idx_next]
+        head_pose = linear_pose_interpolation(head_prev, head_next, alpha)
+        
+        # Linear interpolation for antennas
+        ant_prev = self._antennas[idx_prev]
+        ant_next = self._antennas[idx_next]
+        antennas = np.array([
+            lerp(ant_prev[0], ant_next[0], alpha),
+            lerp(ant_prev[1], ant_next[1], alpha)
+        ])
+        
+        return head_pose, antennas, 0.0
+
+
+class MovementGenerator:
+    """Generates and executes movements on Reachy Mini.
+    
+    Handles:
+    - Simple goto poses
+    - Complex keyframe animations
+    - Movement state tracking
+    """
+    
+    def __init__(self, reachy: ReachyMini):
+        """Initialize with a ReachyMini instance.
+        
+        Args:
+            reachy: Connected ReachyMini robot instance.
+        """
+        self.reachy = reachy
+        self.last_movement: Optional[GeneratedMove] = None
+        self._is_playing = False
+        self._move_library = MoveLibrary()
+        self._idle_enabled = True
+        
+        # Pre-load the move library to avoid first-command latency
+        print("📚 Pre-loading movement library...")
+        self._move_library.load()
+        print("✅ Movement library ready!")
+        
+        # Motor control thread components
+        self._motor_thread = None
+        self._motor_stop_event = None
+        self._current_move = None  # Move to play (thread-safe via _is_playing flag)
+        self._move_done_event = None  # Signal when move completes
+        
+    def start_motor_thread(self, stop_event):
+        """Start the dedicated motor control thread.
+        
+        This runs synchronously like the default app for guaranteed 50 Hz.
+        """
+        import threading
+        self._motor_stop_event = stop_event
+        self._move_done_event = threading.Event()
+        self._motor_thread = threading.Thread(target=self._motor_loop, daemon=True)
+        self._motor_thread.start()
+        print("🌀 Motor control thread started (100 Hz moves, 50 Hz idle)")
+        
+    def _motor_loop(self):
+        """Synchronous motor control loop - runs in dedicated thread."""
+        import time
+        t0 = time.time()
+        
+        while not self._motor_stop_event.is_set():
+            loop_start = time.time()
+            
+            # Check if a move should be playing
+            if self._is_playing and self._current_move is not None:
+                # Play the move
+                move = self._current_move
+                move_start = time.time()
+                
+                while time.time() - move_start < move.duration:
+                    if self._motor_stop_event.is_set():
+                        break
+                    
+                    t = time.time() - move_start
+                    try:
+                        head, antennas, body_yaw = move.evaluate(t)
+                        self.reachy.set_target(head=head, antennas=antennas, body_yaw=body_yaw)
+                    except Exception as e:
+                        print(f"Move playback error: {e}")
+                        break
+                    
+                    time.sleep(0.01)  # 100 Hz for move playback
+                
+                # Move finished
+                self._current_move = None
+                self._is_playing = False
+                self._move_done_event.set()
+                
+            elif self._idle_enabled:
+                # Idle animation
+                t = time.time() - t0
+                
+                # Gentle yaw oscillation
+                yaw_deg = 15.0 * np.sin(2.0 * np.pi * 0.1 * t)
+                head_pose = create_head_pose(yaw=yaw_deg, degrees=True)
+                
+                # Gentle antenna wave
+                amp_deg = 15.0
+                a = amp_deg * np.sin(2.0 * np.pi * 0.3 * t)
+                antennas_rad = np.deg2rad(np.array([a, -a]))
+                
+                try:
+                    self.reachy.set_target(head=head_pose, antennas=antennas_rad, body_yaw=0.0)
+                except Exception as e:
+                    pass  # Ignore idle errors
+                
+                time.sleep(0.02)  # 50 Hz for idle
+            else:
+                time.sleep(0.02)
+        
+        print("🌀 Motor control thread stopped")
+    
+    def queue_move(self, move) -> None:
+        """Queue a move to be played by the motor thread."""
+        import threading
+        self._move_done_event = threading.Event()
+        self._current_move = move
+        self._is_playing = True
+        
+    def wait_for_move(self, timeout=30.0) -> bool:
+        """Wait for the current move to complete. Returns True if completed."""
+        if self._move_done_event:
+            return self._move_done_event.wait(timeout=timeout)
+        return True
+        
+    def play_move_by_name(self, name: str) -> str:
+        """
+        Plays a move from the library by its ID/name.
+        Returns a status string.
+        """
+        record = self._move_library.get_move(name)
+        
+        # If exact match fails, try search
+        if not record:
+            results = self._move_library.search_moves(name)
+            if results:
+                # Pick the first result's ID (best guess)
+                best_match = results[0]["name"]
+                record = self._move_library.get_move(best_match)
+                print(f"Exact match not found for '{name}', playing '{best_match}' instead.")
+        
+        if not record:
+            return f"Move '{name}' not found."
+
+        try:
+            move = self.create_from_keyframes(record.keyframes)
+            import asyncio
+            # We need to run this on the event loop, but play_move is async.
+            # Usually this is called from async context.
+            # If called from async handle_tool_call, we should return the Move or waitable.
+            # Wait, handle_tool_call awaits this. This method should be async.
+            return record  # Let caller handle playing, or better: make this async.
+        except Exception as e:
+            return f"Error creating move: {e}"
+
+    async def play_library_move(self, name: str) -> str:
+        """Play a library move using the dedicated motor thread for smooth playback."""
+        record_or_error = self.play_move_by_name(name)
+        if isinstance(record_or_error, str):
+            return record_or_error # Error message
+        
+        record = record_or_error
+        
+        # Use RecordedMove for smooth playback (like dashboard)
+        move = record.to_recorded_move()
+        
+        print(f"\n   ⏱️ PROFILING MOVE: duration={move.duration:.2f}s")
+        import time
+        start_time = time.time()
+        
+        # Queue move to motor thread and wait
+        self.queue_move(move)
+        self.wait_for_move(timeout=move.duration + 5.0)
+        
+        elapsed = time.time() - start_time
+        drift = elapsed - move.duration
+        print(f"   ⏱️ PROFILE RESULTS:")
+        print(f"      Expected: {move.duration:.3f}s")
+        print(f"      Actual:   {elapsed:.3f}s")
+        print(f"      Drift:    {drift:+.3f}s ({abs(drift/move.duration)*100:.1f}%)")
+        
+        return f"Playing movement: {record.description}"
+        
+    def create_from_keyframes(
+        self, 
+        keyframes: Union[List[KeyFrame], List[dict]]
+    ) -> GeneratedMove:
+        """Create a Move from keyframes.
+        
+        Args:
+            keyframes: List of KeyFrame objects or dicts to convert.
+            
+        Returns:
+            GeneratedMove ready to play.
+        """
+        # Convert dicts to KeyFrame objects if needed
+        if keyframes and isinstance(keyframes[0], dict):
+            keyframes = [KeyFrame.from_dict(kf) for kf in keyframes]
+            
+        move = GeneratedMove(keyframes)
+        self.last_movement = move
+        return move
+        
+    async def goto_pose(
+        self, 
+        roll: float = 0, 
+        pitch: float = 0, 
+        yaw: float = 0,
+        body_yaw: float = 0,
+        duration: float = 0.5
+    ) -> None:
+        """Go to a specific head and body pose smoothly.
+        
+        Args:
+            roll: Roll angle in degrees (-30 to 30).
+            pitch: Pitch angle in degrees (-30 to 30).
+            yaw: Yaw angle in degrees (-45 to 45).
+            body_yaw: Body rotation angle in degrees (-45 to 45).
+            duration: Time to reach pose in seconds.
+        """
+        # Clamp values to safe ranges
+        roll = max(-30, min(30, roll))
+        pitch = max(-30, min(30, pitch))
+        yaw = max(-45, min(45, yaw))
+        body_yaw_rad = np.deg2rad(max(-45, min(45, body_yaw)))
+        
+        head = create_head_pose(roll=roll, pitch=pitch, yaw=yaw, degrees=True)
+        antennas = np.array([0.0, 0.0])  # Keep antennas at current position
+        self.reachy.goto_target(head=head, antennas=antennas, body_yaw=body_yaw_rad, duration=duration)
+        
+    async def play_move(self, move: Move, profile: bool = True) -> None:
+        """Play a move asynchronously with optional profiling.
+        
+        Args:
+            move: Move object to play.
+            profile: If True, logs timing statistics.
+        """
+        import time
+        
+        self._is_playing = True
+        
+        if profile:
+            print(f"\n   ⏱️ PROFILING MOVE: duration={move.duration:.2f}s")
+            start_time = time.time()
+        
+        try:
+            await self.reachy.async_play_move(move)
+        finally:
+            self._is_playing = False
+            
+            if profile:
+                elapsed = time.time() - start_time
+                expected = move.duration
+                drift = elapsed - expected
+                
+                print(f"   ⏱️ PROFILE RESULTS:")
+                print(f"      Expected: {expected:.3f}s")
+                print(f"      Actual:   {elapsed:.3f}s")
+                print(f"      Drift:    {drift:+.3f}s ({abs(drift/expected)*100:.1f}%)")
+                
+                if abs(drift) > 0.1:
+                    print(f"      ⚠️ Significant timing drift detected!")
+        
+    async def stop(self) -> None:
+        """Stop current movement and return to neutral position."""
+        self._is_playing = False
+        # Return to neutral
+        head = create_head_pose(roll=0, pitch=0, yaw=0, degrees=True)
+        antennas = np.array([0.0, 0.0])
+        self.reachy.goto_target(head=head, antennas=antennas, body_yaw=0.0, duration=0.3)

reachy_mini_danceml/movement_tools.py

@@ -0,0 +1,220 @@
+"""Movement tools for voice-controlled Reachy Mini.
+
+Defines KeyFrame dataclass and tool schemas for OpenAI function calling.
+"""
+
+from dataclasses import dataclass
+from typing import Tuple
+
+
+@dataclass
+class KeyFrame:
+    """A single keyframe in an animation sequence.
+    
+    Attributes:
+        t: Time in seconds from start of animation
+        head: Dict with roll, pitch, yaw in degrees
+        antennas: Tuple of (left, right) antenna angles in degrees
+    """
+    t: float
+    head: dict  # {"roll": 0, "pitch": 0, "yaw": 0}
+    antennas: Tuple[float, float] = (0.0, 0.0)
+    
+    @classmethod
+    def from_dict(cls, data: dict) -> "KeyFrame":
+        """Create KeyFrame from dictionary (for JSON deserialization)."""
+        return cls(
+            t=data.get("t", 0),
+            head=data.get("head", {}),
+            antennas=tuple(data.get("antennas", [0, 0]))
+        )
+
+
+# Tool schemas for OpenAI Realtime API function calling
+GOTO_POSE_TOOL = {
+    "type": "function",
+    "name": "goto_pose",
+    "description": "Move the robot's head and/or body to a specific pose smoothly",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "roll": {
+                "type": "number",
+                "description": "Head roll angle in degrees (-30 to 30). Positive tilts head right.",
+                "default": 0
+            },
+            "pitch": {
+                "type": "number",
+                "description": "Head pitch angle in degrees (-30 to 30). NEGATIVE = look UP, POSITIVE = look DOWN.",
+                "default": 0
+            },
+            "yaw": {
+                "type": "number",
+                "description": "Head yaw angle in degrees (-45 to 45). Positive looks left.",
+                "default": 0
+            },
+            "body_yaw": {
+                "type": "number",
+                "description": "Body rotation angle in degrees (-45 to 45). Positive turns body left. Use for 'turn around', 'face left/right'.",
+                "default": 0
+            },
+            "duration": {
+                "type": "number",
+                "description": "Duration of movement in seconds",
+                "default": 0.5
+            }
+        }
+    }
+}
+
+GENERATE_MOTION_TOOL = {
+    "type": "function",
+    "name": "generate_motion",
+    "description": "Generate smooth procedural motion. Use for expressive animations. Supports position offsets, waveforms, and transients!",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "motion_type": {
+                "type": "string",
+                "enum": ["oscillate", "wave", "bounce", "spiral", "sway", "nod", "shake", "figure8", "peek", "recoil", "chicken", "dizzy", "yeah", "groove"],
+                "description": "Motion pattern: nod (yes), shake (no), wave (flowing), bounce (energetic), spiral (circular), sway (gentle), peek (peekaboo), recoil (surprise), chicken (peck), dizzy (circles), yeah (emphatic), groove (funky)"
+            },
+            "duration": {
+                "type": "number",
+                "description": "Duration in seconds (1-10)",
+                "default": 3.0
+            },
+            "pitch_amplitude": {
+                "type": "number",
+                "description": "Up/down motion in degrees (0-30)",
+                "default": 15
+            },
+            "yaw_amplitude": {
+                "type": "number",
+                "description": "Left/right turn in degrees (0-45)",
+                "default": 20
+            },
+            "roll_amplitude": {
+                "type": "number",
+                "description": "Head tilt in degrees (0-30)",
+                "default": 5
+            },
+            "antenna_amplitude": {
+                "type": "number",
+                "description": "Antenna movement in degrees (0-60)",
+                "default": 30
+            },
+            "body_yaw_amplitude": {
+                "type": "number",
+                "description": "Body rotation/swivel amplitude in degrees (0-45). Use for dancing, turning!",
+                "default": 0
+            },
+            "x_offset_amplitude": {
+                "type": "number",
+                "description": "Forward/back oscillation in meters (0-0.02). Keep small to avoid IK errors.",
+                "default": 0
+            },
+            "y_offset_amplitude": {
+                "type": "number",
+                "description": "Side-to-side oscillation in meters (0-0.02). Keep small to avoid IK errors.",
+                "default": 0
+            },
+            "z_offset_amplitude": {
+                "type": "number",
+                "description": "Up/down oscillation in meters (0-0.02). Keep small to avoid IK errors.",
+                "default": 0
+            },
+            "x_drift": {
+                "type": "number",
+                "description": "Gradual forward/back movement in meters (-0.02 to 0.02). Keep small!",
+                "default": 0
+            },
+            "y_drift": {
+                "type": "number",
+                "description": "Gradual left/right movement in meters (-0.02 to 0.02).",
+                "default": 0
+            },
+            "z_drift": {
+                "type": "number",
+                "description": "Gradual rise (+) or sink (-) in meters (-0.02 to 0.02). Use for snake charmer effect.",
+                "default": 0
+            },
+            "tempo": {
+                "type": "number",
+                "description": "Speed: 0.3=slow, 1.0=normal, 2.0=fast",
+                "default": 1.0
+            },
+            "intensity": {
+                "type": "number",
+                "description": "Overall scale 0.0-1.0",
+                "default": 1.0
+            },
+            "waveform": {
+                "type": "string",
+                "enum": ["sin", "triangle", "square", "sawtooth"],
+                "description": "Wave shape: sin (smooth), triangle (linear ramps), square (snappy), sawtooth (ramp)",
+                "default": "sin"
+            },
+            "transient_enabled": {
+                "type": "boolean",
+                "description": "Enable impulse/transient modifier for sharper movements",
+                "default": False
+            }
+        },
+        "required": ["motion_type"]
+    }
+}
+
+STOP_MOVEMENT_TOOL = {
+    "type": "function",
+    "name": "stop_movement",
+    "description": "Stop any currently playing movement and return to neutral position"
+}
+
+PLAY_MOVE_TOOL = {
+    "type": "function",
+    "name": "play_move",
+    "description": "Play a pre-defined movement from the library by its name (e.g., 'joy', 'fear', 'chicken_dance'). Prefer this over creating sequences manually.",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "name": {
+                "type": "string",
+                "description": "Name or ID of the movement to play"
+            }
+        },
+        "required": ["name"]
+    }
+}
+
+SEARCH_MOVES_TOOL = {
+    "type": "function",
+    "name": "search_moves",
+    "description": "Search the movement library for available expressions or dances.",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "query": {
+                "type": "string",
+                "description": "Keywords to search for (e.g., 'happy', 'dance', 'scared')"
+            }
+        },
+        "required": ["query"]
+    }
+}
+
+GET_CHOREOGRAPHY_GUIDE_TOOL = {
+    "type": "function",
+    "name": "get_choreography_guide",
+    "description": "Read the choreography guide to learn how to create safe and expressive custom movements."
+}
+
+# All tools for easy import
+ALL_TOOLS = [
+    GOTO_POSE_TOOL, 
+    GENERATE_MOTION_TOOL, 
+    STOP_MOVEMENT_TOOL,
+    PLAY_MOVE_TOOL,
+    SEARCH_MOVES_TOOL,
+    GET_CHOREOGRAPHY_GUIDE_TOOL
+]

reachy_mini_danceml/procedural_motion.py

@@ -0,0 +1,370 @@
+"""Enhanced procedural motion generation with position offsets, waveforms, transients, and phrases.
+
+Generates continuous motion from parameters. Inspired by the dance.py module.
+"""
+
+import numpy as np
+from typing import Optional, Tuple, List
+from reachy_mini.motion.move import Move
+from reachy_mini.utils import create_head_pose
+
+
+# ──────────────────────────── WAVEFORM FUNCTIONS ────────────────────────────
+
+def waveform_sin(phase: float) -> float:
+    """Standard sine wave."""
+    return np.sin(phase)
+
+def waveform_triangle(phase: float) -> float:
+    """Triangle wave - linear ramps."""
+    return 2.0 * np.abs(2.0 * (phase / (2*np.pi) - np.floor(phase / (2*np.pi) + 0.5))) - 1.0
+
+def waveform_square(phase: float) -> float:
+    """Square wave - hard on/off."""
+    return 1.0 if np.sin(phase) >= 0 else -1.0
+
+def waveform_sawtooth(phase: float) -> float:
+    """Sawtooth wave - linear ramp up, instant drop."""
+    return 2.0 * (phase / (2*np.pi) - np.floor(phase / (2*np.pi) + 0.5))
+
+WAVEFORMS = {
+    "sin": waveform_sin,
+    "triangle": waveform_triangle,
+    "square": waveform_square,
+    "sawtooth": waveform_sawtooth,
+}
+
+
+# ──────────────────────────── EASING FUNCTIONS ────────────────────────────
+
+def ease_smooth(t: float) -> float:
+    """Smooth step easing (cubic hermite)."""
+    t = np.clip(t, 0.0, 1.0)
+    return t * t * (3 - 2 * t)
+
+def ease_bounce(t: float) -> float:
+    """Bounce at the end."""
+    t = np.clip(t, 0.0, 1.0)
+    if t < 0.5:
+        return 8 * t * t * t * t
+    else:
+        t = t - 1
+        return 1 - 8 * t * t * t * t
+
+
+# ──────────────────────────── TRANSIENT MOTION ────────────────────────────
+
+def transient_impulse(t: float, duration: float, delay: float = 0.0, repeat: float = 0.0) -> float:
+    """Generate a transient impulse (quick hit and decay).
+    
+    Args:
+        t: Current time
+        duration: How long the impulse lasts
+        delay: Delay before impulse starts
+        repeat: If > 0, repeat every this many seconds
+        
+    Returns:
+        Value from 0 to 1 representing the impulse
+    """
+    if repeat > 0:
+        t = t % repeat
+    
+    t_adjusted = t - delay
+    if t_adjusted < 0 or t_adjusted > duration:
+        return 0.0
+    
+    # Quick attack, slow decay
+    progress = t_adjusted / duration
+    return (1.0 - progress) * (1.0 - progress)
+
+
+# ──────────────────────────── PROCEDURAL MOVE ────────────────────────────
+
+class ProceduralMove(Move):
+    """Enhanced procedural motion with position offsets, waveforms, transients, and phrases."""
+    
+    MOTION_TYPES = [
+        "oscillate", "wave", "bounce", "spiral", "sway", "nod", "shake", "figure8",
+        # New types
+        "peek", "recoil", "chicken", "dizzy", "yeah", "groove"
+    ]
+    
+    def __init__(
+        self,
+        motion_type: str = "wave",
+        duration: float = 3.0,
+        # Rotation amplitudes (degrees)
+        pitch_amplitude: float = 15.0,
+        yaw_amplitude: float = 20.0,
+        roll_amplitude: float = 5.0,
+        antenna_amplitude: float = 30.0,
+        body_yaw_amplitude: float = 0.0,
+        # Position offsets (meters) - oscillating
+        x_offset_amplitude: float = 0.0,  # forward/back oscillation
+        y_offset_amplitude: float = 0.0,  # left/right oscillation
+        z_offset_amplitude: float = 0.0,  # up/down oscillation
+        # Position drift (meters) - gradual movement over duration
+        x_drift: float = 0.0,  # total forward/back movement
+        y_drift: float = 0.0,  # total left/right movement  
+        z_drift: float = 0.0,  # total up/down movement (positive = rise)
+        # Timing
+        tempo: float = 1.0,
+        # Style
+        intensity: float = 1.0,
+        waveform: str = "sin",  # NEW: sin, triangle, square, sawtooth
+        phase_offset: float = 0.0,
+        # Transient (impulse) settings - NEW!
+        transient_enabled: bool = False,
+        transient_repeat: float = 0.5,  # seconds between impulses
+    ):
+        self.motion_type = motion_type
+        self._duration = duration
+        
+        # Apply intensity scaling
+        self.pitch_amp = np.deg2rad(pitch_amplitude * intensity)
+        self.yaw_amp = np.deg2rad(yaw_amplitude * intensity)
+        self.roll_amp = np.deg2rad(roll_amplitude * intensity)
+        self.antenna_amp = np.deg2rad(antenna_amplitude * intensity)
+        self.body_yaw_amp = np.deg2rad(body_yaw_amplitude * intensity)
+        
+        # Position amplitudes (meters) - oscillation
+        self.x_amp = x_offset_amplitude * intensity
+        self.y_amp = y_offset_amplitude * intensity
+        self.z_amp = z_offset_amplitude * intensity
+        
+        # Position drift (gradual movement over duration)
+        self.x_drift = x_drift
+        self.y_drift = y_drift
+        self.z_drift = z_drift
+        
+        self.tempo = tempo
+        self.phase_offset = phase_offset
+        self.waveform_fn = WAVEFORMS.get(waveform, waveform_sin)
+        
+        # Transient settings
+        self.transient_enabled = transient_enabled
+        self.transient_repeat = transient_repeat
+        
+    @property
+    def duration(self) -> float:
+        return self._duration
+        
+    def evaluate(self, t: float) -> Tuple[np.ndarray, np.ndarray, float]:
+        """Generate pose at time t."""
+        # Base phase
+        phase = 2.0 * np.pi * self.tempo * t + self.phase_offset
+        
+        # Get motion values
+        motion_fn = getattr(self, f"_motion_{self.motion_type}", self._motion_wave)
+        pitch, yaw, roll, ant_l, ant_r, body, x_off, y_off, z_off = motion_fn(t, phase)
+        
+        # Apply transient modifier if enabled
+        if self.transient_enabled:
+            impulse = transient_impulse(t, 0.2, 0.0, self.transient_repeat)
+            pitch *= (1 + impulse * 0.5)
+            yaw *= (1 + impulse * 0.5)
+        
+        # Create head pose with position offset
+        # Note: create_head_pose builds a 4x4 matrix, we add translation
+        head = create_head_pose(
+            roll=np.rad2deg(roll),
+            pitch=np.rad2deg(pitch),
+            yaw=np.rad2deg(yaw),
+            degrees=True
+        )
+        
+        # Add position offset (oscillation) to the transformation matrix
+        head[0, 3] += x_off
+        head[1, 3] += y_off
+        head[2, 3] += z_off
+        
+        # Add position drift (linear interpolation over duration)
+        progress = t / self._duration if self._duration > 0 else 0
+        progress = min(1.0, max(0.0, progress))  # clamp to 0-1
+        head[0, 3] += self.x_drift * progress
+        head[1, 3] += self.y_drift * progress
+        head[2, 3] += self.z_drift * progress
+        
+        antennas = np.array([ant_l, ant_r])
+        return head, antennas, body
+    
+    # ──────────────────────── MOTION PATTERNS ────────────────────────
+    
+    def _motion_oscillate(self, t: float, phase: float):
+        """Synchronized oscillation."""
+        s = self.waveform_fn(phase)
+        return (
+            self.pitch_amp * s, self.yaw_amp * s, self.roll_amp * s,
+            self.antenna_amp * s, -self.antenna_amp * s, self.body_yaw_amp * s,
+            self.x_amp * s, self.y_amp * s, self.z_amp * s
+        )
+    
+    def _motion_wave(self, t: float, phase: float):
+        """Wave with phase offsets."""
+        return (
+            self.pitch_amp * self.waveform_fn(phase),
+            self.yaw_amp * self.waveform_fn(phase + np.pi/4),
+            self.roll_amp * self.waveform_fn(phase + np.pi/2),
+            self.antenna_amp * self.waveform_fn(phase + np.pi/3),
+            self.antenna_amp * self.waveform_fn(phase - np.pi/3),
+            self.body_yaw_amp * self.waveform_fn(phase + np.pi/6),
+            self.x_amp * self.waveform_fn(phase),
+            self.y_amp * self.waveform_fn(phase + np.pi/4),
+            self.z_amp * self.waveform_fn(phase + np.pi/2)
+        )
+    
+    def _motion_bounce(self, t: float, phase: float):
+        """Bouncy up-down."""
+        bounce = abs(self.waveform_fn(phase))
+        side = self.waveform_fn(phase * 2)
+        return (
+            -self.pitch_amp * bounce, self.yaw_amp * side * 0.3, self.roll_amp * side * 0.2,
+            self.antenna_amp * bounce, self.antenna_amp * bounce, 0.0,
+            0.0, 0.0, self.z_amp * bounce
+        )
+    
+    def _motion_spiral(self, t: float, phase: float):
+        """Circular pattern."""
+        return (
+            self.pitch_amp * np.sin(phase), self.yaw_amp * np.cos(phase),
+            self.roll_amp * np.sin(phase * 2) * 0.5,
+            self.antenna_amp * np.sin(phase + np.pi/2), self.antenna_amp * np.sin(phase - np.pi/2),
+            self.body_yaw_amp * np.cos(phase),
+            self.x_amp * np.cos(phase), self.y_amp * np.sin(phase), 0.0
+        )
+    
+    def _motion_sway(self, t: float, phase: float):
+        """Gentle swaying."""
+        slow = self.waveform_fn(phase * 0.5)
+        fast = self.waveform_fn(phase * 1.5)
+        return (
+            self.pitch_amp * 0.3 * slow, self.yaw_amp * slow, self.roll_amp * 0.5 * fast,
+            self.antenna_amp * fast * 0.5, -self.antenna_amp * fast * 0.5,
+            self.body_yaw_amp * slow * 0.3,
+            0.0, self.y_amp * slow, 0.0
+        )
+    
+    def _motion_nod(self, t: float, phase: float):
+        """Nodding (yes)."""
+        nod = self.waveform_fn(phase)
+        return (
+            self.pitch_amp * nod, self.yaw_amp * 0.1 * nod, 0.0,
+            self.antenna_amp * 0.5 * abs(nod), self.antenna_amp * 0.5 * abs(nod), 0.0,
+            0.0, 0.0, self.z_amp * 0.5 * abs(nod)
+        )
+    
+    def _motion_shake(self, t: float, phase: float):
+        """Head shake (no)."""
+        shake = self.waveform_fn(phase)
+        return (
+            self.pitch_amp * 0.1 * shake, self.yaw_amp * shake, self.roll_amp * 0.3 * shake,
+            self.antenna_amp * shake, -self.antenna_amp * shake, 0.0,
+            0.0, 0.0, 0.0
+        )
+    
+    def _motion_figure8(self, t: float, phase: float):
+        """Figure-8 pattern."""
+        return (
+            self.pitch_amp * np.sin(phase * 2), self.yaw_amp * np.sin(phase),
+            self.roll_amp * np.cos(phase * 2) * 0.5,
+            self.antenna_amp * np.sin(phase + np.pi/4), self.antenna_amp * np.sin(phase - np.pi/4),
+            self.body_yaw_amp * np.sin(phase) * 0.5,
+            self.x_amp * np.sin(phase), self.y_amp * np.sin(phase * 2), 0.0
+        )
+    
+    # ──────────────────────── NEW MOTION TYPES ────────────────────────
+    
+    def _motion_peek(self, t: float, phase: float):
+        """Peekaboo - duck down then peek side to side."""
+        period = self._duration
+        t_norm = (t % period) / period
+        
+        # 5 phases: duck, peek-left, duck, peek-right, rise
+        if t_norm < 0.2:
+            # Ducking down
+            prog = ease_smooth(t_norm / 0.2)
+            return (0, 0, 0, 0, 0, 0, 0, 0, -self.z_amp * prog)
+        elif t_norm < 0.4:
+            # Peek left
+            prog = ease_smooth((t_norm - 0.2) / 0.2)
+            return (
+                -self.pitch_amp * 0.3, self.yaw_amp * prog, 0,
+                self.antenna_amp * prog, 0, 0,
+                0, self.y_amp * prog, -self.z_amp * (1 - prog * 0.5)
+            )
+        elif t_norm < 0.6:
+            # Back to center
+            prog = ease_smooth((t_norm - 0.4) / 0.2)
+            return (
+                -self.pitch_amp * 0.3 * (1-prog), self.yaw_amp * (1-prog), 0,
+                self.antenna_amp * (1-prog), 0, 0,
+                0, self.y_amp * (1-prog), -self.z_amp * 0.5
+            )
+        elif t_norm < 0.8:
+            # Peek right
+            prog = ease_smooth((t_norm - 0.6) / 0.2)
+            return (
+                -self.pitch_amp * 0.3, -self.yaw_amp * prog, 0,
+                0, self.antenna_amp * prog, 0,
+                0, -self.y_amp * prog, -self.z_amp * 0.5
+            )
+        else:
+            # Rise back up
+            prog = ease_smooth((t_norm - 0.8) / 0.2)
+            return (0, -self.yaw_amp * (1-prog), 0, 0, 0, 0, 0, -self.y_amp * (1-prog), -self.z_amp * 0.5 * (1-prog))
+    
+    def _motion_recoil(self, t: float, phase: float):
+        """Quick backward recoil - surprise reaction."""
+        impulse = transient_impulse(t, 0.3, 0.0, self.transient_repeat or 1.0)
+        return (
+            -self.pitch_amp * impulse, 0, 0,
+            self.antenna_amp * impulse, self.antenna_amp * impulse, 0,
+            -self.x_amp * impulse, 0, 0
+        )
+    
+    def _motion_chicken(self, t: float, phase: float):
+        """Chicken peck - forward thrust."""
+        impulse = transient_impulse(t, 0.3, 0.0, 1.0 / self.tempo if self.tempo > 0 else 1.0)
+        return (
+            self.pitch_amp * impulse, 0, 0,
+            self.antenna_amp * impulse, self.antenna_amp * impulse, 0,
+            self.x_amp * impulse, 0, 0
+        )
+    
+    def _motion_dizzy(self, t: float, phase: float):
+        """Dizzy circular motion."""
+        return (
+            self.pitch_amp * np.sin(phase),
+            self.yaw_amp * np.cos(phase),
+            self.roll_amp * np.sin(phase),
+            self.antenna_amp * np.sin(phase * 2),
+            -self.antenna_amp * np.sin(phase * 2),
+            0,
+            self.x_amp * np.cos(phase), self.y_amp * np.sin(phase), 0
+        )
+    
+    def _motion_yeah(self, t: float, phase: float):
+        """Emphatic 'yeah' double-nod."""
+        period = 1.0 / self.tempo if self.tempo > 0 else 1.0
+        t_in_period = t % period
+        
+        # Two quick nods
+        nod1 = transient_impulse(t_in_period, period * 0.3, 0, 0)
+        nod2 = transient_impulse(t_in_period, period * 0.2, period * 0.4, 0)
+        nod = nod1 + nod2 * 0.7
+        
+        return (
+            self.pitch_amp * nod, 0, 0,
+            self.antenna_amp * nod, self.antenna_amp * nod, 0,
+            0, 0, 0
+        )
+    
+    def _motion_groove(self, t: float, phase: float):
+        """Groovy sway with roll."""
+        sway = self.waveform_fn(phase)
+        roll = self.waveform_fn(phase + np.pi/4)
+        return (
+            self.pitch_amp * 0.3 * sway, 0, self.roll_amp * roll,
+            self.antenna_amp * sway, -self.antenna_amp * sway, self.body_yaw_amp * sway,
+            0, self.y_amp * sway, 0
+        )

reachy_mini_danceml/realtime_handler.py

@@ -0,0 +1,413 @@
+"""OpenAI Realtime API handler for voice-controlled Reachy Mini.
+
+Integrates OpenAI Realtime API with fastrtc for bidirectional voice
+conversations with Reachy Mini.
+"""
+
+import asyncio
+import threading
+import json
+import base64
+from typing import Optional, AsyncGenerator
+import numpy as np
+
+from .movement_tools import ALL_TOOLS, KeyFrame
+from .movement_generator import MovementGenerator
+
+
+# System instructions for the AI
+SYSTEM_INSTRUCTIONS = """You are Reachy, a friendly and expressive robot companion with a movable head and antennas.
+
+## Tool Selection Guide
+
+### 🎯 SIMPLE POSITIONING (use `goto_pose`)
+- "Look left/right/up/down"
+- "Turn your body"
+→ Call `goto_pose(pitch, yaw, roll, body_yaw, duration)`
+
+### 🎭 LIBRARY MOVES (use `search_moves` → `play_move`) **PREFERRED!**
+- "Show me happy/sad/scared"
+- "Do a dance"
+→ Call `search_moves(query)` first, then `play_move(name)`
+
+### 🌊 PROCEDURAL MOTION (use `generate_motion`) **SMOOTH & EXPRESSIVE!**
+When library has nothing, use procedural motion instead of keyframes!
+
+**Basic Motions:**
+- `nod` - Yes motion (pitch dominant)
+- `shake` - No motion (yaw dominant)  
+- `wave` - Flowing, natural (phase offsets)
+- `bounce` - Energetic, playful
+- `spiral` - Circular, hypnotic
+- `sway` - Gentle, idle-like
+- `figure8` - Playful loops
+- `oscillate` - Simple back and forth
+
+**Expressive Motions:**
+- `peek` - Peekaboo, duck down and peek side to side
+- `recoil` - Surprise/shock reaction, quick pullback
+- `chicken` - Pecking motion, head bobs forward
+- `dizzy` - Wobbly circles like confused/dizzy
+- `yeah` - Emphatic nodding, excited agreement
+- `groove` - Funky dance groove with attitude
+
+**Body Rotation (swivel the base):**
+- `body_yaw_amplitude` (degrees, 0-45) - Oscillate body left/right. USE THIS FOR DANCING!
+
+**Position Offsets (oscillating head movement):**
+- `x_offset_amplitude` (meters, max 0.02) - Oscillate head FORWARD/BACK
+- `y_offset_amplitude` (meters, max 0.02) - Oscillate head LEFT/RIGHT
+- `z_offset_amplitude` (meters, max 0.02) - Oscillate head UP/DOWN
+
+**Position Drift (gradual movement over duration):**
+- `z_drift` (meters, max ±0.02) - Rise UP (+) or sink DOWN (-) over duration. Use for snake charmer rising!
+- `x_drift` (meters, max ±0.02) - Move FORWARD (+) or BACK (-) over duration
+- `y_drift` (meters, max ±0.02) - Move LEFT or RIGHT over duration
+
+Example snake charmer rising: `generate_motion(motion_type="spiral", z_drift=0.02, duration=5)`
+Example sinking sad: `generate_motion(motion_type="sway", z_drift=-0.02, duration=4)`
+
+### 🛑 STOP (use `stop_movement`)
+- "Stop", "Freeze"
+
+## Physical Conventions
+- **Pitch**: NEGATIVE = look UP, POSITIVE = look DOWN
+- Head: roll ±30°, pitch ±30°, yaw ±45°
+- Body: yaw ±45°
+- Antennas: ±60° each
+
+## Personality
+Be friendly, brief. ALWAYS SPEAK IN ENGLISH.
+"""
+
+
+class RealtimeHandler:
+    """Handles OpenAI Realtime API connections for voice control.
+    
+    This class manages:
+    - WebSocket connections to OpenAI Realtime API
+    - Audio streaming (input and output)
+    - Tool calling for movement execution
+    """
+    
+    def __init__(self, openai_key: str, movement_generator: MovementGenerator, audio_device_name: Optional[str] = None):
+        """Initialize the realtime handler.
+        
+        Args:
+            openai_key: OpenAI API key with Realtime API access.
+            movement_generator: MovementGenerator instance for executing moves.
+            audio_device_name: Optional name of the audio device to use for output.
+        """
+        self.openai_key = openai_key
+        self.generator = movement_generator
+        self.audio_device_name = audio_device_name
+        self._connection = None
+        
+        # Listening state - controlled by UI button
+        self.is_listening = False
+        self._on_listening_change = None  # Callback for UI updates
+    
+    def set_listening(self, listening: bool):
+        """Set listening state and notify callback."""
+        self.is_listening = listening
+        print(f"🎤 Listening: {'ON' if listening else 'OFF'}")
+        if self._on_listening_change:
+            self._on_listening_change(listening)
+    
+    def toggle_listening(self):
+        """Toggle listening state."""
+        self.set_listening(not self.is_listening)
+
+    async def handle_tool_call(self, name: str, arguments: dict) -> str:
+        """Execute a tool call and return the result.
+        
+        Args:
+            name: Name of the tool to call.
+            arguments: Arguments for the tool.
+            
+        Returns:
+            String result describing what happened.
+        """
+        print(f"\n{'='*60}")
+        print(f"🔧 TOOL CALL: {name}")
+        print(f"   Arguments: {json.dumps(arguments, indent=2)}")
+        print(f"{'='*60}")
+        
+        try:
+            if name == "goto_pose":
+                roll = arguments.get("roll", 0)
+                pitch = arguments.get("pitch", 0)
+                yaw = arguments.get("yaw", 0)
+                body_yaw = arguments.get("body_yaw", 0)
+                duration = arguments.get("duration", 0.5)
+                
+                print(f"   🎯 Moving to: head(roll={roll}°, pitch={pitch}°, yaw={yaw}°), body_yaw={body_yaw}° over {duration}s")
+                await self.generator.goto_pose(
+                    roll=roll, pitch=pitch, yaw=yaw, body_yaw=body_yaw, duration=duration
+                )
+                result = f"Moved head to roll={roll}°, pitch={pitch}°, yaw={yaw}°, body_yaw={body_yaw}°"
+                print(f"   ✅ {result}")
+                return result
+                
+            elif name == "generate_motion":
+                from .procedural_motion import ProceduralMove
+                
+                motion_type = arguments.get("motion_type", "wave")
+                duration = arguments.get("duration", 3.0)
+                pitch_amp = arguments.get("pitch_amplitude", 15)
+                yaw_amp = arguments.get("yaw_amplitude", 20)
+                roll_amp = arguments.get("roll_amplitude", 5)
+                antenna_amp = arguments.get("antenna_amplitude", 30)
+                body_yaw_amp = arguments.get("body_yaw_amplitude", 0)
+                x_offset = arguments.get("x_offset_amplitude", 0)
+                y_offset = arguments.get("y_offset_amplitude", 0)
+                z_offset = arguments.get("z_offset_amplitude", 0)
+                tempo = arguments.get("tempo", 1.0)
+                intensity = arguments.get("intensity", 1.0)
+                waveform = arguments.get("waveform", "sin")
+                transient = arguments.get("transient_enabled", False)
+                
+                print(f"   🌊 Generating {motion_type} motion: {duration}s, tempo={tempo}, waveform={waveform}")
+                print(f"      Rotation: pitch={pitch_amp}°, yaw={yaw_amp}°, roll={roll_amp}°")
+                if body_yaw_amp:
+                    print(f"      Body swivel: ±{body_yaw_amp}°")
+                if x_offset or y_offset or z_offset:
+                    print(f"      Oscillation: x={x_offset}m, y={y_offset}m, z={z_offset}m")
+                
+                # Drift parameters (gradual movement)
+                x_drift = arguments.get("x_drift", 0)
+                y_drift = arguments.get("y_drift", 0)
+                z_drift = arguments.get("z_drift", 0)
+                if x_drift or y_drift or z_drift:
+                    print(f"      Drift: x={x_drift}m, y={y_drift}m, z={z_drift}m")
+                
+                move = ProceduralMove(
+                    motion_type=motion_type,
+                    duration=duration,
+                    pitch_amplitude=pitch_amp,
+                    yaw_amplitude=yaw_amp,
+                    roll_amplitude=roll_amp,
+                    antenna_amplitude=antenna_amp,
+                    body_yaw_amplitude=body_yaw_amp,
+                    x_offset_amplitude=x_offset,
+                    y_offset_amplitude=y_offset,
+                    z_offset_amplitude=z_offset,
+                    x_drift=x_drift,
+                    y_drift=y_drift,
+                    z_drift=z_drift,
+                    tempo=tempo,
+                    intensity=intensity,
+                    waveform=waveform,
+                    transient_enabled=transient
+                )
+                
+                # Queue to motor thread
+                self.generator.queue_move(move)
+                self.generator.wait_for_move(timeout=duration + 2.0)
+                
+                result = f"Played {motion_type} motion for {duration}s"
+                print(f"   ✅ {result}")
+                return result
+                
+            elif name == "stop_movement":
+                print(f"   🛑 Stopping all movement")
+                await self.generator.stop()
+                result = "Stopped movement, returned to neutral"
+                print(f"   ✅ {result}")
+                return result
+
+            elif name == "play_move":
+                move_name = arguments.get("name")
+                print(f"   🎬 Playing library move: '{move_name}'")
+                if not move_name:
+                    return "Error: Name required"
+                result = await self.generator.play_library_move(move_name)
+                print(f"   ✅ {result}")
+                return result
+
+            elif name == "search_moves":
+                query = arguments.get("query", "")
+                print(f"   🔍 Searching library for: '{query}'")
+                if not query:
+                    return "Error: Query required"
+                
+                # Access the library directly from the generator
+                if hasattr(self.generator, "_move_library"):
+                    results = self.generator._move_library.search_moves(query)
+                    if not results:
+                        print(f"   ❌ No moves found")
+                        return f"No moves found for '{query}'"
+                    
+                    # Format results for the agent
+                    print(f"   📚 Found {len(results)} moves:")
+                    response = "Found movements:\n"
+                    for r in results:
+                        print(f"      - {r['name']} ({r['source']})")
+                        response += f"- {r['name']} ({r['source']}): {r['description'][:100]}...\n"
+                    return response
+                else:
+                    print(f"   ⚠️ Move library not available")
+                    return "Error: Move library not available"
+
+            elif name == "get_choreography_guide":
+                print(f"   📖 Loading choreography guide")
+                try:
+                    with open("docs/CHOREOGRAPHY_GUIDE.md", "r") as f:
+                        content = f.read()
+                        print(f"   ✅ Loaded guide ({len(content)} bytes)")
+                        return content
+                except FileNotFoundError:
+                    print(f"   ❌ Guide not found")
+                    return "Error: Choreography guide not found at docs/CHOREOGRAPHY_GUIDE.md"
+                
+            else:
+                print(f"   ❓ Unknown tool: {name}")
+                return f"Unknown tool: {name}"
+                
+        except Exception as e:
+            print(f"   ❌ Error: {str(e)}")
+            return f"Error executing {name}: {str(e)}"
+
+    async def run_local_session(self, audio_queue: asyncio.Queue, stop_event: Optional[threading.Event] = None):
+        """Run a persistent session with local audio input/output.
+        
+        Uses OpenAI's server-side VAD.
+        
+        Args:
+            audio_queue: Queue receiving raw PCM16 audio bytes from microphone.
+            stop_event: Optional event to signal cancellation.
+        """
+        from openai import AsyncOpenAI
+        import sounddevice as sd
+        
+        client = AsyncOpenAI(api_key=self.openai_key)
+        
+        print("Connecting to OpenAI Realtime API (Local Mode)...")
+        
+        # Find output device index if specified
+        output_device_index = None
+        if self.audio_device_name:
+            try:
+                devices = sd.query_devices()
+                for i, dev in enumerate(devices):
+                    if self.audio_device_name in dev['name'] and dev['max_output_channels'] > 0:
+                        output_device_index = i
+                        print(f"Selected output device '{self.audio_device_name}' (Index: {i})")
+                        break
+                if output_device_index is None:
+                     print(f"Warning: Audio output device '{self.audio_device_name}' not found. Using default.")
+            except Exception as e:
+                print(f"Error querying devices for output: {e}")
+
+        async with client.beta.realtime.connect(
+            model="gpt-realtime"
+        ) as conn:
+            print("Connected!")
+            
+            # Configure session with VAD
+            await conn.session.update(session={
+                "modalities": ["text", "audio"],
+                "instructions": SYSTEM_INSTRUCTIONS,
+                "tools": ALL_TOOLS,
+                "input_audio_format": "pcm16",
+                "output_audio_format": "pcm16",
+                "turn_detection": {
+                    "type": "server_vad",
+                    "threshold": 0.5,
+                    "prefix_padding_ms": 300,
+                    "silence_duration_ms": 500
+                }
+            })
+            
+            # Output stream setup (PyAudio/SoundDevice)
+            # We use a simple callback to play audio as it arrives
+            output_stream = sd.OutputStream(
+                samplerate=24000, 
+                channels=1, 
+                dtype=np.int16,
+                device=output_device_index
+            )
+            output_stream.start()
+
+            
+            # Task to stream mic input to OpenAI
+            async def send_mic_audio():
+                import queue as sync_queue
+                chunk_count = 0
+                try:
+                    while True:
+                        if stop_event and stop_event.is_set():
+                            break
+                            
+                        try:
+                            chunk = audio_queue.get(timeout=0.05)
+                            
+                            # Only send audio when listening is enabled
+                            if not self.is_listening:
+                                continue
+                            
+                            # Check for silence (all zeros)
+                            if chunk_count % 50 == 0:
+                                max_amp = np.frombuffer(chunk, dtype=np.int16).max() if len(chunk) > 0 else 0
+                                print(f"Audio chunks sent: {chunk_count} | Max Amp: {max_amp}")
+                                
+                            chunk_count += 1
+                            
+                            # Base64 encode
+                            b64_chunk = base64.b64encode(chunk).decode()
+                            await conn.input_audio_buffer.append(audio=b64_chunk)
+                        except sync_queue.Empty:
+                            await asyncio.sleep(0.01)
+                except asyncio.CancelledError:
+                    print(f"Audio send cancelled after {chunk_count} chunks")
+                except Exception as e:
+                    print(f"Error sending audio: {e}")
+
+            send_task = asyncio.create_task(send_mic_audio())
+            
+            # Start motor control thread (runs synchronously for smooth 50Hz)
+            self.generator.start_motor_thread(stop_event)
+            
+            try:
+                # Process events from OpenAI
+                async for event in conn:
+                    # Only log important events (skip noisy deltas)
+                    
+                    if event.type == "response.audio.delta":
+                        if event.delta:
+                            audio_bytes = base64.b64decode(event.delta)
+                            # Write to output stream (blocking call, but short)
+                            output_stream.write(np.frombuffer(audio_bytes, dtype=np.int16))
+                            
+                    elif event.type == "input_audio_buffer.speech_started":
+                        print(">>> User started speaking!")
+                        
+                    elif event.type == "input_audio_buffer.speech_stopped":
+                        print(">>> User stopped speaking!")
+                        
+                    elif event.type == "error":
+                        print(f">>> OpenAI Error: {event}")
+                        
+                    elif event.type == "response.function_call_arguments.done":
+                        print(f"Tool call: {event.name}")
+                        try:
+                            args = json.loads(event.arguments)
+                        except:
+                            args = {}
+                        
+                        result = await self.handle_tool_call(event.name, args)
+                        
+                        await conn.conversation.item.create(item={
+                            "type": "function_call_output",
+                            "call_id": event.call_id,
+                            "output": result
+                        })
+                        await conn.response.create()
+                        
+            except Exception as e:
+                print(f"Session error: {e}")
+            finally:
+                send_task.cancel()
+                # Motor thread will stop via stop_event
+                output_stream.stop()
+                output_stream.close()

tests/__init__.py

@@ -0,0 +1 @@
+# Test package

tests/check_device_details.py

@@ -0,0 +1,13 @@
+
+import sounddevice as sd
+
+print(f"Default input device: {sd.default.device[0]}")
+print("Searching for 'Reachy Mini Audio'...")
+devices = sd.query_devices()
+for i, dev in enumerate(devices):
+    if "Reachy Mini Audio" in dev['name']:
+        print(f"\nFOUND DEVICE {i}: {dev['name']}")
+        print(f"  Max Input Channels: {dev['max_input_channels']}")
+        print(f"  Default Sample Rate: {dev['default_samplerate']}")
+        print(f"  Host API: {dev['hostapi']}")
+        print(f"  Full Info: {dev}")

tests/check_keyframes.py

@@ -0,0 +1,22 @@
+
+"""Check keyframe counts in library moves."""
+import sys
+sys.path.append(".")
+
+from reachy_mini_danceml.dataset_loader import MoveLibrary
+
+lib = MoveLibrary()
+lib.load()
+
+# Check a few moves
+sample_moves = ["a_firm_categorical_no", "a_robotic_grid_snapping", "you_look_around_use"]
+
+for name in sample_moves:
+    record = lib.get_move(name)
+    if record:
+        print(f"\n{name}:")
+        print(f"  Keyframes: {len(record.keyframes)}")
+        print(f"  Duration: {record.keyframes[-1].t:.2f}s")
+        print(f"  Rate: {len(record.keyframes) / record.keyframes[-1].t:.1f} keyframes/sec")
+    else:
+        print(f"\n{name}: NOT FOUND")

tests/test_mic.py

@@ -0,0 +1,43 @@
+import sounddevice as sd
+import numpy as np
+import time
+import queue
+
+def test_microphone(duration=3):
+    devices = sd.query_devices()
+    print("\nScanning all input devices for signal...\n")
+    
+    for i, dev in enumerate(devices):
+        if dev['max_input_channels'] > 0:
+            print(f"Testing Device {i}: {dev['name']}")
+            try:
+                q = queue.Queue()
+                def callback(indata, frames, time, status):
+                    q.put(indata.copy())
+
+                # Use device specific sample rate to avoid mismatches
+                sr = int(dev['default_samplerate'])
+                
+                with sd.InputStream(device=i, samplerate=sr, channels=1, dtype=np.int16, callback=callback):
+                    start = time.time()
+                    max_amp = 0
+                    while time.time() - start < 1.0: # Test for 1 second
+                        try:
+                            data = q.get(timeout=0.1)
+                            current_max = np.max(np.abs(data))
+                            max_amp = max(max_amp, current_max)
+                        except queue.Empty:
+                            pass
+                            
+                    print(f"  -> Max Amplitude: {max_amp}")
+                    if max_amp > 100:
+                        print("  ✅ SIGNAL DETECTED!")
+                    else:
+                        print("  ❌ No Signal (Silence)")
+                        
+            except Exception as e:
+                print(f"  -> Error: {e}")
+            print("-" * 30)
+
+if __name__ == "__main__":
+    test_microphone()

tests/test_movement_generator.py

@@ -0,0 +1,164 @@
+"""Unit tests for movement generator."""
+
+import pytest
+import numpy as np
+from reachy_mini_danceml.movement_tools import KeyFrame
+from reachy_mini_danceml.movement_generator import GeneratedMove
+
+
+class TestKeyFrame:
+    """Tests for KeyFrame dataclass."""
+    
+    def test_from_dict_full(self):
+        """Test KeyFrame.from_dict with all fields."""
+        data = {
+            "t": 1.5,
+            "head": {"roll": 10, "pitch": -5, "yaw": 30},
+            "antennas": [15, -15]
+        }
+        kf = KeyFrame.from_dict(data)
+        
+        assert kf.t == 1.5
+        assert kf.head["roll"] == 10
+        assert kf.head["pitch"] == -5
+        assert kf.head["yaw"] == 30
+        assert kf.antennas == (15, -15)
+        
+    def test_from_dict_minimal(self):
+        """Test KeyFrame.from_dict with minimal fields."""
+        data = {"t": 0.5}
+        kf = KeyFrame.from_dict(data)
+        
+        assert kf.t == 0.5
+        assert kf.head == {}
+        assert kf.antennas == (0, 0)
+        
+    def test_from_dict_defaults(self):
+        """Test KeyFrame.from_dict uses defaults for missing fields."""
+        data = {}
+        kf = KeyFrame.from_dict(data)
+        
+        assert kf.t == 0
+        assert kf.head == {}
+        assert kf.antennas == (0, 0)
+
+
+class TestGeneratedMove:
+    """Tests for GeneratedMove class."""
+    
+    def test_requires_minimum_keyframes(self):
+        """Test that at least 2 keyframes are required."""
+        with pytest.raises(ValueError, match="at least 2 keyframes"):
+            GeneratedMove([KeyFrame(t=0, head={})])
+            
+    def test_duration(self):
+        """Test that duration equals max keyframe time."""
+        keyframes = [
+            KeyFrame(t=0.0, head={"yaw": 0}),
+            KeyFrame(t=1.5, head={"yaw": 30}),
+            KeyFrame(t=3.0, head={"yaw": 0}),
+        ]
+        move = GeneratedMove(keyframes)
+        assert move.duration == 3.0
+        
+    def test_evaluate_returns_correct_types(self):
+        """Test that evaluate returns correct data types."""
+        keyframes = [
+            KeyFrame(t=0.0, head={"roll": 0, "pitch": 0, "yaw": 0}, antennas=(0, 0)),
+            KeyFrame(t=1.0, head={"roll": 10, "pitch": 10, "yaw": 30}, antennas=(20, -20)),
+        ]
+        move = GeneratedMove(keyframes)
+        head, antennas, body_yaw = move.evaluate(0.5)
+        
+        # Head should be 4x4 matrix
+        assert head.shape == (4, 4)
+        # Antennas should be array of 2
+        assert len(antennas) == 2
+        assert isinstance(antennas, np.ndarray)
+        # Body yaw should be 0
+        assert body_yaw == 0.0
+        
+    def test_evaluate_at_boundaries(self):
+        """Test evaluation at start and end times."""
+        keyframes = [
+            KeyFrame(t=0.0, head={"yaw": 0}),
+            KeyFrame(t=1.0, head={"yaw": 30}),
+        ]
+        move = GeneratedMove(keyframes)
+        
+        # Should not raise at boundaries
+        head_start, _, _ = move.evaluate(0.0)
+        head_end, _, _ = move.evaluate(1.0)
+        
+        assert head_start is not None
+        assert head_end is not None
+        
+    def test_evaluate_clamps_time(self):
+        """Test that evaluation clamps time to valid range."""
+        keyframes = [
+            KeyFrame(t=0.0, head={"yaw": 0}),
+            KeyFrame(t=1.0, head={"yaw": 30}),
+        ]
+        move = GeneratedMove(keyframes)
+        
+        # Should not raise for out-of-range times
+        head_before, _, _ = move.evaluate(-1.0)
+        head_after, _, _ = move.evaluate(5.0)
+        
+        assert head_before is not None
+        assert head_after is not None
+        
+    def test_interpolation_midpoint(self):
+        """Test that interpolation produces reasonable midpoint values."""
+        keyframes = [
+            KeyFrame(t=0.0, head={"yaw": 0}, antennas=(0, 0)),
+            KeyFrame(t=1.0, head={"yaw": 30}, antennas=(30, -30)),
+        ]
+        move = GeneratedMove(keyframes)
+        
+        # At midpoint, values should be roughly halfway
+        # (Cubic spline may vary slightly from linear midpoint)
+        _, antennas, _ = move.evaluate(0.5)
+        
+        # Convert back to degrees for comparison
+        left_deg = np.rad2deg(antennas[0])
+        right_deg = np.rad2deg(antennas[1])
+        
+        # Should be roughly 15 and -15 (within tolerance for cubic spline)
+        assert 10 < left_deg < 20
+        assert -20 < right_deg < -10
+
+
+class TestMovementToolSchemas:
+    """Tests for movement tool schemas."""
+    
+    def test_all_tools_have_required_fields(self):
+        """Test that all tool schemas have required fields."""
+        from reachy_mini_danceml.movement_tools import ALL_TOOLS
+        
+        for tool in ALL_TOOLS:
+            assert "type" in tool
+            assert tool["type"] == "function"
+            assert "name" in tool
+            
+    def test_goto_pose_schema(self):
+        """Test goto_pose tool schema structure."""
+        from reachy_mini_danceml.movement_tools import GOTO_POSE_TOOL
+        
+        assert GOTO_POSE_TOOL["name"] == "goto_pose"
+        params = GOTO_POSE_TOOL["parameters"]["properties"]
+        
+        assert "roll" in params
+        assert "pitch" in params
+        assert "yaw" in params
+        assert "duration" in params
+        
+    def test_create_sequence_schema(self):
+        """Test create_sequence tool schema structure."""
+        from reachy_mini_danceml.movement_tools import CREATE_SEQUENCE_TOOL
+        
+        assert CREATE_SEQUENCE_TOOL["name"] == "create_sequence"
+        params = CREATE_SEQUENCE_TOOL["parameters"]["properties"]
+        
+        assert "keyframes" in params
+        assert params["keyframes"]["type"] == "array"

tests/test_startup.py

@@ -0,0 +1,44 @@
+
+import asyncio
+import os
+import sys
+
+# Mock Reachy libraries to avoid full robot connection
+from unittest.mock import MagicMock
+sys.modules["reachy_mini"] = MagicMock()
+sys.modules["reachy_mini.utils"] = MagicMock()
+sys.modules["reachy_mini.motion"] = MagicMock()
+sys.modules["reachy_mini.motion.move"] = MagicMock()
+sys.modules["reachy_mini.io"] = MagicMock()
+sys.modules["reachy_mini.io.zenoh_client"] = MagicMock()
+
+from reachy_mini_danceml.realtime_handler import RealtimeHandler
+from reachy_mini_danceml.movement_generator import MovementGenerator
+
+# Mock Generator
+gen = MagicMock(spec=MovementGenerator)
+
+async def test_startup():
+    print("Testing RealtimeHandler initialization...")
+    try:
+        handler = RealtimeHandler("fake-key", gen)
+        print("Handler created.")
+        
+        print("Creating stream...")
+        # This calls run_until_complete internally
+        stream = handler.create_stream()
+        print("Stream created successfully.")
+        
+    except Exception as e:
+        print(f"CRASH: {e}")
+        import traceback
+        traceback.print_exc()
+
+if __name__ == "__main__":
+    try:
+        # Check if we can run the internal logic
+        handler = RealtimeHandler("fake-key", gen)
+        stream = handler.create_stream()
+        print("Sync execution success.")
+    except Exception as e:
+        print(f"Sync execution failed: {e}")

tests/verify_fix.py

@@ -0,0 +1,71 @@
+
+import sys
+import os
+import time
+import numpy as np
+
+# Ensure we can import the module
+sys.path.append(os.getcwd())
+
+from reachy_mini_danceml.audio_capture import LocalAudioCapture
+
+def verify_fix():
+    print("Testing Audio Capture with 'Reachy Mini Audio'...")
+    
+    # Initialize with the specific device name
+    capture = LocalAudioCapture(device_name="Reachy Mini Audio")
+    
+    # Check if the device index resulted in a valid index (not None)
+    # The class prints "Selected audio device..." or "Warning..."
+    # We can inspect the internal property
+    if capture._device_index is None:
+        print("❌ Verification Failed: Device 'Reachy Mini Audio' not found.")
+        return
+        
+    print(f"✅ Device found! Index: {capture._device_index}")
+    
+    capture.start()
+    time.sleep(1) # Let it warm up
+    
+    print("Capturing 2 seconds of audio...")
+    chunks = []
+    start_time = time.time()
+    while time.time() - start_time < 2:
+        chunk = capture.get_chunk()
+        if chunk:
+            chunks.append(chunk)
+            
+    capture.stop()
+    
+    total_chunks = len(chunks)
+    print(f"Total chunks captured: {total_chunks}")
+    
+    if total_chunks == 0:
+        print("❌ Verification Failed: No audio chunks captured.")
+        return
+
+    # Concatenate all chunks
+    all_audio = b"".join(chunks)
+    audio_array = np.frombuffer(all_audio, dtype=np.int16)
+    
+    max_amp = np.max(np.abs(audio_array))
+    print(f"Max Amplitude: {max_amp}")
+    
+    if max_amp < 100:
+        print("⚠️ Warning: Amplitude is very low (Silence?). Check mic mute.")
+    else:
+        print("✅ Signal Detected! Amplitude looks good.")
+        
+    # Check sample rate consistency
+    # 2 seconds * 24000 samples/sec = 48000 samples
+    expected_samples = 48000
+    actual_samples = len(audio_array)
+    print(f"Samples Captured: {actual_samples} (Expected ~{expected_samples})")
+    
+    if abs(actual_samples - expected_samples) > 10000:
+        print("⚠️ Warning: Sample count deviation is high. Check specific timing/buffer issues.")
+    else:
+        print("✅ Sample rate looks consistent.")
+
+if __name__ == "__main__":
+    verify_fix()

tests/verify_generative_mode.py

@@ -0,0 +1,41 @@
+import asyncio
+import os
+from reachy_mini_danceml.movement_generator import MovementGenerator
+from reachy_mini_danceml.realtime_handler import RealtimeHandler, SYSTEM_INSTRUCTIONS
+
+# Mock Generator
+class MockGenerator(MovementGenerator):
+    def __init__(self):
+        pass
+    async def stop(self):
+        pass
+
+async def test_generative_tools():
+    print("--- Testing Hybrid Generative Tools ---")
+    
+    # Init Handler
+    handler = RealtimeHandler(openai_key="fake-key", movement_generator=MockGenerator())
+    
+    # 1. Test get_choreography_guide
+    print("\n[Test] Calling 'get_choreography_guide'...")
+    # Emulate tool call
+    result = await handler.handle_tool_call("get_choreography_guide", {})
+    
+    if "Error" in result:
+        print(f"FAIL: {result}")
+        # Hint: check if we are running from root
+        print(f"CWD: {os.getcwd()}")
+    else:
+        print(f"SUCCESS: Retrieved guide ({len(result)} chars)")
+        print(f"Snippet: {result[:100]}...")
+        assert "Reachy Mini Choreography Guide" in result
+
+    # 2. Check System Instructions for Router Logic
+    print("\n[Test] Checking System Instructions...")
+    if "get_choreography_guide" in SYSTEM_INSTRUCTIONS:
+        print("SUCCESS: System instructions mention the guide tool.")
+    else:
+        print("FAIL: System instructions missing router logic reference.")
+
+if __name__ == "__main__":
+    asyncio.run(test_generative_tools())

tests/verify_smart_tools.py

@@ -0,0 +1,72 @@
+import asyncio
+from reachy_mini_danceml.movement_generator import MovementGenerator
+from reachy_mini_danceml.dataset_loader import MoveLibrary
+
+# Mock ReachyMini since we verify logic, not physical hardware
+class MockReachyMini:
+    async def async_play_move(self, move):
+        print(f"Mock: Playing move (duration={move.duration:.2f}s)")
+    
+    def goto_target(self, head=None, antennas=None, duration=0.0):
+        print("Mock: goto_target")
+
+async def test_smart_architecture():
+    print("--- 1. Testing MoveLibrary ---")
+    library = MoveLibrary()
+    library.load()
+    
+    moves = library.list_moves()
+    print(f"Loaded {len(moves)} moves.")
+    if not moves:
+        print("FAIL: No moves loaded!")
+        return
+
+    print(f"Sample move: {moves[0]}")
+
+    print("\n--- 2. Testing Search ---")
+    results = library.search_moves("happy")
+    print(f"Search 'happy' found {len(results)} results:")
+    for r in results[:3]:
+        print(f" - {r['name']}: {r['description'][:50]}...")
+
+    if not results:
+        print("WARN: Search 'happy' returned nothing.")
+
+    print("\n--- 3. Testing MovementGenerator Integration ---")
+    # Using a real name from the library or the first loaded one
+    test_move_name = results[0]['name'] if results else moves[0]
+    
+    reachy = MockReachyMini()
+    generator = MovementGenerator(reachy)
+    
+    print(f"Attempting to generate move: '{test_move_name}'")
+    # We call the synchronous version to verify retrieval and generation logic
+    # Note: The generator will reload the library internally if not shared, 
+    # but that's fine for testing. 
+    # In real app, we might want to share the instance or optimize loading.
+    
+    # We access the internal library to verify same state or just let it load
+    # For speed in test, let's inject our loaded library if possible, 
+    # but simpler to just let it load again (caching should handle it).
+    
+    # Actually, let's call play_move_by_name which returns the record
+    record = generator.play_move_by_name(test_move_name)
+    
+    if isinstance(record, str):
+        print(f"FAIL: Generator returned error: {record}")
+    else:
+        print(f"SUCCESS: Retrieved record for '{record.name}'")
+        print(f" - Source: {record.dataset_source}")
+        print(f" - Keyframes: {len(record.keyframes)}")
+        
+        # Test converting to Move
+        move = generator.create_from_keyframes(record.keyframes)
+        print(f" - Generated Move duration: {move.duration:.2f}s")
+        
+        # Test async play wrapper (simulating tool call)
+        print("\n--- 4. Testing Async Tool Wrapper ---")
+        result = await generator.play_library_move(test_move_name)
+        print(f"Tool Result: {result}")
+
+if __name__ == "__main__":
+    asyncio.run(test_smart_architecture())