# KAOS — Hand to MIDI CC

Low-latency hand-gesture → MIDI CC controller for Logic Pro on Apple Silicon.

```
Webcam → OpenCV (AVFoundation) → MediaPipe Hands → MIDI CC → IAC Driver → Logic Pro
```

---

## 1. Mac MIDI Setup (do this once)

### Enable IAC Driver

1. Open **Audio MIDI Setup** (`/Applications/Utilities/Audio MIDI Setup.app`)
2. In the menu bar: **Window → Show MIDI Studio**
3. Double-click **IAC Driver**
4. Tick **Device is online**
5. Click **+** to add a port — name it exactly: **`Vision MIDI`**
6. Click **Apply**

### Enable in Logic Pro

1. Logic Pro → **Settings → MIDI → Inputs**
2. Enable **IAC Driver — Vision MIDI**

---

## 2. Python environment

Requires **Python 3.11** (tested on M3 Max).

```bash
cd ~/Kaos
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

---

## 3. Run

```bash
source .venv/bin/activate
python main.py
```

Open **http://127.0.0.1:5000** in your browser.

Console output on start:
```
[MIDI] Available ports: ['Vision MIDI']
[MIDI] Opened 'Vision MIDI'
[Capture] 640×480 @ 60 fps
[KAOS] → http://127.0.0.1:5000
```

---

## 4. Calibrating the hand proxy

The app works uncalibrated (rough auto-scale), but calibration gives you the full 0–127 CC range:

1. Hold your hand **far from the camera** (flat, open) → click **Set MIN**
2. Bring your hand **close to the camera** (spread wide) → click **Set MAX**

The calibration indicator turns green when both are set.

---

## 5. Workflow in Logic Pro

1. Open the MIDI CC Assign panel for any knob/parameter (Right-click → **Learn MIDI CC Assignment**)
2. Move your hand — Logic will detect the CC coming in on CC 11 (or whichever you selected)
3. Click **Done**

Recommended: assign to a Smart Control mapped to Reverb mix, filter cutoff, expression, etc.

---

## 6. Performance tips (lowest latency)

| Setting | Recommendation |
|---|---|
| **Preview** | Keep OFF — saves ~5–10 ms per vision frame |
| **Smoothing α** | 0.10–0.20 for fast response; 0.35+ for glides |
| **Deadzone** | 1–2 to eliminate jitter without sluggishness |
| **Rate limit** | 60 Hz matches most DAW automation rates |
| **Curve** | ON for musical logarithmic feel; OFF for linear |
| **Camera** | Avoid OBS virtual cam; use the native webcam |
| **Resolution** | 640×480 is already set; do not raise it |
| **Background apps** | Close Chrome, Zoom, Electron apps |

---

## 7. Endpoints

| Method | Path | Description |
|---|---|---|
| GET | `/` | UI page |
| GET | `/events` | SSE stats stream (~20 Hz) |
| GET | `/video_feed` | MJPEG preview stream |
| POST | `/set_min` | Capture MIN calibration |
| POST | `/set_max` | Capture MAX calibration |
| POST | `/reset_calibration` | Clear calibration |
| POST | `/toggle_invert` | Flip CC direction |
| POST | `/toggle_curve` | Toggle x^1.6 curve |
| POST | `/set_params` | Update alpha, deadzone, rate_limit, cc_number, mute, preview_enabled |
| GET | `/health` | JSON state dump for debugging |

---

## 8. How the proxy works

MediaPipe gives 21 hand landmarks in normalized [0,1] image coordinates.
Four landmarks are used:

```
wrist (0) ─────────── middle_mcp (9)   →  height
index_mcp (5) ──────── pinky_mcp (17)  →  spread

proxy = spread × height
```

This proxy is proportional to the *area* of the palm visible to the camera — large when the hand is open and close, small when it's folded or far away. It's stable because it's scale-invariant within the frame.

The proxy is then:
1. Normalised against calibration min/max
2. Optionally inverted
3. Optionally shaped with an x^1.6 curve
4. EMA-smoothed
5. Hysteresis-gated (deadzone)
6. Rate-limited before sending as MIDI CC

---

## 9. Troubleshooting

**"Vision MIDI not found"**
→ Open Audio MIDI Setup, enable IAC Driver, add a port named `Vision MIDI` (exact spelling).

**Camera won't open**
→ System Settings → Privacy & Security → Camera → allow Terminal (or your IDE).

**High latency**
→ Disable Preview. Lower Smoothing α. Close background apps.

**CC jumps on hand first appearing**
→ Normal — EMA starts at 64 and converges in ~4 frames. Set α=0 to eliminate.

**No MIDI in Logic**
→ Logic Pro → Settings → MIDI → Inputs → enable IAC Driver — Vision MIDI.
