samios/docs/voice-integration-architecture.md

# SamiType × SamiOS Voice Integration Architecture

## Vision

Voice-first OS control — "Computer, open Firefox", "Computer, go to youtube.com", 
"Computer, volume up" — Star Trek computer style, powered by SamiType (Sami's 
custom speech-to-text engine).

## Architecture Overview

```
┌─────────────────────────────────────────────────┐
│                  SamiOS Desktop                  │
│                                                  │
│  ┌──────────┐   ┌──────────┐   ┌─────────────┐ │
│  │  Audio   │──▶│ SamiType │──▶│  Command    │ │
│  │  Capture │   │   STT    │   │  Router     │ │
│  └──────────┘   └──────────┘   └──────┬──────┘ │
│                                       │         │
│                    ┌──────────────────┼─────┐   │
│                    ▼                  ▼     ▼   │
│              ┌──────────┐  ┌──────┐ ┌──────┐  │
│              │  KDE/WM  │  │ Web  │ │ Sys  │  │
│              │  Control │  │ Nav  │ │ Ctrl │  │
│              └──────────┘  └──────┘ └──────┘  │
│                                                  │
│  ┌──────────────────────────────────────────┐   │
│  │        SamiType TTS (Voice Response)     │   │
│  └──────────────────────────────────────────┘   │
└─────────────────────────────────────────────────┘
```

## Components

### 1. Wake Word Engine
- **Trigger:** "Computer" (configurable)
- **Implementation:** Porcupine or openWakeWord (offline, low-latency)
- **Always listening** via systemd service, minimal CPU (< 2%)
- Push-to-talk fallback (keyboard shortcut or dedicated button)

### 2. SamiType STT (Speech-to-Text)
- Sami's existing speech-to-text engine
- **Offline-first** — models run locally, no cloud dependency
- **Streaming mode** — starts transcribing before user finishes speaking
- Outputs text + confidence score

### 3. Command Router
- Parses SamiType transcription → structured command
- Intent classification:
  - **Navigation:** "go to [website]", "open [app]", "search for [query]"
  - **System:** "volume [up/down/mute]", "brightness [N]", "screenshot", "lock screen"
  - **Window:** "minimize", "maximize", "close window", "switch to [app]"
  - **File:** "open [file]", "create folder [name]", "move [file] to [location]"
  - **Query:** "what time is it", "what's the weather", "system status"
- **Fallback:** If no command match → pass to web search or LLM

### 4. Execution Layer
- **KDE/WM Control:** kdialog, qdbus, kstart, wmctrl, xdotool
- **Web Navigation:** xdg-open, firefox --new-tab [url]
- **System Control:** pactl (volume), brightnessctl, systemctl, loginctl

### 5. Voice Response (TTS)
- SamiType TTS engine for spoken confirmations
- "Opening Firefox", "Volume set to 50%", "Screenshot saved"
- Subtle — not chatty, only speaks when action confirmed or error

## Command Examples

| Voice Input | Action |
|-------------|--------|
| "Computer, open Firefox" | Launch Firefox |
| "Computer, go to youtube.com" | Open youtube.com in default browser |
| "Computer, volume up" | Increase volume by 10% |
| "Computer, mute" | Mute audio |
| "Computer, take a screenshot" | spectacle -f (fullscreen capture) |
| "Computer, lock screen" | loginctl lock-session |
| "Computer, what time is it" | TTS: "It's 3:42 PM" |
| "Computer, minimize all windows" | Show desktop (Meta+D) |
| "Computer, switch to Firefox" | Activate Firefox window |
| "Computer, check for updates" | Run pacman -Syu, TTS result |

## Integration Points

### SamiType → SamiOS
```
samiType.onTranscription(text, confidence) {
    if (confidence < 0.6) return;  // ignore low-confidence
    
    command = CommandRouter.parse(text);
    if (command) {
        result = command.execute();
        if (result.shouldSpeak) {
            samiTypeTTS.speak(result.message);
        }
    }
}
```

### systemd Service
```ini
[Unit]
Description=SamiOS Voice Assistant (SamiType)
After=pipewire.service

[Service]
ExecStart=/usr/local/bin/samios-voice
Restart=always
User=sami

[Install]
WantedBy=default.target
```

## Phased Rollout

### Phase 1: Foundation (MVP)
- [ ] SamiType running as systemd service in SamiOS
- [ ] Push-to-talk activation (keyboard shortcut)
- [ ] Basic command set: open apps, web nav, volume
- [ ] Text output in notification (no TTS yet)

### Phase 2: Always-On Wake Word
- [ ] Wake word "Computer" detection
- [ ] Expanded command set: window management, screenshots
- [ ] TTS voice feedback

### Phase 3: Natural Language
- [ ] LLM-powered intent parsing for complex commands
- [ ] "Computer, find all PDFs from last week and move them to Documents"
- [ ] Context awareness (current app, current selection)

### Phase 4: Full Voice OS
- [ ] Voice-controlled settings (display, network, bluetooth)
- [ ] Dictation mode (SamiType → any text field)
- [ ] Custom command creation ("Computer, when I say X, do Y")
- [ ] Multi-turn voice interactions

## Dependencies
- SamiType (STT + TTS)
- pipewire (audio capture/playback)
- KDE Plasma (window/desktop control via qdbus)
- wmctrl / xdotool (X11 window manipulation)
- Optional: openWakeWord or Porcupine (wake word detection)

## Security Considerations
- All processing is local (no cloud, no data leaves the machine)
- Microphone indicator (notification when listening)
- Manual disable switch (system tray toggle)
- No voice data stored unless explicitly saved by user