Skip to content

isair/jarvis

Repository files navigation

Jarvis

A private AI voice assistant that lives on your computer - not in someone else's cloud. Leave it running 24/7 and treat it like a third person in the room: say "Jarvis" anywhere in your sentence ("what do you think, Jarvis?" or "Jarvis, look at this error") and get thoughtful, contextual responses. It remembers your preferences, helps with code, tracks your health goals, searches the web, and connects to 500+ tools via MCP.

🔒 100% local processing. No subscriptions. No data harvesting. Automatic redaction of sensitive info.


Support Jarvis GitHub Sponsors Ko-fi


Jarvis Face

Memory Viewer - Conversations Memory Viewer - Nutrition

Why Jarvis?

🔒 Your data stays yours - 100% local AI processing. No cloud, no subscriptions, no data harvesting. Automatic redaction of sensitive info. This is non-negotiable.

🗣️ A third person in the room - Unlike voice assistants that only respond to rigid commands, Jarvis understands conversations. It maintains a short temporary rolling context of what's being discussed, so when you ask "Jarvis, what do you think?" it knows exactly what you're talking about. Have it chime into discussions with friends, help debug code while you talk through problems, or weigh in on decisions.

🧠 Never forgets - Unlimited memory across conversations. Context-aware personalities (developer/business/life coach). Learns your preferences over time.

🔌 Extensible - MCP integration connects Jarvis to 500+ tools: smart home, GitHub, Slack, databases, and more.

📊 Transparent progress - We track what works (and what doesn't) with automated evals. See current accuracy →

🚧 Known limitations: Jarvis is under active development. Primary development happens on macOS. Windows/Linux support may lag behind. We're building in the open, issues and contributions welcome!

  • Voice-only for now—no text chat interface yet (#35)
  • No mobile apps (#17)
  • "Stop" commands during speech sometimes get filtered as echo (#24)
See it in action (example conversations)

Chiming into conversations (the magic moment):

👤 Alice: I wonder what the weather will be like tomorrow
👤 Bob: Yeah, we should check before planning the picnic
👤 Alice: Jarvis, what do you think?

✨ Working on it: what do you think about the weather tomorrow for the picnic
🧰 Tool: getWeather…

🤖 Jarvis (life)
Tomorrow looks great for a picnic! Sunny with highs around 72°F...

Jarvis understood the entire conversation and gave a contextual answer - no need to repeat the question.

Debugging with screen analysis:

✨ Working on it: i'm getting this weird graphql error
🧰 Tool: screenshot…
🧰 MCP: vscode:findReferences…

🤖 Jarvis (developer)
I can see the error on your screen - "Cannot return null for non-nullable field".
Your User type has `email: String!` but the resolver is returning null...

Health-aware advice:

✨ Working on it: should i order pizza tonight considering my health goals
🧰 Tool: fetchMeals…

🤖 Jarvis (life)
Looking at your week - you mentioned wanting to lose 10 pounds by March...
Today you've had about 860 calories so far. Given your 1,800 calorie target,
you've got room for pizza! But maybe consider thin crust to save calories.

Personalized news search:

✨ Working on it: what are some news from today that might interest me?
  🧠 Looking back at our past conversations…
  🌐 Searching the web…

🤖 Jarvis (business)
Here's a quick snapshot of today's headlines...

Natural wake word placement (say "Jarvis" anywhere):

👂 Heard: give me a random topic to discuss jarvis
✨ Working on it: give me a random topic to discuss

🤖 Jarvis (life)
How about the Fermi Paradox? Given the vast number of stars...

Quick Install

1. Install Prerequisites

Platform Requirement
All Ollama
Windows Visual C++ Redistributable (most PCs already have this)

2. Download Jarvis

Get the latest from GitHub Releases:

Platform Download Run
Windows Jarvis-Windows-x64.zip Extract → Run Jarvis.exe
macOS Jarvis-macOS-arm64.zip Extract → Move to Applications → Right-click → Open
Linux Jarvis-Linux-x64.tar.gz tar -xzf → Run ./Jarvis/Jarvis

Click the system tray icon → Start Listening → Say "Jarvis" and talk!

Setup - Initial Check Setup - Model Selection Setup - Whisper Setup - Complete

Real-time Logs

Features

  • Conversational Awareness - Understands ongoing discussions. Ask "Jarvis, what do you think?" and it knows what you're talking about. Works naturally in multi-person conversations.
  • Unlimited Memory - Never forgets. Searches across all your conversation history. Memory Viewer GUI included.
  • Smart Personalities - Adapts tone: Developer (debugging), Business (planning), Life Coach (health/wellness)
  • Built-in Tools - Screenshot OCR, web search (with auto-fetch), weather, file access, nutrition tracking, location awareness
  • Natural Voice - Say "Jarvis" anywhere in your sentence, interrupt with "stop", follow up without repeating the wake word
  • MCP Integration - Connect to 500+ external tools (Home Assistant, GitHub, Slack, etc.)

System Requirements

Hardware RAM Model
Most users 8GB+ llama3.2:3b (default)
High-end 16GB+ gpt-oss:20b

The setup wizard will guide you through model selection and installation on first launch.

Configuration

Most users won't need to change anything. Config file: ~/.config/jarvis/config.json

Speech Recognition (Whisper)

Language Modes

  • English Only (default, better accuracy): "whisper_model": "small.en"
  • Multilingual (99 languages): "whisper_model": "small"

Model Sizes

Model English Multilingual Size VRAM Speed
Tiny tiny.en tiny ~75MB ~1GB ~10x
Base base.en base ~150MB ~1GB ~7x
Small small.en small ~500MB ~2GB ~4x
Medium medium.en medium ~1.5GB ~5GB ~2x
Large V3 Turbo - large-v3-turbo ~1.6GB ~6GB ~8x

Speed is relative to original large model. Source

Voice Interface (Advanced)

LLM Intent Judge - Jarvis uses llama3.2:3b for intelligent voice intent classification (echo detection, query extraction, stop commands). This model is automatically installed alongside your chosen chat model during setup. The intent judge cannot be disabled but gracefully falls back to simpler text matching if Ollama is unavailable.

Audio-Level Wake Word Detection (optional) - More precise wake word timing using openWakeWord:

{
  "audio_wake_enabled": true,
  "audio_wake_threshold": 0.5
}

Requires openWakeWord library. Falls back to text detection if unavailable.

Text-to-Speech

Piper TTS (default) - Neural TTS that auto-downloads on first use (~60MB):

  • Works out of the box - no setup required
  • High-quality British English male voice (en_GB-alan-medium)
  • Fast local synthesis with exact duration tracking

To use different Piper voices, download from HuggingFace and set:

{
  "tts_piper_model_path": "~/.local/share/jarvis/models/piper/en_GB-alan-medium.onnx"
}

Chatterbox - AI voice with emotion control (requires running from source):

{ "tts_engine": "chatterbox" }

Voice cloning with Chatterbox - add a 3-10 second .wav sample:

{
  "tts_engine": "chatterbox",
  "tts_chatterbox_audio_prompt": "/path/to/voice.wav"
}
MCP Tool Integration

Connect Jarvis to external tools via MCP servers:

{
  "mcps": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": { "GITHUB_TOKEN": "your-token" }
    }
  }
}

Popular integrations:

  • Home Assistant - Voice control for smart home
  • Google Workspace - Gmail, Calendar, Drive, Docs
  • GitHub - Issues, PRs, workflows
  • Notion - Knowledge management
  • Slack/Discord - Team communication
  • Databases - MySQL, PostgreSQL, MongoDB
  • Composio - 500+ apps in one integration

See full MCP setup guide below.

MCP Integrations

Home Assistant - Smart home voice control
  1. Add MCP Server integration in Home Assistant (Settings → Devices & services)
  2. Expose entities you want to control (Settings → Voice assistants → Exposed entities)
  3. Create Long-lived Access Token (Profile → Security → Create token)
  4. Install proxy: uv tool install git+https://github.com/sparfenyuk/mcp-proxy
  5. Add to config:
{
  "mcps": {
    "home_assistant": {
      "command": "mcp-proxy",
      "args": ["http://localhost:8123/mcp_server/sse"],
      "env": { "API_ACCESS_TOKEN": "YOUR_TOKEN" }
    }
  }
}

"Jarvis, turn on the living room lights" / "set bedroom to 72°" / "run good night scene"

Google Workspace - Gmail, Calendar, Drive, Docs, Sheets
{
  "mcps": {
    "google_workspace": {
      "command": "npx",
      "args": ["-y", "google-workspace-mcp"],
      "env": {
        "GOOGLE_CLIENT_ID": "your-client-id",
        "GOOGLE_CLIENT_SECRET": "your-client-secret"
      }
    }
  }
}

Setup: taylorwilsdon/google_workspace_mcp

GitHub - Repos, issues, PRs, workflows
{
  "mcps": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": { "GITHUB_TOKEN": "your-token" }
    }
  }
}
Notion, Slack, Discord, Databases

Notion:

{ "mcps": { "notion": { "command": "npx", "args": ["-y", "@makenotion/mcp-server-notion"], "env": { "NOTION_API_KEY": "your-token" } } } }

Slack:

{ "mcps": { "slack": { "command": "npx", "args": ["-y", "slack-mcp-server"], "env": { "SLACK_BOT_TOKEN": "xoxb-...", "SLACK_USER_TOKEN": "xoxp-..." } } } }

Discord:

{ "mcps": { "discord": { "command": "npx", "args": ["-y", "discord-mcp-server"], "env": { "DISCORD_BOT_TOKEN": "your-token" } } } }

Databases: bytebase/dbhub (SQL), mongodb-mcp-server (MongoDB)

Composio - 500+ apps in one integration
{
  "mcps": {
    "composio": {
      "command": "npx",
      "args": ["-y", "@composiohq/rube"],
      "env": { "COMPOSIO_API_KEY": "your-key" }
    }
  }
}

Get API key at composio.dev

Troubleshooting

Common issues

Jarvis doesn't hear me - Check microphone permissions, speak clearly after "Jarvis"

Responses are slow - Ensure you have enough RAM (8GB+ for default model)

Windows: App won't start - Extract full zip first, check Windows Defender

macOS: "App can't be opened" - Right-click → Open, or System Settings → Privacy & Security → Allow

Linux: No tray icon - sudo apt install libayatana-appindicator3-1

For Developers

Running from source
git clone https://github.com/isair/jarvis.git
cd jarvis

# macOS
bash scripts/run_macos.sh

# Windows (with Micromamba)
pwsh -ExecutionPolicy Bypass -File scripts\run_windows.ps1

# Linux
bash scripts/run_linux.sh

Running from source enables Chatterbox TTS (AI voice with emotion/cloning). Piper TTS works in both bundled and source modes.

Privacy hardening (stay 100% offline)
{
  "web_search_enabled": false,
  "mcps": {},
  "location_auto_detect": false,
  "location_enabled": false
}

Verify: sudo lsof -i -n -P | grep jarvis (should only show 127.0.0.1 to Ollama)

Privacy & Storage

  • 100% offline - No cloud services required
  • Auto-redaction - Emails, tokens, passwords automatically removed
  • Local storage - Everything in ~/.local/share/jarvis

License

  • Personal use: Free forever
  • Commercial use: Contact us

Support

Report issues · Discussions · Sponsor

About

Your AI assistant that never forgets and runs 100% privately on your computer. Leave it on 24/7 - it learns your preferences, helps with code, manages your health goals, searches the web, and connects to any tool via MCP servers (e.g. home automation). No subscriptions, no cloud, just say "Jarvis" anywhere in your sentence and talk naturally.

Topics

Resources

License

Stars

Watchers

Forks

Sponsor this project

Languages