Just A Rather Very Intelligent System
Voice-Cloned • Fully Autonomous • Zero-Latency • Real-World OS Control
-4E73DF?style=flat-square&logo=apple){kind=icon}
FS FERDOWS
Lead AI Architect & Software Engineer • LateBright
GitHub (@fsferdows) •
YouTube Channel •
Contact
"Why build a toy chatbot when you can build a genuine, voice-cloned, autonomous Iron Man JARVIS that controls your operating system, decompiles software, browses the web, and handles tasks before you finish speaking?" — FS FERDOWS
J.A.R.V.I.S. is a complete, production-grade recreation of Tony Stark's iconic artificial intelligence system. Unlike typical voice wrappers or chatbot UIs, J.A.R.V.I.S. is a fully integrated autonomous OS companion featuring real-time audio cloning, zero-screenshot browser vision, reverse-engineering capabilities, and full system actuation on macOS.
Say "Hey Jarvis", and he wakes, listens, reasons, and executes actions across your operating system in real-time.
[ Human Speech / Microphone ]
│
▼
[ WebGL 3D Arc-Reactor HUD ] ── (http://localhost:5173)
│
▼
[ Full-Duplex Bridge WebSocket (ws://localhost:8787) ]
│
┌──────────────┴──────────────┐
▼ ▼
[ OmniRoute (Port 20128) ] [ OmniVoice (Port 8008) ]
• Smart Multi-Model Router • Zero-Shot Neural Voice Clone
• 350+ Local & Cloud Models • Paul Bettany JARVIS Persona
• Auto-fallback & Caching • Sub-500ms Cloned Speech (MPS)
│ │
▼ ▼
[ Autonomous Execution Engines ] [ macOS System Audio Output ]
├── 🔬 REA (153 Reverse-Engineering CLI Commands & 207 MCP Tools)
├── 🦞 OpenClaw (Autonomous Multi-Step Agent & Channel Messaging)
├── 👁️ Agent Reach (Zero-Screenshot Live Browser DOM Vision)
├── 🦾 macOS Actuation (Apps, WhatsApp, Spotify, Notes, Calendar)
└── 🧰 109 Developer Skills & 66 Growth CLIs
- Acoustic Fidelity: Cloned directly from Paul Bettany's original J.A.R.V.I.S. voice with crisp British butler cadence (
male, british accent, low pitch). - Hardware-Accelerated: Runs locally on Apple Silicon Metal Performance Shaders (
mps) usingk2-fsa/OmniVoice. - Instant Response Cache: 51 pre-warmed conversational acknowledgments deliver audio playback in ~405 ms.
- 100% Private: Your voice never leaves your machine.
- Smart Model Routing: Seamlessly arbitrates queries between fast local inference (
ollama-local/qwen2.5) and over 350 cloud models. - Failover & Resilience: Automatic provider circuit breakers, token compression, and intelligence scoring.
- Zero Configuration: Authenticated locally via dedicated autonomous key pairs.
- 153 CLI Commands & 207 Analysis Tools: Built-in developer toolkit to decompile, analyze, and inspect binaries.
- Target Coverage: Native Mach-O/ELF binaries, Electron unpacked ASARs, Android APKs, Web single-page app bundles, and firmware.
- Direct Voice Directives: Simply say "Jarvis, reverse engineer this binary and inspect its functions" or "Decompile this Electron application".
- Autonomous Agent Turns: Delegates multi-step objectives to OpenClaw (
/opt/homebrew/bin/openclaw). - Channel Dispatch: Natively dispatches messages and files through WhatsApp Web, Telegram, and automated channels.
- ClawHub Skills: Discovers, verifies, and installs skills on demand.
- High-Speed DOM Inspection: Reads open Chrome tabs, extracts live text, evaluates JavaScript, and tracks Meta/Google Ads metrics without burning tokens on expensive vision screenshots.
- Live Tab Navigation: Switch, query, and interact with browser workflows hands-free.
- Application Control: Open or cleanly close applications (Google Chrome, WhatsApp, Spotify, Terminal, Calendar, Notes, System Settings).
- Media & Hardware: Adjust system volume, toggle mute, control Spotify/Apple Music tracks, query battery status, or lock screen.
- Calendar & Notes: Book events with date/time parsing and append formatted entries to Apple Notes.
- Terminal Execution: Safely run shell commands with butler confirmation reports.
- Three.js & Custom GLSL: Shaders render an authentic glowing holographic HUD that pulses and ripples in direct sync with audio frequency bands.
- Interactive Controls: Touch, orbit, and hand-tracking support via MediaPipe.
| Feature | Apple Siri | Amazon Alexa | Standard LLMs | J.A.R.V.I.S. (This Repo) |
|---|---|---|---|---|
| Voice Persona | Generic Robot | Generic Voice | Text Only | Cloned Paul Bettany (JARVIS) |
| Response Latency | 2s – 4s | 2s – 3s | 3s – 10s | 405 ms (Prewarmed) / < 3s (Full Turn) |
| Local Privacy | ❌ Cloud | ❌ Cloud | ❌ Cloud | ✅ 100% Local & Private |
| Reverse Engineering | ❌ None | ❌ None | ✅ 153 REA Decompiler Tools | |
| Multi-Agent Engine | ❌ None | ❌ None | ✅ OpenClaw Autonomous Loops | |
| Browser Vision | ❌ None | ❌ None | ✅ Agent Reach Zero-Screenshot DOM | |
| OS Actuation | Limited Shortcuts | Smart Home Only | None | ✅ Full macOS Control & Apps |
| Model Routing | Fixed | Fixed | Single Vendor | ✅ OmniRoute 350+ Providers |
- macOS (Apple Silicon M1/M2/M3/M4 recommended for MPS acceleration)
- Node.js
>= 20.0.0 - Python
>= 3.10 - Google Chrome (for WebGL HUD and Agent Reach)
git clone https://github.com/fsferdows/Jarvis.git
cd Jarvisnpm install# Starts both the face (React Vite HUD) and brain (WebSocket Bridge & Actions)
node scripts/start.mjs --writesOpen http://localhost:5173 in Google Chrome, click INITIALISE, and say:
"Hey Jarvis, status report."
| Domain | What to Say to J.A.R.V.I.S. |
|---|---|
| Status Check | "Jarvis, are all systems operational?" |
| Reverse Engineering | "Jarvis, reverse engineer this application and decompile its entry point." |
| Autonomous Agent | "Jarvis, run an OpenClaw agent turn to summarize my reminders." |
| Browser Inspection | "Jarvis, inspect open browser tabs using Agent Reach." |
| Real-World Action | "Jarvis, launch Google Chrome and open YouTube." |
| Messaging | "Jarvis, send a WhatsApp message to Alex saying I am on my way." |
| Calendar Booking | "Jarvis, schedule a meeting with engineering tomorrow at 2 PM for 45 minutes." |
| Hardware & Audio | "Jarvis, set the volume to 80% and play some music." |
| Developer Skills | "Jarvis, dispatch coding-agent to review my latest git commits." |
- Frontend HUD: React 19, Vite, Three.js, WebAudio API, GLSL Shaders, Tailwind CSS.
- Bridge Engine: Node.js ESM WebSocket Gateway (
ws://localhost:8787), Model Context Protocol (MCP) server. - Voice Engine:
k2-fsa/OmniVoice, PyTorch MPS (Metal Performance Shaders), SoundFile. - Model Router: OmniRoute smart proxy (
http://127.0.0.1:20128), Ollama (qwen2.5:3b,qwen2.5-coder). - Autonomous Agent: OpenClaw CLI, ClawHub registry.
- Reverse Engineering:
rea-agentsv6.7.0 CLI & decompiler plugins. - Browser Automation: Agent Reach CDP (Chrome DevTools Protocol) bridge.
├── bridge/ # WebSocket Bridge & Action Servers
│ ├── server.mjs # Central duplex gateway & /tts audio pipeline
│ ├── actions.mjs # macOS actuation, REA, OpenClaw & skill dispatch
│ └── ui.mjs # Three.js HUD remote control
├── voice/ # OmniVoice Neural TTS
│ ├── omnivoice_service.py # FastAPI service on port 8008
│ ├── test_jarvis.wav # Paul Bettany voice clone reference audio
│ └── start_omnivoice.sh # MPS daemon launcher
├── skills/ # 109 Local Agent Skills
│ ├── reverse-engineer-anything/ # REA core skill & references
│ ├── coding-agent/ # Autonomous feature builders
│ ├── python-debugpy/ # Interactive debugger
│ ├── diagram-maker/ # Mermaid/Architecture generator
│ └── agent-reach/ # Zero-screenshot browser vision
├── tools/ # 66 CLI Tools & Agent Reach scripts
│ ├── browser_reach.py # CDP tab inspection
│ └── clis/ # Marketing & productivity CLIs
├── src/ # React Three.js HUD Web App
│ ├── scene/Scene.tsx # 3D Arc-Reactor holographic canvas
│ ├── lib/tts.ts # Low-latency WebAudio player
│ └── lib/voice.ts # Voice activity detection & wakeword
└── scripts/
└── start.mjs # One-click full stack orchestrator
Contributions make the open-source community an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/IronManFeature) - Commit your Changes (
git commit -m 'Add awesome Jarvis capability') - Push to the Branch (
git push origin feature/IronManFeature) - Open a Pull Request
If you love J.A.R.V.I.S., please leave a Star ⭐ to support the project and help it trend!
Distributed under the MIT License. See LICENSE for more details.
Built with ❤️ by FS FERDOWS
"Sometimes you gotta run before you can walk."