Voice Mode: Real-Time Spoken Conversations With Your Agent

Hermes Agent supports full voice interaction across the CLI and messaging platforms. Talk to the agent using your microphone, hear spoken replies, and have live voice conversations in Discord voice channels. Voice Mode comes in three flavors, each tuned to a different surface:

• Interactive Voice (CLI) — press Ctrl+B to record; the agent auto-detects silence and responds
• Auto Voice Reply (Telegram, Discord) — the agent sends spoken audio alongside text responses
• Voice Channel (Discord) — the bot joins a voice channel, listens to users speaking, and speaks replies back

Whether you want hands-free terminal conversation or a bot that talks to your team in Discord, the same plumbing — speech-to-text (STT) and text-to-speech (TTS) — powers all of it.

Prerequisites

Before enabling voice, make sure you have a basic working setup: Hermes Agent installed, an LLM provider configured via hermes model or credentials in ~/.hermes/.env, and a base setup where hermes responds to text. The ~/.hermes/ directory and default config.yaml are created automatically the first time you run hermes; you only create ~/.hermes/.env manually for API keys.

Then install the extras you need:

cd ~/.hermes/hermes-agent && uv pip install -e ".[voice]"       # CLI microphone + playback
cd ~/.hermes/hermes-agent && uv pip install -e ".[messaging]"   # Discord & Telegram voice
cd ~/.hermes/hermes-agent && uv pip install -e ".[tts-premium]" # ElevenLabs premium TTS

System dependencies include PortAudio for microphone input and playback, ffmpeg for audio conversion, and Opus for Discord voice channels.

CLI Voice Mode

Start the CLI with hermes, then enable voice with /voice on. Press Ctrl+B — a beep plays and recording starts. Speak, and after three seconds of silence the recording stops; the audio is transcribed via Whisper and sent to the agent. If TTS is on, the reply is spoken aloud. Recording restarts automatically, so you can keep talking without pressing any key.

A two-stage algorithm detects when you've finished: it waits for audio above the RMS threshold for at least 0.3 seconds, then triggers once 3 seconds of silence follow. If no speech is detected for 15 seconds, recording stops on its own. Say "stop" — and nothing else — to end the voice chat hands-free.

TTS streams sentence-by-sentence as the agent generates text, so you don't wait for the full response. The pipeline strips markdown, emoji, and reasoning blocks before speaking. You can barge in at any point: speak while the agent is thinking to interrupt the in-flight turn, or talk over the reply to cut playback instantly.

Voice on Messaging Platforms

Via the gateway (hermes gateway setup, then hermes gateway), voice replies work in Telegram and Discord. On Telegram, replies arrive as a native Voice bubble (Opus/OGG) that plays inline. Control spoken output with the same /voice commands used in the CLI:

• /voice on — spoken replies only when you send a voice message
• /voice tts — spoken replies for all messages
• /voice off — disable voice replies

The most immersive experience is the Discord voice channel: the bot joins a voice channel, listens to each user's audio stream independently, detects silence, transcribes speech, processes it through the full agent pipeline, and speaks the reply back in the channel. It pauses its own listener while playing replies (echo prevention), and only users in the allowed list can interact via voice.

Speech and Voice Quality

STT is handled by Whisper — a local provider needs no API key at all if faster-whisper is installed, with Groq and OpenAI as cloud fallbacks. Provider priority is local > groq > openai. A hallucination filter of 26 known phrases across multiple languages cleans up phantom text Whisper sometimes generates from silence. TTS providers span the free Edge TTS, local NeuTTS, and premium ElevenLabs. On the desktop app, a full-duplex GPT-Live mode lets the voice model handle interruptions itself and delegate real work to Hermes.

Key Takeaways

• Voice Mode covers three surfaces: interactive CLI push-to-talk, Telegram and Discord auto voice replies, and full Discord voice channels.
• Install with uv pip install -e ".[voice]" (CLI), ".[messaging]" (Telegram/Discord), and ".[tts-premium]" for ElevenLabs.
• CLI voice streams TTS sentence by sentence, auto-detects silence (3s), supports barge-in at any point, and ends hands-free by saying "stop".
• STT needs no API key when faster-whisper is installed local, and a 26-phrase hallucination filter cleans up phantom transcripts.