← All use cases

Use case P49 · Human

Build a gadget that lets you talk to Claude through your home server: an old desk phone that calls you back, and a push-button ESP32 voice box

James Coates and Willy (WillyV3, Breaking Shit & Fixing It) ·

The short version

What they did

Two home-lab hobbyists built a physical way to talk to Claude: speech goes to their own server, gets transcribed, goes to Claude, and the answer is spoken back. In both cases Claude runs in the cloud, not on the gadget.

An old desk phone that calls you back

James Coates connected an old desk phone on his home network to speech recognition, Claude Code and spoken replies. He built a separate read-only connection to his media server so he could ask about disk space without giving the AI that connection’s credential. Calls, replies and callbacks worked during his experiments, but the build had registration and speech-library failures, and the latest speed-test fixes still needed a real call test. He calls it mainly a learning project rather than a useful replacement for the phone app; the cloud AI does not run on the desk phone. His write-up, 2026-10-15.

A push-button ESP32 voice box

Willy built Raspy, a small button-operated device for talking to Claude through his home server. He says back-and-forth Claude Code sessions produced most of the firmware and server work, including the button controls and network switching. The ESP32 sends audio and plays the answer; it does not run Claude itself. His case worked, but a custom wake word remained unfinished and the source code was not public. His write-up, 2026-01-22.

Result (as claimed)

The desk phone: “I pick up a phone that was designed to call the room down the corridor, ask how full my disks are, hear a couple of soft blips, and a calm British voice tells me.”

“As a tool, mostly not.”

The voice box: “Push a button, talk, release. A few seconds later Claude responds through the speaker.”

“The green sparkle case is v1 - functional but chunky. Will reprint eventually.”

Prompt

Written for this library, one for each example.

The desk phone, adapted for an already configured, read-only server connection:

I’m talking to Claude through a desk phone linked to my home server. The phone, speech tools and callback route are already configured. I want to check free disk space on [my media server] using [the connected read-only tool].

1. Read the current disk-space report and identify the drive or drives it covers.
2. Give a spoken answer under 35 words with available space and clear units.
3. If the answer takes longer than [number of seconds], use the configured callback to this phone.

Use a calm, direct tone. If the tool fails, returns an old reading or leaves a drive unclear, say so rather than inventing a result. Ask me to repeat anything you could not transcribe. Do not change files or settings, and do not claim a callback happened unless it succeeded.

The voice box, adapted from his build, not a published prompt:

I want Claude Code to help build a button-operated voice box using [ESP32-S3 board, microphone and speaker] and [my home server]. The device should send speech to the server, get a Claude answer and play it back.

1. Explain which parts run on the device, on my server and at the outside AI service.
2. Build and test recording while I hold the button, then transcription, then a short reply spoken through the speaker. Test one stage at a time.
3. Keep spoken answers under 35 words and show a short log of each stage’s timing and failures.

Give setup instructions in plain words and ask about missing hardware details. Do not claim Claude runs on the box itself. Leave out a wake word and computer-control tools for this first version. Report failed audio or connection tests before adding more features.

Do it yourself

These steps are my suggested way in, not any one person’s setup.

Level: Advanced for the gadget. The first version below runs on your computer and is Intermediate.

What you need

  • An always-on home computer or server, and either a Claude API key or Claude Code.
  • Free speech tools you can run at home, such as Whisper for speech to text and Kokoro for text to speech.
  • The gadget: an ESP32-S3 board with a microphone, speaker and button, or an old desk phone and a phone system such as Asterisk.
  • An AI coding agent, such as Claude Code, to write the glue.

Start simple

  1. On your computer, ask Claude Code for a script: hold a key, record, transcribe with Whisper, send the text to Claude, speak the reply with Kokoro.
  2. Build and test one step at a time: record a clip, transcribe it, then add the rest.
  3. Tell Claude to keep spoken answers under 35 words.
  4. Time each step so you know where the delay is.

Take it further

  • Move the microphone and speaker to an ESP32-S3 box with a push button that sends audio to your server.
  • Or connect an old desk phone through Asterisk, with a callback when an answer takes a while.
  • Add one read-only tool, such as disk space on your media server, with its own limited credential.

Check your work

  • Read the transcript of each request. Most wrong answers start as a wrong transcription.
  • Give the assistant read-only tools first, never your network or account passwords.
  • Keep the server off the public internet unless you know how to secure it.

Others did this too

  • Hardware hobbyist uses Claude Code to turn a pocket keyboard device into a push-to-talk voice terminal. Leon Furze, 2026-06-25, blog post. “It works on my home Wi-Fi, and it works from a phone hotspot on the other side of the country.”

Source checked. The link, author, date and quote were confirmed against the source. The result itself has not been reproduced. Prompt adapted for this library, inspired by the case. Business size: Individual. Results are quoted as the source claimed them and never improved. Spot an error or a broken link? Email charlie@cbarmorecpa.com.

More in Gadgets, smart home and home lab

New cases go out every Saturday in Inside AI.