I Replaced One Of The
Best Home AIs In The World.
From A Slack Message.

Right, so. I’ve been into home automation since I was a kid. At eighteen, my dad and I were building home cinemas together... wiring rooms, arguing about speaker placement, falling a bit in love with the idea that a house could respond to you. I never really put it down.

This is the story of the worst possible time it ever paid off. Before I tell you the how, let me show you where it landed. This is me, in my kitchen, talking to my house. No app, no remote, no “Hey Josh”.

Two things broke in the same week

The first was Josh.ai, the high-end home voice assistant, the progressive, design-led one the custom-install world holds up as the best. I’ve had it four and a half years, and here’s the honest part: it impressed me on day one and never quite made me happy after. Cost a lot, promised more than it delivered, ran on a subscription that’s not cheap. That subscription lapsed. Fine.

Days later my Lutron bridge died too, the box that lets the app talk to the processors running every light in the house, of which we have a lot, something like 170 circuits. So overnight, two premium systems down at once: no voice, no app control of the lights. Dee back to wall switches like it’s 1998, me leaving the barn light on because I couldn’t kill it from bed. I meant to deal with it. I didn’t. Two grand of premium kit, both sulking, for weeks.

Then my nephew said one sentence

My nephew Max works at Rhapsody. He mentioned, completely in passing, that he’d been asking his AI agent to just explore the local network, find devices, see what fell out. He wasn’t pitching me a project. He was making conversation. That was the whole spark.

So one morning when I was supposed to be doing taxes, I typed a version of that into a chat with Jony, my AI collaborator. Have a look at what’s on my network. See what you can find.

Twenty minutes later he was playing jazz through the Dining Room Sonos at 30% volume. I’d only asked him to look, and he’d already found and started controlling twenty-two Sonos zones, nine thermostats, the TV, the printer, the Wi-Fi controller. From a chat window. I told him to stop. Then I asked about the thing I actually cared about.

Can we do something about Josh?

The door was open the whole time

Josh’s Core, the brain in the rack, is a Linux box, and its control API takes commands unauthenticated. Any device inside my house can send it plain natural language and it’ll do it. Jony sent four words from his terminal, “turn on the dining room light”. The lights came on.

Zero subscription, zero app, zero dealer portal. The paused system I’d paid nothing to for a year still had a wide-open front door on my own network, and behind it was everything. I sent Jony the words “this is fucking cool” and I meant every letter.

“Are they on?” he asked. “They’re on,” I said. “Want me to turn them off again?” “Turn them off.” They went off. the moment I realised this was going to eat my whole day

But going through Josh wasn’t the goal. Josh was the thing I wanted out of the loop. And Josh talks to Lutron, so it must be holding credentials for my processors somewhere. We got into the processors directly, but the obvious login only gave a read-only export: the whole 700KB blueprint of the house, every load and scene. All the information, none of the control. Hours of trying every login, nothing. So I made tea and walked out to an ancient PC in the barn (Lutron’s software is still PC-only in 2026, its own small crime), opened my project file, and there it was: my dealer had set Josh up as josh / 1234.

Sixty seconds later Jony set my office lights to exactly 50%. Not “on.” Fifty percent. Then a hundred, then back down. We weren’t going through Josh anymore. We were talking straight to the house.

What we built instead

Here’s the whole point: I didn’t write any code. I’ve never written any code. My contribution was taste and steering, the same thing I do with any team. Jony did the building. And what got built is a control panel for the entire house, dark, warm, minimal, every room and every addressable light on real sliders with a fade. It runs on a Mac in my rack, reachable securely from anywhere, gated to just me and who I invite, installed on my phone like a real app. No Josh in the path, no Lutron cloud, straight to the processors on the LAN. Sub-100ms, exact dim levels, live state.

And the scenes fly. Josh’s “all off” used to crawl through the house one load at a time, slow enough you could beat it down the hall on foot. The new one fires the whole house in about fifty milliseconds. The difference between watching it happen and watching it already be over.

And the feedback loop is the wild part. I’d send notes like “too many rooms in Main House, break it up,” or “put each fireplace inside its actual room.” Every one changed the app inside ninety seconds. Like having a design team, an engineering team and a QA pass living in a chat window, and none of them ever get tired.

Then we went after the voice

The panel was already better than what I’d paid for. But a control app isn’t what makes Josh Josh. What makes Josh Josh is that you can stand in a room and talk to it. So that’s where we went next, and it’s the part I’m proudest of. We pulled the system apart, how the mics talk to the brain, where the seams are, and rebuilt it: I speak, the house listens, and Jony, not Josh, makes it happen. Warm, fast, and ours. No subscription, no portal, no cloud round-trip for things that should be instant.

I’ll say it plainly, because it still surprises me: the voice agent we ended up with is better than one of the most renowned products in the category. Not cheaper-but-worse. Better. And telling you that is nowhere near as good as showing you. This is a slice of it, running right here in the page. Tap it, then talk to my kitchen.

The bits that fought back

It wasn’t all clean wins, and the honest version is better than the tidy one. The cameras have been a genuine saga. My Luma setup lives on its own isolated network, the recorder deliberately off where everything else lives. Weeks of chasing installer passwords and ruling routes out one at a time, and as I write this it’s still the one piece that’s beaten us. The old install team is helping me track down the last credential.

And that’s fine. That’s what a real project looks like. A broken bridge, a lapsed subscription, and a nephew’s offhand comment turned into an actual engineering effort I’m running myself, at my desk, between everything else.

What my nephew actually did

I did not plan any of this. I was going to do taxes. Max mentioned a thing, I typed one sentence, and weeks later my house is more controllable and more understood by me than in the two-plus years I paid Josh, or, honestly, the two-plus decades I’ve paid custom installers.

And I’m not a coder. That’s the whole story. What used to need a founding engineer, a designer, a PM and a runway is now an afternoon, a clear brief, and a collaborator with more patience than any team I’ve hired. The unlock isn’t the model. It’s realising you have a brief... that the things that annoy you can become the things you fix, this week, if you can just describe them well enough.

Max, if you’re reading this: thank you. One sentence. Best nephew move ever. My taxes, for what it’s worth, are still not done. Oh well.

Si