The problem
Most assistant projects are a wrapper around somebody else's API, which means they stop working the moment the connection drops or the key expires. This one runs locally and is scoped to automation that is genuinely useful on a desktop.
What I built
Speech recognition and synthesis run locally. The command layer is a registry of intent handlers rather than a chain of if statements, so adding a capability is a matter of registering one function.
Everything that touches the filesystem or launches a process is behind an explicit allow list, because an assistant with shell access and loose parsing is a liability, not a feature.
Architecture
Python with local speech recognition and text to speech. Intent matching is keyword and pattern based, deliberately, so behaviour is predictable and debuggable rather than probabilistic.
Problems worth writing down
Recognition accuracy
Local models mishear short commands. Adding a confirmation step for anything destructive was cheaper than chasing a better model.
Blocking audio
Listening on the main thread froze the interface. Moving capture to a worker thread with a queue solved it.