An offline wake-word detector

What you are building

The device listens to a continuous microphone stream and emits one event when it hears one chosen phrase. It must distinguish the phrase from silence, ordinary speech and background sound. The final result is judged as an event detector, not by accuracy on overlapping audio frames.

You can prepare the data and train the model on a laptop. For the complete build, use a board with a microphone, enough memory for the converted model and a supported inference runtime. Keep a push-button way to disable listening and an indicator that shows when audio is being processed.

Run the detector from saved audio first

Build a laptop runner before using the microphone board. It should feed saved audio to the detector in the same sample blocks and at the same rate that the microphone will use. A fixed recording then produces the same feature frames, scores and events on every run, which makes timing and model changes much easier to check.

1. Replay complete quiet, speech and background-noise sessions. Preserve the gaps between words instead of cutting every recording into convenient keyword clips.
2. Repeat with controlled gain changes, clipping, missing blocks and added room noise. Keep the random seed and report event recall, false alarms per hour and response latency.
3. Send a small set of golden recordings through the training code, converted model and target runtime. Compare the features and scores at each boundary before testing live audio.

After replay passes, connect the real board and feed it the same saved samples if the audio interface allows it. Live microphone tests come last because room acoustics and device timing are now the only new variables. See Simulating Physical Systems for replay, fault injection and hardware-in-the-loop tests.

Project milestones

1. Write the event contract.

   Choose the phrase, maximum response latency, acceptable misses and a target false-alarm rate per hour. Decide how long the indicator remains on and how soon another event may be accepted.

2. Record labelled sessions, not isolated clips.

   Record several people saying the keyword and other speech, plus long background sessions in several rooms. Keep consent, retention and deletion rules with the recordings. Split by person and recording session before cutting windows.

3. Turn the stream into repeatable frames and features.

   Fix sample rate, frame length, hop, window function, filterbank or MFCC settings, log floor and normalisation. Save a few feature arrays as golden examples.

   What you need courseSound and Signals courseThe Frequency Domain lessonStreaming features
4. Establish a simple reference.

   Try energy, template similarity or a tiny classifier. Measure it on complete held-out sessions. The reference shows whether the neural model improves the required event metrics rather than only fitting more parameters.

5. Train a small keyword, unknown and silence classifier.

   Use group-safe training and validation partitions. Add plausible noise and gain changes without moving copies of one recording across the split. Inspect misses and false alarms rather than selecting by frame accuracy.

   What you need courseIntroduction to Machine Learning lessonChoose the deployment split
6. Convert, quantise and compare stage outputs.

   Run the same golden audio through the training feature code, converted model and target runtime. Define tolerances for features and logits. Inspect operators, peak SRAM, flash and any CPU fallback.

7. Run continuously without losing samples.

   Use a ring buffer and process one hop at a time. Measure the worst inference time under normal firmware load and prove it completes before the next required hop.

   What you need courseReal-Time Systems lessonModel conversion
8. Turn frame scores into one event.

   Smooth consecutive scores, choose a threshold and add a refractory period. Select these settings on validation sessions, then freeze them before the final test.

9. Run an all-day negative test and a prompted positive test.

   Report event recall, false alarms per operating hour, response latency, memory, tail compute time and energy. Break results down by person, room, device and noise condition with the number of events in each slice.

10. Package a signed, rollback-capable release.

    Version the model, feature code, thresholds, runtime and firmware together. Reject incompatible updates, retain the old working artefact and keep counters rather than raw audio unless recording is explicitly enabled.

Review the detector in four passes

Optional extension: a small on-device foundation model

Do not replace the always-on detector with a generative model. Let the small detector wake a larger local model only after the event. Measure the added model's weights, context cache, latency, heat and energy, validate structured results, and keep user permission around every tool call. The detector remains the low-power gate and the manual button remains the fallback.

Relevant lessons: context and cache memory and typed on-device model boundaries.

Completion record

Keep the event specification, dataset split IDs, feature and model versions, golden vectors, memory map, latency and current traces, field confusion counts and update/rollback test. Those artefacts make the result reproducible.