AI on the device: what edge AI can do on a microcontroller in 2026

Most talk about AI is about very large models in very large data centres. For connected products, the more useful change has happened at the other end of the scale. Chipmakers are putting neural processing units into microcontrollers. ST's STM32N6, the first in its range with one, is claimed to run machine learning 600 times faster than ST's previous high-end STM32 (New Electronics). The tools are consolidating too: in March 2025 Qualcomm announced it was buying Edge Impulse, a platform for building models for small devices that serves more than 170,000 developers (Edge IR).

The result is that a small, battery-powered device can now run a trained model on its own sensor data without sending anything anywhere. That changes what a connected product can do, though not always in the way the pitches suggest.

Why run AI on the device at all

Airtime. Sending raw vibration, audio or video from thousands of devices costs money on a cellular network. A device that sends "bearing wear detected" sends a few bytes instead of the thousands in a second of raw vibration.

Power. On a battery device the radio is usually the biggest drain. Running a small model locally and sending only the result can use far less energy than sending the data.

Speed. A decision made on the device does not wait for a round trip to the cloud, which matters when a machine needs to stop or a gate needs to open.

Privacy. An image that never leaves the device cannot leak from a server. Under POPIA in South Africa and GDPR in the UK and Europe, counting people without storing their faces is a much simpler conversation than streaming video.

Resilience. A device that can decide for itself keeps working when the signal drops, which in much of South Africa is something you design for from day one.

What works well today

Anomaly detection on machines. A model learns what normal vibration, current or temperature looks like for one motor, pump or compressor, and flags when it drifts. It is the most proven use and often the best first project, because a fault caught early has an obvious rand value.

Sound detection. Recognising a glass break, a leak's hiss, a chainsaw in a reserve or a spoken command, on a device that listens without recording.

Simple vision. Is there a person, a vehicle or an animal in frame, how many, and is the shelf empty? Small models handle presence and counting well. Reading fine detail, like a number plate at an angle at night, usually still needs more compute.

Sensor fusion. Combining motion, temperature and position to classify what a vehicle, a collared animal or a worker's device is doing, and sending the classification instead of the raw streams.

Where the cloud still wins

Big models. Language models and high-accuracy vision models need far more memory and compute than a microcontroller has. The device can decide whether something is worth a closer look and send that one frame to a bigger model.

Learning across the fleet. A model on one device knows one machine. Comparing thousands of machines, spotting that one batch of bearings fails early and retraining the model is cloud work.

Frequent change. If the model will change every week while you learn, keep it in the cloud until it settles. Updating a model on ten thousand devices is a firmware rollout, with all the care that needs.

Explaining decisions. When someone asks why an alarm fired, the cloud can keep the evidence. A device with a few hundred kilobytes of memory cannot.

The hard part is the data

The model is rarely the hard part. The hard part is enough labelled data from the real environment: vibration from the motors you will actually monitor, audio from the rooms the device will actually sit in, images in the light it will actually see. A model trained on a clean data set and deployed to a dusty plant room will produce confident nonsense.

So the first version of an AI device is often a data-collection device. Ship it with the sensors, the storage and the bandwidth to send raw samples back, label what happens, and train on that. When the model is good enough, update the firmware to run it on the device and stop sending the raw data. That only works if the device can be updated over the air from day one, and if the model is treated as a versioned part of the firmware, with a way to roll back.

Measure the right thing, too. In the field, the false alarm rate matters more than the headline accuracy. A model that is 98% accurate and raises a false alarm every day on every machine will be switched off within a month.

Three questions before you build

What decision will the model make, and what happens when it is wrong? Can we collect real data from the field before we commit to a model? And does the device need to decide by itself, or can it send a short summary and let a bigger model in the cloud decide? If the answer to the last question is the cloud, you may not need edge AI at all, and that is a perfectly good outcome.

If you are weighing up AI on a device, send us a brief, or write to hello@handee.co.za.