Azure IoT and AI

Covering everything from anomaly detection to computer vision and audio, Azure Cognitive Services are an excellent match for Azure’s IoT tooling, allowing you to quickly build AI-powered IoT applications from your existing sensors and systems.

Azure’s Cognitive Services are an important part of its offering, wrapping Microsoft Research developed machine learning algorithms as easy-to-use REST APIs with SDKs for most common languages and platforms.

While there are a lot of Cognitive Services APIs, not all of them are suitable for IoT applications. However some have surprising utility; for example the speech services can be used to build a custom voice control service that can help with accessibility or make it easier for staff to control machinery. There’s also the option of using Azure ML Studio to build, train, and test custom models using frameworks like TensorFlow and then offering inferencing endpoints or exporting ONNX-format models for use with code written using technologies like ML.NET.

Some Cognitive Services, like the Anomaly Detector work with time series data, can be used to provide a framework for predictive maintenance. You can start with a set of tie series data to examine a batch of sensor outputs for anomalies, using the data you send to generate a model of what’s normal and identifying data points that indicate possible issues. Alternatively, you can stream data to your endpoint, allowing near real-time anomaly detection.

The most obvious Cognitive Service is its computer vision tools. These can be trained to identify objects, for example using them to monitor production lines or to identify stock on shelves. The default models work well for general purpose computer vision, but if you’re working on a specific problem you can build custom models that can be trained on real-world data, allowing you to take account of camera optics as well as your work environment.

Azure services like IoT Hub act as a gateway between your IoT devices and Azure, routing messages to cloud hosted applications and on to Cognitive Services. When you’re working with many thousands of devices, you can use message routing to deliver data quickly and effectively. If you’re using Azure IoT Hub with Cognitive Services it’s worth considering using Azure’s messaging services to manage delivering messages to your ML endpoints, working with Azure Functions to handle asynchronous calls to the Cognitive Service APIs or to your custom ML models.

Containers

One of the most useful aspects of Cognitive Services is the ability to bring its trained models into your own network and onto your own devices, using Cognitive Services containers. Available as part of Microsoft’s edge Kubernetes tooling, they’re quick to deploy and use, and easy to integrate into IoT services. Key IoT-focused services are the anomaly detector, computer vision, and spatial analysis. These Linux-based containers run on any Docker-enabled system, billing through Azure and deploying from Microsoft’s own container registry.

One option for hosting your containers is Microsoft’s own Azure Stack Edge hardware. Three different options provide rack and standalone hardware that can run in your network and connect to your IoT hardware. For machine learning applications the Azure Stack Edge Pro GPU offers a machine learning accelerator, using hardware-accelerated ML containers alongside the Azure IoT Edge modular platform. As well as using GPUs for inferencing, there’s the option of using an FPGA to accelerate your own custom machine learning models via an FPGA with ONNX runtime support.

Running ML models at the edge of your network makes a lot of sense, especially if you’re using tools like the anomaly detector to identify possible hardware failures on machines or spatial analysis for safety. You can use the Azure IoT Central management platform to build and deploy modules and workloads to edge devices, including machine learning containers. This allows you to have a consistent set of intelligent applications running across your IoT device fleet, delivering on Microsoft’s vision of “an intelligent cloud and an intelligent edge”.

Azure Percept

If you don’t need to run an entire edge server for your machine learning applications, Microsoft has bundled a mix of its IoT tooling into the standalone industrial IoT Azure Percept service. Focused on computer vision and audio recognition, Azure Percept provides users with smart cameras and microphones that provide on-device AI services as well as integrating with cloud services and local Azure IoT Hub instances.

Applications are developed in the cloud-hosted Azure Percept Studio, using low code tools that share concepts with Azure Logic Apps. Use it to build workflows that link Azure Cognitive Services with Azure Percept Devices. A developer kit is available that includes an edge processing unit and a camera with an Intel image processor, both with standard industry mounting for use on factory floors.

You can use Percept to quickly roll out custom vision solutions, using IoT Hub to deliver code to your device and pass data on to cloud services. Training models takes advantage of cloud compute capabilities, using the image stream from a Percept camera. Once a model is trained it’s downloaded to the Percept device on the edge, where it runs as a local inferencing service in a Percept container under Microsoft’s own CBL-Mariner Linux. Results are sent back to Azure-hosted services as a stream of images and messages, in JSON format, ready for use in your code.

Industry-targeted IoT products

Microsoft is also working to turn its Azure AI services and IoT tools into specific industry-targeted products. The first of these, Azure Farmbeats, is a precision agriculture service that builds on Microsoft Research work on building tools to help farmers identify how and where to water, feed, and treat plants. Remote sensors and Azure Stack Edge hardware work together to provide a farm IoT network that can work even when contending with high latency, low bandwidth rural networking.

AI and IoT Insider Labs

If you’re thinking about using Azure Cognitive Services with Azure IoT, you might want to consider applying for the AI and IoT Insider Labs service. This gives you access to Azure experts who can help you build a first prototype in a 1-week sprint. Labs are currently virtual, with daily stand-ups and access to engineering resources while you work on your code.

The ability to mix different aspects of the Azure platform with IoT devices is a key reason for choosing it. With containers and technologies like ONNX making its AI services portable, you’re able to put code where it’s needed: on devices, on the edge, or in the cloud. It’s a flexible way of delivering machine learning to your devices, able to work the way you want, when you want.