Azure Percept – First Look

The Azure Percept is a Microsoft Developer Kit designed to fast track development of AI (Artificial Intelligence) applications at the Edge.

Azure Percept Developer Kit

Highly coupled to Azure, the Percept Developer Kit is custom built to enable developers to harness the power of edge computing to tackle complex edge computing scenarios around the mainstays of AI – Vision and Audio.

Percept Specifications

At a high level, the Percept Developer kit has the following specifications;

Carrier (Processor) Board:

Azure Percept - Carrier Board
Azure Percept – Carrier Board
  • NXP iMX8m processor
  • Trusted Platform Module (TPM) version 2.0
  • Wi-Fi and Bluetooth connectivity

Vision SoM:

Azure Percept - Vision SoM
Azure Percept – Vision SoM
  • Intel Movidius Myriad X (MA2085) vision processing unit (VPU)
  • RGB camera sensor

Audio SoM:

Azure Percept – Audio SoM
  • Four-microphone linear array and audio processing via XMOS Codec
  • 2x buttons, 3x LEDs, Micro USB, and 3.5 mm audio jack

Percept Target Verticals

The Percept is a Developer Kit with a variety of target industries in mind. As such, Microsoft have created some pre-trained AI models aimed at those markets.

Azure Percept - Sample Vision Models
Azure Percept – Sample Vision Models

These models can be deployed to the percept to quickly configure the Percept to recognise objects in a set of environments, such as;

  • General Object Detection
  • Items on a Shelf
  • Vehicle Analytics
  • Keyword and Command recognition
  • Anomaly Detection etc

Azure Percept Studio

Microsoft provide a suite of software to interact with the Percept, centred around Azure Percept Studio, an Azure based dashboard for the Percept;

Azure Percept Studio is broken down into several main sections;

Overview:

Azure Percept Studio - Overview
Azure Percept Studio – Overview

This section gives us an overview of Percept Studio, including;

  • A Getting Started Guide
  • Demos & Tutorials,
  • Sample Applications
  • Access to some Advanced Tools including Cloud and Local Development Environments as well as setup and samples for AI Security.

Devices:

Azure Percept Studio - Devices
Azure Percept Studio – Devices

The Devices Page gives us access to the Percept Devices we’ve registered to the solution’s IoT Hub.

We’re able to click into each registered device for information around it’s operations;

Azure Percept Studio - Device Details
Azure Percept Studio – Device Details

This area is broken down into;

  • A General page with information about the Device Specs and Software Version
  • Pages with Software Information for the Vision and Speech Modules deployed to the device as well as links to Capture images, View the Vision Video Stream, Deploy Models and so on
  • We’re able to open the Device in the Azure IoT Hub Directly
  • View the Live Telemetry from the Percept
  • Links with help if we need to Troubleshoot the Percept

Vision:

Azure Percept Studio - Vision
Azure Percept Studio – Vision

The Vision Page allows us to create new Azure Custom Vision Projects as well as access any existing projects we’ve already created.

Speech:

Azure Percept Studio - Speech
Azure Percept Studio – Speech

The Speech page gives us the facility to train Custom Keywords which allow the device to be voice activated;

Azure Percept Studio - Create Custom Keyword
Azure Percept Studio – Create Custom Keyword

We can also create Custom Commands which will initiate an action we configure;

Azure Percept Studio - Create Custom Command
Azure Percept Studio – Create Custom Command

Percept Speech relies on various Azure Services including LUIS (Language Understanding Intelligent Service) and Azure Speech.

Other Resources

Next Steps

In the next few posts I’ll be looking at;

  • Getting Started with the Azure Percept Developer Kit
  • Azure Percept Vision First Steps
  • Azure Percept Audio First Steps
  • Azure Percept Audio – Custom Keywords and Commands
  • Training and Deploying Vision Models
  • Training and Deploying Audio Models