Speed up Myriad 2 and Myriad X with Xailient AI

Computer Vision software needs hardware, and combined innovations from Xailient and silicon manufacturers are accelerating the move to AI at the Edge.

Intel Movidius™ is one leader in AI hardware innovation, transforming the future of computer vision and artificial intelligence (AI). They have enabled new levels of intelligence for AI assistants, drones, robots, cameras, virtual and augmented reality, and many other devices at the edge. One example of an application is an AI-powered backpack for the visually impaired to help them navigate their surroundings.

This AI-powered backpack was built using OpenCV’s Artificial Intelligence Kit with Depth (OAK-D), powered by Intel. It consists of a OAK-D spatial AI camera by Luxonis, a host computing unit, a pocket-sized battery pack and a Bluetooth-enabled earphone for providing real-time alerts and approximate location of nearby obstacles. [11]

With the high demand of deep-learning based AI on such edge devices, Intel is focusing on optimizing its processors for inferencing the machine learning models, which led to the birth of Myriad Vision Processing Unit (VPU).

Myriad Vision Processing Unit (VPU), an AI-optimized chip for accelerating vision computing based on convolutional neural networks (CNN). According to Intel, Myriad VPUs have dedicated architecture for high-quality image processing, computer vision, and deep neural networks, making them suitable to drive the demanding mix of vision-centric tasks in modern smart devices. [5]

In this blog, we introduce the Intel Movidius Myriad 2 and Intel Movidius Myriad X and compare performance of publicly available pre-trained Face Detector models (MobileNetSSDV2 Face Detector and Retail Face Detector) and Xailient Detectum™ Face Detector on these two systems.

Intel Movidius Myriad 2

The Movidius Myriad X VPU (Credit: Intel Corporation)

The Myriad 2 by Movidius, an Intel company, is a vision processing unit (VPU) that provides low-power, high-performance vision processing solutions across various target applications including embedded deep neural networks, pose estimation, 3D depth-sensing, visual inertial odometry and gesture/eye tracking. [3]

Using this same chip, Intel created Movidius Neural Compute Stick (NCS), a modular artificial intelligence (AI) accelerator in a standard USB 3.0 stick form factor. This AI accelerator is easy to use and can directly be attached to edge devices such as Raspberry Pi. By using the Intel Movidius NCS with your existing edge device, you can deploy deep-learning based applications on your device and get real-time inference. [2]

Intel Movidius Neural Compute Stick (Image Source)

At Xailient, we did an experiment by attaching Intel Movidius NCS1 to Raspberry Pi3B+ and running two different face detectors (MobileNetSSDV2 Face Detector and Xailient Detectum Face Detector) to compare its performance.

Xailient Face Detector on Intel Movidius Neural Compute Stick

When running MobileNetSSDV2 Face Detector on Intel Movidius NCS1 on Raspberry Pi 3B+ we observed an inference speed of 5 PFS. When running Xailient Detectum Face Detector, the inference speed increased 14.8 times.

	MobileNetSSDV2 Face Detector	Close Range Multi-Face Detection (bounding boxes)
Inference Speed	5 FPS	74 FPS
Size	70 MB	88 KB

Intel Movidius Myriad X

The latest version of intel Movidius VPU is the Myriad X, which is the first VPU to feature the Neural Compute Engine — a dedicated hardware accelerator for deep neural network inference.

The Neural Compute Engine in conjunction with the 16 powerful SHAVE cores and high throughput intelligent memory fabric makes Intel Movidius Myriad X ideal for on-device deep neural networks and computer vision applications. [1]

We wanted to compare the performance of Myriad X compared to Myriad 2 so we got our hands on the OAK device (The OpenCV AI Kit), which is powered by the Myriad X chip.

We observed 3 times acceleration in interference when using the Xailient Close Range Face Detector and 1.2 times speedup when using Xailient Long Range Face Detector.

	Retail Face Detector	Xailient Detectum Close Range Multi-Face Detection (bounding boxes)
Inference Speed	54 FPS (4 SHAVES, 4 CMXs) 32 FPS (1 SHAVES, 1 CMXs)	117 FPS (4 SHAVES, 4 CMXs) 90 FPS (1 SHAVES, 1 CMXs)
Size	1.3Mb	88 KB

The OAK board’s max FPS rate is 120FPS. In this experiment, we reached this limit using just 4 SHAVEs.

Running CV at greater than 45 FPS has diminishing value. But what this illustrates is that because Xailient’s Detectum is so efficient we leave a lot of extra capacity available for running additional models andor processing the regions of interest that the Detectum identifies..

Summary Table

	Intel Movidius Neural Compute Stick (NCS) 1	Intel Movidius Neural Compute Stick (NCS) 1	OpenCV AI Kit (OAK)	OpenCV AI Kit (OAK)
	MobileNetSSDV2 Face Detector	Xailient Detectum Close Range Multi-Face Detection (bounding boxes)	Retail Face Detector	Xailient Detectum Close Range Multi-Face Detection (bounding boxes)
Inference Speed	5 FPS	74 FPS	54 FPS (4 SHAVES, 4 CMXs) 32 FPS (1 SHAVES, 1 CMXs)	117 FPS (4 SHAVES, 4 CMXs) 90 FPS (1 SHAVES, 1 CMXs)
Size	70 MB	88 KB	1.3Mb	88 KB

Xailient and Myriad X won the shootout, with an inference acceleration of 117 FPS. The hardware update from Intel Movidius NCS1 to Myriad X shows well in our test, with Retail Face Detector 5 times faster and Xailient’s software 1.6 times faster versus the prior hardware.

Combining Xailient and Movidius will bring greater capabilities for real-time, edge Computer Vision.

Additional use-cases of computer vision that require high computational power on the edge are:

Person Detection
Face Recognition
Person Counting
Car detection
Pedestrian Detection
Object Tracking
Gesture Detection and Recognition
License Plate Detection and Recognition

The Xailient Detectum SDK for Intel Movidius is available for select customers.

Please reach out to Ray Edwards: [email protected] for more information.