The AI Engine will be supported on Snapdragon 845, 835, 820 and 660 mobile platforms, with cutting-edge on-device AI processing found in the Snapdragon 845.
Maximizing intelligence on client devices residing on the edge of the network ensures AI-powered user experiences can be realized with better overall performance and with or without a network connection. The key benefits of on-device AI include real-time responsiveness, improved privacy, and enhanced reliability. Supported within the AI Engine, Snapdragon core hardware architectures – Qualcomm® Hexagon™ Vector Processor, Qualcomm® Adreno™ GPU and Qualcomm® Kryo™ CPU – are engineered to run AI applications quickly and efficiently on-device. This heterogeneous computing approach gives developers and OEMs the ability to optimize AI user experiences on smartphones and other edge devices.
Key software-centric components of Qualcomm AI Engine, include:
- Snapdragon Neural Processing Engine (NPE) software framework is designed to make it easy for developers to choose the optimal Snapdragon core for the desired user experience – Hexagon Vector Processor, Adreno GPU and Kryo CPU – and accelerate their AI user experiences on device. The Snapdragon NPE supports the Tensorflow, Caffe and Caffe2 frameworks, in addition to the Open Neural Network Exchange (ONNX) interchange format, offering developers greater flexibility and choice on multiple Snapdragon platforms and operating systems.
- Support for the Android Neural Networks API, first released in Google’s Android Oreo, gives developers access to Snapdragon platforms directly through the Android operating system. Support for Android NN will appear first in Snapdragon 845.
- Hexagon Neural Network (NN) library allows developers to run AI algorithms directly on the Hexagon Vector Processor. This provides optimized implementation for the fundamental machine learning blocks and significantly accelerates AI operations such as convolution, pooling and activations.
“On-device AI demands multiple hardware architectures, software tools, and frameworks to meet developer demand and performance thresholds for the myriad of AI-powered features and apps arriving on smartphones this year,” said Gary Brotman, director of product management, Qualcomm Technologies, Inc. “The AI Engine capabilities of select Snapdragon mobile platforms have and will continue to fuel AI innovation among our OEMs and ecosystem partners, and are engineered to accelerate the development and delivery of rich, seamless and compelling features for consumers.”
Ecosystem Momentum
Many smartphone manufacturers have taken advantage of AI Engine components on Snapdragon mobile platforms to accelerate AI applications on their devices. These include Xiaomi, OnePlus, vivo, OPPO, Motorola, ASUS, ZTE, Nubia, Smartisan and Blackshark, among some of which are planning to optimize AI applications using the AI Engine on future flagship smartphones using Snapdragon.
Leading AI softare developers deliver purpose-built use cases optimized for Snapdragon mobile platforms. SenseTime and Face++ offer a variety of pre-trained neural networks for image and camera features including single camera bokeh, face unlock, and scene detection. Elliptic Labs provides Snapdragon customers with ultrasound-based gesture controls for smartphones. ArcSoft provides single camera and dual camera algorithm to Snapdragon customers and is actively working on the development of AI user experiences. Thundercomm offers comprehensive solutions for AI vision use cases and devices. Uncanny Vision provides optimized models for people, vehicle and license plate detection and recognition. These companies and many more all utilize Qualcomm AI Engine components for accelerating performance of their AI models and user experiences.
Cloud leaders have also optimized AI features in applications on Snapdragon-based smartphones. For example, Tencent recently launched an engaging feature in the popular Mobile QQ social platform called High Energy Dance Studio.The Mobile QQ application for Android uses AI Engine components to accelerate frame rates of the popular game. Additionally, Baidu intends to fully support Qualcomm AI Engine and its ecosystem.
Qualcomm Technologies will showcase the following demonstrations at the Qualcomm Booth #E310 in Hall 3 at Mobile World Congress 2018: Deep Portrait, INNER MAGIC™ from Elliptic Labs, Style Transfer and 2D Face Unlock from Sensetime, Animoji from Face++ and the Mobile QQ application High Energy Dance Studio from Tencent.