Google Releases Machine Learning Development API for Devices

Google has released LiteRT Tensor API, a lightweight C++ library for running machine learning models directly on devices. Using this API, developers can build, customize, and optimize models on devices such as smartphones and embedded devices without depending on the cloud.

To begin with, "on-device AI" refers to a mechanism in which inference processing is completed on the device in hand rather than on a server. When going through the cloud, network latency and privacy concerns tend to arise, whereas on-device computing makes it easier to achieve faster response times and data processing that stays within the device. Against this backdrop, demand for lightweight AI runtime environments for smartphones and IoT devices has continued to grow in recent years, and LiteRT Tensor API is positioned to respond to this trend.

According to Google, LiteRT Tensor API's distinguishing feature is that it combines native flexibility that allows models to be written from scratch with cross-platform portability that enables operation across different operating systems and hardware. It adopts C++ as the base language and is designed to handle situations requiring performance-focused implementation.

The importance of this API lies in the fact that developers can create on-device models that support multiple platforms using a single codebase. Previously, there was the hassle of rewriting implementations for each target environment, but having an API with portability can lead to reduced development costs. The fact that tool development is advancing as use cases for running AI on devices expand is noteworthy from the perspective of ecosystem formation.

Going forward, how the developer community utilizes LiteRT Tensor API for various applications will be a key indicator of the API's practical value. In particular, comparisons with existing on-device inference frameworks and validation results of performance and power consumption on actual devices will be critical in determining whether adoption expands. It will be important to continue monitoring how much support and documentation Google provides in the future.