Latest Generation of Intelligent Sensors integrate AI and Machine Learning

Member News published by Anglia, under Artificial Intelligence / Machine learning, Components, PR, Sensors

What if data computation along with the AI and Machine Learning that is typically carried out in Edge devices today could be integrated into the sensors themselves? Gregorio Vidal, Field Application Engineer at Anglia, introduces the current generation of sensors from STMicroelectronics featuring a Machine Learning Core.

Growth of ‘Edge’ processing
Sensors have now become pervasive across almost all market segments, from wearable devices which utilise them to monitor users’ movements and vital signs, to sensors attached to industrial machines for Condition Based Monitoring (CbM) to give early warning of wear or failure helping in the drive towards Industry 4.0.

The latest generation of MEMS sensors bring ever increasing levels of reliability and measurement accuracy enabling even deeper insight for the applications which employ them. As this more detailed sensor data increases, the need to process it effectively so it can be interpreted correctly becomes ever more important. Devices with onboard sensors are now networked and connected to the IoT allowing the data they gather to be processed in the cloud and interpreted and acted on efficiently. To reduce some of the limitations associated with cloud processing, such as network availability and bandwidth, and to further maximise speed and efficiency some of this data processing can be carried out locally by “Edge” computing devices. Edge computing helps to increase data privacy, reduce latency and lightens the load on the cloud servers and associated bandwidth.

These Edge devices can also employ Artificial Intelligence (AI) and Machine Learning to further improve performance particularly for time-sensitive applications. For example, in a wearable device measuring a user’s vital signs, it is important to be able to provide a real-time interpretation of the sensor data, whereas in an air quality monitoring device some latency can be tolerated due to the intermittent sampling rates. However, there are also some disadvantages with Edge computing. For example, it requires additional equipment in the form of hubs or router devices to be installed locally further adding cost and complexity.

Sensors with Machine Learning Core
STMicroelectronics is addressing these issues by offering integrated computation within its sensors. The company already offers the widest range of sensors covering a full spectrum of applications from low-power devices for IoT and battery-operated applications to high-end devices for accurate Navigation and Positioning, Industry 4.0, Augmented and Virtual Reality, and Smartphones. They are at the forefront of MEMS sensor development, today their portfolio of sensors covers temperature, humidity, microphone, pressure, proximity, accelerometers, gyroscopes, e-Compasses and multi-axis inertial modules.

These sensors have and continue to be refined by leveraging STMicroelectronics vast application experience and robust and mature manufacturing processes. STMicroelectronics various MEMS sensing elements are manufactured using specialized micromachining processes, while the IC interfaces are developed using CMOS technology that allows the design of a dedicated circuit which is trimmed to better match the characteristics of the sensing element. This enables STMicroelectronics to offer sensors with the lowest power consumption and package sizes in the industry along with excellent stability over the temperature range and low noise which is critical for precision sensing applications.

STMicroelectronics latest generation of sensors, however, are radically different in that they feature an integrated Machine Learning Core (MLC). This allows the sensor to process data locally within the application device rather than via an external Edge device or in the cloud which is common with current setups. These sensors with integrated computation will not be relevant for every application, however they are of particular interest for time-sensitive applications where the data requires local processing to avoid latency issues associated with remote handling and processing of the data.

iNEMO inertial module unit
One of the first sensors from STMicroelectronics to feature an MLC is the iNEMO inertial module unit (IMU) LSM6DSOX. This system-in-package devices features a 3D digital accelerometer and a 3D digital gyroscope; it is ideally suited to motion detection applications.

The sensor comprises of 3 key blocks. In the first block the sensor data is captured, this is then fed into the second computational block where analog and digital signal filtering is applied to enhance the accuracy of the data. Finally, this data passes to the preprogramed decision tree block where the machine learning core processes the data outputting the interpreted result. The MLC effectively moves Edge computing onto the sensor node.

The LSM6DSOX also features a Finite State Machine (FSM). Whereas the MLC allows for activity tracking by comparing the sensor data to predefined or trained patterns in the decision tree, the FSM allows the IMU to perform simple and complex Gesture Recognition using preprogramed state parameters with defined transitions. This FSM capability allows the sensors to be utilised in more advance Augmented Reality, Virtual Reality and sensor fusion applications.

Furthermore, it is also possible to connect additional external sensors to the LSM6DSOX IMU via the serial interface. The external sensor data flows into the computational block of the IMU and benefits from the integrated high pass, band pass and digital filters. The captured sensor data is then processed through a set of decision trees which run the machine learning core and provides the host system processor with results classified by the module.

Energy consumption and performance
Energy consumption is an important consideration where sensors are to be used in battery powered devices such as wearables and remote sensor nodes. The LSM6DSOX is ideal for battery powered applications, it has a low power consumption of 0.55 mA in high-performance mode falling to just 4.4 µA when used in low power modes, this enables always-on low-power functionality providing an optimal experience for the end user. The LSM6DSOX supports mainstream operating system requirements, offering real, virtual and batch sensors with an onboard 9 Kbyte FIFO for dynamic data batching. Using the onboard MLC, the sensor is able to save between 10 and 1000 times more energy vs. using the host system MCU or application processor, in addition detecting accuracy is improved.

The LSM6DSOX has a selectable full-scale acceleration range of ±2/±4/±8/±16 g and an angular rate range of ±125/±250/±500/±1000/±2000 dps making it suitable for broad market use. The device fully supports Electronic and Optical Image Stabilisation (EIS and OIS) camera applications as the module includes a dedicated configurable signal processing path for OIS and an auxiliary SPI, configurable for both the gyroscope and accelerometer. The OIS can also be configured using the SPI, I²C and MIPI I3CSM primary interfaces.

The LSM6DSOX has also been designed with a high tolerance to mechanical shock making it an excellent choice for system designers developing and manufacturing high reliability products for use in industrial or automotive markets.

Other IMU sensors are also available which have been specifically tailored to dedicated applications such as the ISM330DHCX which is ideal for CbM in Industry 4.0 and Industrial IoT applications. STMicroelectronics also recognise AI will be key for supporting real-time applications that rely on sensor data. The next generation of sensors being released employ the STRed – Intelligent Sensor Processing Unit (ISPU) an ultra-low-power, high-performance programmable core which can execute signal processing and AI algorithms.

Development tools
To assist designers, STMicroelectronics have developed a comprehensive suite of support material including product presentations, tutorial videos, application libraries and algorithms along with hardware and software evaluation kits. Example libraries and algorithms are available for common motion detection applications such as walking, running, cycling, gesture recognition and driving.

One example of this is the SensorTile.box ready to go IoT node, this evaluation kit is suitable for users of any skill level to support learning and prototyping. 

 

Housed in an IP54 case the kit integrates multiple motion sensors, including the LSM6DSOX IMU, multiple environmental sensors, a low power STM32L4R9 MCU, Bluetooth Low Energy connectivity module and a battery power management circuit.

There is an easy-to-use BLE app available on both Google Play and the App Store which can be used with the evaluation kit to view the captured data from the onboard sensors.

Other resources include video’s demonstrating the SensorTile.box being used in real world applications such a smart kettle. In this application the machine learning sensor detects the water has boiled and alerts the user, helping prevent the water from going cold thereby saving energy by reducing the need to re-boil the kettle. A whole host of other application examples and on demand webinars can be found on the STMicroelectronics website.

Design support
Anglia offers support for customer designs with free evaluation kits, demonstration boards and samples of STMicroelectronics products via the EZYsample service which is available to all registered Anglia Live account customers.

Anglia’s engineering team are also on hand to support designers with their extensive experience of sensor-based designs and can offer advice and support at component and system level. This expertise is available to assist customers with all aspects of their product design, providing hands on support and access to additional comprehensive STMicroelectronics resources including technical application notes and reference designs.

Visit www.anglia-live.com to see the full range of STMicroelectronics products available from Anglia.

Subscribe to the CW newsletter

This site uses cookies.

We use cookies to help us to improve our site and they enable us to deliver the best possible service and customer experience. By clicking accept or continuing to use this site you are agreeing to our cookies policy. Learn more

Start typing and press enter or the magnifying glass to search

Sign up to our newsletter
Stay in touch with CW

Choosing to join an existing organisation means that you'll need to be approved before your registration is complete. You'll be notified by email when your request has been accepted.

i
Your password must be at least 8 characters long and contain at least 1 uppercase character, 1 lowercase character and at least 1 number.

I would like to subscribe to

Select at least one option*