Facial landmark detection technology is the foundation of emotion recognition and expression analysis. Features of the face, such as movements in the eyes, mouth, and eyebrows, can reflect human emotional states and levels of fatigue. With the latest machine learning software stack and facial landmark detection models, this technology can run on the NuMicro^® M55M1 microcontroller equipped with a Neural Processing Unit (NPU).

What are Facial Landmarks?

Facial landmarks refer to key features identified in facial images, such as the eyebrows, eyes, nose, and mouth. These landmarks can describe the geometric shape and feature locations of the face and can be further used to train deep learning models that determine expressions, emotions, levels of fatigue, and concentration.

NuMicro^® M55M1 ML MCU
The NuMicro M55M1 series is a high-performance microcontroller based on the Arm^® Cortex^®-M55 core, featuring 1.5 MB of SRAM and 2 MB of Flash memory. A key feature of the product is its integration of the Arm^® Ethos™-U55 NPU, specifically designed to accelerate neural network computations. It can execute 256 multiply-accumulate operations per clock cycle and provides hardware support for the operations required by Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), significantly enhancing the efficiency and speed of machine learning inference performed by the MCU.

The M55M1 MCU supports TFLite format neural networks, which are trained using the TensorFlow framework. The neural network weights are quantized to INT8 format using the LiteRT tool (formerly known as TensorFlow Lite) and compiled into instructions for the Ethos-U55 using the Vela compiler, enabling execution on the NPU.

Applications of Facial Landmark Detection in Embedded Systems

Facial landmark detection in embedded systems has broad potential, particularly in products like interactive toys, smart home, and fatigue monitoring systems. Utilizing the M55M1 ML MCU, this technology can achieve low-power, real-time emotion recognition and expression analysis, providing users with a more personalized experience.

For instance, in interactive toys, facial landmark detection allows the toy to recognize the child's emotional state and respond accordingly. When the toy detects the child’s happiness, it can play cheerful music or engage in interaction, whereas when sadness or frustration is detected, it can offer comforting words or stories, enhancing the child's emotional communication and learning experience. In television or computer screens, this technology can be used for fatigue monitoring, reminding users to take breaks by analyzing their facial expressions.

The M55M1 ML MCU combines complete controller functionality with the acceleration capabilities of the NPU, allowing resource-constrained embedded systems to run complex deep learning models, showcasing significant practical value. With the aid of facial landmark detection technology and the M55M1 ML MCU, embedded systems can intelligently understand user states, further promoting the application of artificial intelligence across various products.

System developers are welcome to contact the Nuvoton team through Nuvoton AI webpage www.nuvoton.com/ai "Contact Us" form to explore the new value of endpoint AI together.

Watch the application design video: Facial Landmark Detection