With the rapid development of artificial intelligence technology, gesture recognition systems have become increasingly popular in fields such as smart homes, human-computer interaction, and virtual reality. These technologies not only enhance user convenience but also introduce revolutionary ways of interaction. To meet these application needs, Nuvoton Technology has introduced a reference design for a gesture recognition system, aimed at providing a low-power, high-performance solution for various real-time application scenarios.

The core hardware of this system adopts the Arm® Cortex®-M55- and Arm® Ethos™-U55- based microcontroller. This NuMicro® M55M1 microcontroller not only offers powerful computing capabilities but also features excellent low-power characteristics, making it especially suitable for resource-constrained embedded systems.

The NuMicro® M55M1 comes with 1.5 MB of built-in SRAM and supports a variety of external memory options, including QSPI, SDHC, EBI 16-bit, and HYPERRAM™. Most importantly, the system integrates a powerful Neural Processing Unit (NPU) — the Arm Ethos-U55 — specifically designed to provide robust AI processing power for embedded applications, while also maintaining exceptional energy efficiency. With this hardware architecture, we can run lightweight YOLO models in embedded environments, enabling real-time gesture recognition and providing flexible and efficient solutions for various applications.

The system uses a CMOS camera to capture gesture images, which are displayed in real-time on an LCD. With the M55M1's built-in NPU, the system demonstrates strong AI inference capabilities, excelling in both speed and accuracy. Compared to traditional microcontrollers, the M55M1 offers significantly faster inference speeds and higher precision, which is particularly important for applications requiring real-time responses, such as gesture control, security monitoring, and smart homes.

The Ethos-U55 NPU plays a critical role in this system by leveraging hardware acceleration to greatly enhance neural network computation efficiency. It performs exceptionally well in core deep learning tasks such as matrix multiplication and accumulation (MAC). This allows the lightweight YOLO model to process input images in a very short time and quickly output gesture recognition results, significantly reducing processing time, improving accuracy and reliability, and enabling real-time applications. Furthermore, we have employed quantization and compression techniques to optimize system performance. These techniques not only reduce the model size but also lower the computational load during runtime, making the YOLO model more efficient on the Ethos-U55 NPU, further improving inference speed and reducing system power consumption, which is ideal for battery-powered embedded devices.

Nuvoton’s gesture recognition reference design has a wide range of applications, particularly suited for smart homes, human-computer interaction, and virtual reality. For example, gesture recognition enables contactless control of smart devices in the home; in virtual reality applications, the technology provides a more natural way of interaction, allowing users to intuitively interact with virtual environments. Additionally, the technology can be applied to security monitoring systems, triggering alarms or automated processes by recognizing specific gestures.

System developers are welcome to contact the Nuvoton team through Nuvoton AI webpage www.nuvoton.com/ai "Contact Us" form to explore the new value of endpoint AI together.

Watch the application design video: Gesture Recognition