What a small device can do: Pose estimation using a fanless edge AI accelerator (YOLOv8-pose)

Introduction

Hello!
I am a member of the development department at Nexty Electronics.
In my daily work, I am in charge of providing technical support for various edge AI devices.

In this column, we will introduce an example of running the edge AI accelerator Hailo-8 on Radxa's single-board computer ROCK 5 Model B.
Excluding the time it takes to download the OS, it will take less than an hour to try it out.

Hailo-8 is an AI accelerator that is also installed in Raspberry Pi's AI HAT+.
It has high computing performance of up to 26 TOPS and can run a variety of image-based AI models.

The ROCK 5 Model B, which is used as the base board, is equipped with four Cortex-A76 cores, the same as the Raspberry Pi 5.
It also comes standard with an M.2 CONNECTORS (M-Key), making it easy to connect SSDs, Hailo-8 M.2 modules, etc.
The recommended POWER SUPPLIES is USB-C PD 12V/3A.

In this article, we will introduce the steps to perform a demo of human pose estimation by combining ROCK 5 Model B and Hailo-8.

Example of human pose estimation using Hailo-8

Product Link

Preparing the ROCK 5 Model B

Installing Armbian

In this article, we will use Armbian as the OS.
Please download the images for ROCK 5 Model B from the link below.

Radxa Rock 5B - Armbian

In the case of image processing like this time, it is a good idea to install a desktop environment.
However, a GUI environment is not required and Hailo-8 will work fine in a headless environment. Choose the environment that best suits your needs.

Also, depending on the version of Armbian, you may not be able to install the PCIe driver due to kernel header inconsistencies, etc. This article uses Armbian 25.11.1 Noble GNOME Desktop, but if the build does not succeed, try a different image.

Once the file is downloaded, you can write it to the SD card on your host PC. Writing can be done on either Windows or Linux.
This time, I used Linux (Ubuntu) on the host PC and wrote it using the following command.
If xz is not found, run sudo apt install xz-utils to install it.

# SDカードのドライブを確認。今回は/dev/sdcとして認識された
$ lsblk
...
sdc       8:32    1  28.9G  0 disk
┗━sdc1    8:33    1  28.5G  0 part /media/armbi_root

# 書き込み先をof=/dev/sdcに指定。間違えると書き込み先のデータがすべて消去されるので注意！
$ xz --decompress --stdout Armbian_25.11.1_Rock-5b_noble_vendor_6.1.115_gnome_desktop.img.xz | sudo dd of=/dev/sdc bs=4M status=progress conv=fsync

# SDカードを取り外す前に同期します
$ sync

The following steps will be performed on the ROCK 5 Model B.

Preparing the Hailo-8 execution environment

Launching the browser

1. Insert the SD card with the OS written to it and the Hailo-8 M.2 module into the ROCK 5 Model B, connect the network CABLES, and then POWER SUPPLIES on.
When you start it for the first time, you will be asked to enter a new root password, username, etc. Once the settings are complete, the Armbian Linux GUI environment will start.

2. Press the application menu button (white rounded rectangle) in the upper left corner and a menu will appear at the bottom of the screen.
Click Show Applications (the nine white dots icon) on the far right.

3. Launch Chromium Web Browser.

Download the Hailo-8 execution environment file

To operate the Hailo-8 M.2 module, you must install the PCIe driver and runtime environment.

1. Enter https://hailo.ai in the address bar of your Chromium Web Browser to go to the Hailo website. If you are not registered, please register.

2. Once you log in, you will have Access to the Developer Zone.

3. Click SW Downloads at the top of the screen. Select it as shown in the image below.

4. Download the following two packages from the list (the latest version at the time of writing is 4.23.0):

HailoRT - PCIe driver Ubuntu package (deb)
HailoRT - Ubuntu package (deb) for arm64

If the file does not download, click the download icon (down arrow) in the top right corner of your browser. If "Unverified download blocked" appears in the Recent download history, click that item to allow the download.

5. Start the terminal software and check that the file has been downloaded.

$ cd Downloads
$ ls 
hailort-pcie-driver_4.23.0_all.deb  hailort_4.23.0_arm64.deb

Installing the Hailo-8 runtime environment

1. Execute the following command to install the PCIe driver and execution environment. Enter Y for all questions about "Do you wish to use DKMS?". You will also be asked about the hailort service, but it is recommended that you enable it.

# 依存するツールの導入
$ sudo apt update
$ sudo apt install -y dkms build-essential linux-headers-vendor-rk35xx

# Hailo-8実行環境のインストール
$ sudo dpkg -i hailort-pcie-driver_4.23.0_all.deb
$ sudo dpkg -i hailort_4.23.0_arm64.deb

# 再起動
$ sudo reboot

2. After restarting the OS, start the terminal and enter the following command. If the device is recognized and the following message is displayed, the command was successful.

$ hailortcli scan
Hailo Devices:
[-] Device: 0000:01:00.0

Running the sample code

Get the source code

1. In the terminal, go to the appropriate folder and run the following:

$ git clone https://github.com/hailo-ai/Hailo-Application-Code-Examples.git
# 本ドキュメント執筆時のバージョンに切り替えます
$ cd Hailo-Application-Code-Examples
$ git checkout 55c64bf

2. Once you have downloaded the source code, go to Hailo-Application-Code-Examples/runtime/hailo-8/cpp. You will find the following sample code:

application	Commentary
zero_shot_classification	Zero-shot image classification with CLIP
classifier	ImageNet image classification
depth_estimation	Depth estimation with scdepthv3 or stereonet
onnxruntime	Inference and post-processing with ONNX Runtime
instance_segmentation	Instance segmentation using YOLOv5 and YOLOv8
object_detection	Object detection
pose_estimation	Human Pose Estimation with YOLOv8
semantic_segmentation	Semantic Segmentation with ResNet-18

This time we will try to build pose_estimation.

Installing Tools and Libraries

1. Install the tools and libraries required for building

$ sudo apt install -y git cmake g++ make pkg-config libopencv-dev

2. Change directory and run get_sources.sh to download the YOLOv8 model compiled for Hailo-8.

$ cd Hailo-Application-Code-Examples/runtime/hailo-8/cpp/pose_estimation/yolov8_pose
$ ./get_sources.sh

Code fixes

The sample code depends on the tensor operation libraries xtensor and xtl. The include paths are different in recent versions, so they need to be modified. Also, errors or warnings may occur depending on the version of gcc installed on Armbian Linux. Modify the path appropriately as shown below.

CMakeLists.txt

Change the installation location of the xtl library

# 修正前（xtl_DIRのパスからx86_64を削除）
ExternalProject_Add(xtensor-test
  GIT_REPOSITORY https://github.com/xtensor-stack/xtensor
  CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${EXTERNAL_INSTALL_LOCATION} -Dxtl_DIR=${BASE_DIR}/build/x86_64/external/share/cmake/xtl/)

# 修正後
ExternalProject_Add(xtensor-test
  GIT_REPOSITORY https://github.com/xtensor-stack/xtensor
  CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${EXTERNAL_INSTALL_LOCATION} -Dxtl_DIR=${BASE_DIR}/build/external/share/cmake/xtl/)

std::move()に関する警告がエラー扱いされるのを抑制

# 修正前
target_compile_options(${PROJECT_NAME} PRIVATE ${COMPILE_OPTIONS} -fconcepts)

# 修正後
target_compile_options(${PROJECT_NAME} PRIVATE ${COMPILE_OPTIONS} -fconcepts -Wno-error=pessimizing-move)

yolov8pose_postprocess.hpp

// 修正前
#include <xtensor/xview.hpp>
#include <xtensor/xsort.hpp>

// 修正後
#include <xtensor/views/xview.hpp>
#include <xtensor/misc/xsort.hpp>

common/math.hpp

// 修正前
#include "xtensor/xarray.hpp"
#include "xtensor/xeval.hpp"
#include "xtensor/xsort.hpp"
#include "xtensor/xview.hpp"
#include "xtensor/xio.hpp"

// 修正後
#include "xtensor/containers/xarray.hpp"
#include "xtensor/core/xeval.hpp"
#include "xtensor/misc/xsort.hpp"
#include "xtensor/views/xview.hpp"
#include "xtensor/io/xio.hpp"

common/tensors.hpp

// 修正前
#include "xtensor/xarray.hpp"

// 修正後
#include "xtensor/containers/xarray.hpp"

Build and run

1. Build. If successful, build/vstream_yolov8pose_example_cpp will be generated.

$ cmake -S. -B build -DCMAKE_BUILD_TYPE=Release
$ cmake --build build

$ ls build

2. When you run vstream_yolov8pose_example_cpp, the detected results will be output to output_image.jpg.
It also displays the average FPS during execution.

option	Commentary
-hef	HEF files
-input	Input image
-num	Number of repetitions

For -hef, specify yolov8s_pose.hef downloaded with get_sources.sh.

Example

$ ./build/vstream_yolov8pose_example_cpp -hef=./yolov8s_pose.hef -input=person.jpg -num=100

person.jpg (input)
output_image.jpg (output)

A total of 17 body parts can be detected: the person's eyes, ears, nose, shoulders, elbows, hands, hips, knees, and ankles.
When I measured it myself, the average FPS after 100 runs was 97.23.

Summary

This article introduced an example of using Hailo-8 to quickly run an image AI model. In addition to the model introduced here, various other models optimized for Hailo-8 are available, making it easy to perform high-performance AI processing. Models can be downloaded from the Hailo Model Explorer Vision website below.

Hailo Model Explorer Vision

You can also convert any model not in the catalog using the free Hailo Dataflow Compiler, which supports the ONNX and TensorFlow Lite (TFLite) formats.

With a wealth of samples provided, including Hailo-Application-Code-Examples, you can quickly create your own applications. We encourage you to take a step into the world of edge AI with Hailo-8!

Additionally, the NEXTY Electronics Development Department provides product development and technical support using NXP's i.MX series.
We can introduce various products tailored to your needs, such as examples of combining i.MX 8 or i.MX 95 with Hailo-8 or Hailo-10H, so please feel free Inquiry.