Renesas
RZ/V AI

4.00

The best solution for
starting your AI applications.

Provided by Renesas Electronics Corporation




To keep you updated,
Watch our GitHub repository
Watch



This project is maintained by renesas-rz

Hosted on GitHub Pages — Theme by orderedlist

AI Navigator Quick Start Guide
~ How to develop and run RZ/V AI applications on e2 studio ~



This page explains how to develop and run RZ/V AI applications using a GUI environment.
For this GUI environment, Renesas provides AI Navigator, which is a set of plugins for the Renesas IDE e2 studio and supports your application development.

Please read the AI Navigator Release Note first. This document describes the changes, restrictions, and some notes.

Target version AI Navigator V1.1.0
Supported environment Ubuntu 20.04 LTS, Renesas e2 studio 2024-04 (or later) for Linux
Supported Devices RZ/V2L, RZ/V2H
Supported Board (RZ/V2L) RZ/V2L Evaluation Board Kit
(RZ/V2H) RZ/V2H Evaluation Board Kit
Supported Functions of AI Applications (RZ/V2L) Q01-Q11 in RZ/V AI SDK V2.10
(RZ/V2H) Q01, Q08, 01, 02, 07, 11 in RZ/V AI SDK V3.00
Recommended Please also watch "AI Navigator Tutorial" videos. These videos will help you quickly learn how to use AI Navigator.
(Link)
Note For the users who want to use your own customized environment (Linux, target board, and so on), see "Build the Linux and Use your Custom PCB Board or 3rd Party Board".

Introduction

AI Navigator is a set of plugins for e2 studio that makes it easy to run AI applications on Renesas devices.
By using AI Navigator, you can try out Renesas AI application from importing to running on the board with just a few clicks.

In addition, AI Navigator includes some plugins for transfer learning tool (RZ/V AI Transfer Learning Tool) and AI model conversion tool (DRP-AI TVM). That’s why you can also develop your AI embedded systems with AI Navigator, such as training and converting AI models and building application source codes on e2 studio.

The development steps for RZ/V AI applications are shown below. All steps can be controlled by the AI Navigator.
The tools “RZ/V AI Transfer Learning Tool” and “DRP-AI TVM” are called by AI Transfer Learning Tool Plugin and AI Model Conversion Tool Plugin, respectively.

AI Navi usecase

Please follow the steps below to start developing your AI applications with AI Navigator. Note that you can skip some steps depending on your use case.

Preparation

Before you start using the AI Navigator, prepare the necessary equipment and software as described in Step 1 and Step 2 in Getting Started.

Note You must first install Docker before using AI Navigator.
To install Docker, see "1. Set up Docker's apt repository" and "2. Install the Docker packages" on the following Docker web page.
Install using the apt repository in Docker docs

In addition, some plugins in AI Navigator run Docker as a non-root user. See "Manage Docker as a non-root user" in the following Docker web page to run as a non-root user.
Manage Docker as a non-root user in Docker docs

Step 1: AI Navigator Installation

Note AI Navigator V1.1.0 works on e2 studio Linux Host only.
Note If you want to update AI Navigator from the previous version, you can follow the same procedures below.


Step 2: Start AI Navigator

Click [Renesas Views] > [Renesas AI] > [AI Navi] and the AI Navigator will launch.
Note For each view and button in the AI Navigator, see the help page for AI Navigator. You can access it by clicking the [Learn more...] in the start view (on the top page of AI Navigator) or by selecting [Help] > [Help Contents] > [Renesas AI Navigator].
Note If you want to restart the project, select it from the pull-down menu in the start view and click [Continue] in the start view.
AI Navigator top


Step 3: Import AI Applications Project

Import an AI application project from Renesas AI server by the following steps.

1. Click [Create New AI] in the start view.


ainavi

2. Select a category of AI application based on your case. You can search the proper category if you enter keywords in Filter.


Note "Bring Your Own Model" is an empty project for utilizing your own AI models and applications. To understand its usage, see this importing step, and then see B. How to use Bring Your Own Model Project.
ainavi app importing step2

3. Select an AI application.


ainavi app importing step3

4. Click [Import] and import an AI application.


Note Once you import an AI application, you cannot import an AI application which has the same function (same number like "Qxx") in the same work space.
ainavi app importing step4

5. The Project and AI information view will appear if the import is successful.

From this view, you should specify the location of the downloaded and extracted RZ/V AI SDK directory.
If you haven't downloaded the RZ/V AI SDK, which is the environment for the imported AI application, click [Download...] to download and extract it. And then, specify the directory path.
If you've already downloaded and extracted the RZ/V AI SDK, click [Set the download folder...] and specify the directory.
ainavi app importing step5



That’s it for the steps to import an AI application project.
Please proceed to each step according to your requirements.

Click the button As you click each button, the guides for that option appear and the color of the button changes to gray. Click them as needed.
When you click a button again, the option guides close and the button returns to its default color.

Option1: Train AI Model
Option2: Convert AI Model
Option3: Edit Application

Option1: Train AI Model

For RZ/V, you can train any AI models provided in any Renesas AI application using the RZ/V AI Transfer Learning Tool (hereinafter referred to as RZ/V AI TLT).
Note Refer to RZ/V AI Transfer Learning Tool How to Re-train AI model to know the supported AI applications and how to use RZ/V AI TLT. This section just explains launching RZ/V AI TLT from AI Navigator.
Note RZ/V AI TLT launched by AI Navigator V1.1.0 only supports RZ/V2L AI Applications v2.10.

1. Configure the environment for the transfer learning tool
Note This is mandatory only for the first time. If you have already completed this step, you can pass to the next 2. Launch the transfer learning tool.
(1) Click [Start Setting...] to configure the environments for RZ/V AI TLT.
At this time, the download page for RZ/V AI Transfer Learning Tool on the Renesas website will open automatically. Read the "SOFTWARE LICENSE AGREEMENT" and click "Accept and download" if you agree.
After the download is completed, select the downloaded file.

ainavi TLT install 1

(2) The confirmation window appears indicating that it may take a long time to complete, then click [OK].

ainavi TLT install 2

(3) The RZ/V AI TLT setup process begins. During the setup, a window appears prompting you to enter the root password. Enter your root password and the process will continue.

ainavi TLT install 3

(4) When the setup process is complete, the installation window will close automatically.
At this point, please confirm the console on e2 studio and logged as below. If it is the same, the setup process has completed successfully.
Finished at exit code 0
Please confirm the console log.
Note The RZ/V AI TLT is installed as below:
(e.g.) If e2 studio is installed in ~/.local/share/renesas/e2_studio, RZ/V AI TLT is installed in ~/.local/share/renesas/rzv_ai_tlt.

In addition, RZ/V AI TLT provides this user's manual (r21ut0254ej0210-rzv-ai-tlt.pdf), which is stored in rzv_ai_tlt/2.10/docs.

2. Launch the transfer learning tool
Click [Transfer Learning...] and open RZ/V AI TLT.
ainavi

After finishing the transfer learning, you need to convert the trained AI model with DRP-AI TVM to run it on the target board.
Click the above button on this page [Option2: Convert AI Model] and see the procedures.

Option2: Convert AI Model

After training an AI model, convert an AI model to generate an executable runtime on the target device. For RZ/V, DRP-AI TVM is used, which is a machine learning compiler plugin for Apache TVM with AI accelerator DRP-AI.
AI Model Conversion Tool Plugin (hereafter referred to as "TVM Plugin") is a plugin for DRP-AI TVM and enables it to run on e2 studio.

For DRP-AI TVM, please refer to the following website and GitHub page.
Note See the AI Model Conversion Tool Plugin help page for details on how to use it. You can view it by selecting [Help] > [Help Contents] > [AI Model Conversion Tool].
This section briefly describes the steps in the AI model conversion procedure usingy DRP-AI TVM on e2 studio.
Note Some AI models available in Renesas AI application need to skip the preprocessing setting. Refer to the AI Navigator Release Note for the status of support.

1. Configure the environment for DRP-AI TVM on e2 studio
(1) Click [Convert AI model] on AI Navigator menu and open the "Convert AI Model" view.

(2) Click [Start Settings...] and open the DRP-AI TVM.
ainavi tvm_step1-1,2



(3) Confirm that the directory path has set and click [Setup now] if there is no problem.
Note The directory path is set automatically when you specified the directory path in Step 3-4. If the path is not set, click [Browse] and set the path manually.
When the setup is completed, the Installation Success screen appears. Click [Close] to close the window.

ainavi tvm_step1-4

2. Input File Setting
You need to set up the target project for DRP-AI TVM first. Specify the following items.

  • Project name
  • Select your target AI application project.
  • Device
  • Select your target device.
  • AI SDK Docker image name
  • Select the RZ/V AI SDK that you have set up to convert your AI model.
  • Select framework
  • f Select an AI framework that the input AI model file uses.
  • Input model file
  • The file path of your AI model is automatically set by RZ/V AI TLT. It is not necessary to specify this field.
  • Output directory
  • Specify the output directory after conversion.
ainavi
After setting, click [Next].
Note [Setup environment] on this view has the same function as 1. Configure the environment for DRP-AI TVM on e2 studio.
If you have already set up the DRP-AI TVM environment for your application, you do not need to click this button.

3. Preprocess Setting
The pre-processing parameters are already set by default and do not need to be changed. Click [Next] on this Preprocessing Settings screen to proceed to the next step, AI model conversion.
However, if you want to use another input, such as using another camera, etc, configure the preprocessing parameters by following the steps below.

(1) Select the target input node.

(2) Confirm the "Preprocessing output directory" and change it if necessary.
By default, "Use DRP-AI to speed up preprocessing" is checked. If you want to skip preprocessing, clear the check box and click [Next] to go to the next step AI model conversion.

(3) Set the input data parameters for preprocessing.
ainavi tvm preprocessing setting 1-3


(4) Select the preprocessing operator parameters and enter each setting value.
ainavi tvm preprocessing setting 4


(5) Specify the preprocessing output data format. These parameters are passed to the AI model.
ainavi tvm preprocessing setting 5

4. AI model conversion
Click the button The AI model conversion procedure is different for each device.
Click the button below that corresponds to your target device and the appropriate conversion procedure will appear.

RZ/V2L
RZ/V2H

Start the AI model conversion to generate its runtime for RZ/V2L by following the steps below.

(1) Configure the option setting.
Select "Optimization level".
If you want to check edit a conversion script here uses this conversion, click [Check conversion script].

(2)Click [Start conversion] and the conversion process will start. At this time, the console on e2 studio will appear and display logs.

ainavi ai model conversion start


(3) When finishing the conversion, the result will be shown in the "Conversion result" area.
ainavi ai model conversion complete


After the conversion is finished, you need to edit and build your AI application.
Click the above button on this page [Option3: Edit Application] and see the procedures.

Note Before moving to the next step, close the AI Model Conversion Tool.

Start the AI model conversion to generate its runtime for RZ/V2H by following the steps below.

(1) Configure the option setting.Select "Optimization level".
If you want to check edit a conversion script here uses this conversion, click [Check conversion script].

ainavi ai model conversion start(v2h)
(2) Configure the quantization setting.
  1. Select the target input node.
  2. Specify the calibration data directory containing data to be used during the quantization process.
  3. Note For the calibration data directory, specify the directory where the data is stored. TVM Plugin supports the following extensions.
    .jpg .jpeg .png .bmp .gif .tif .tiff .npy
  4. The Mean and Standard deviation parameters are already set by default and do not need to be changed.
  5. Note The Mean and Standard deviation parameters are the most important parameters for maintaining the model's accuracy.
    If you want to change these parameters, make sure that the specified parameters match the normalization parameters used during training.

    For more details on how to change the Mean and Standard deviation, please see the TVM Plugin Help.(Click [Help]-[Help Contents] - [AI Model Conversion Tool].)
(3)Click [Start conversion] and the conversion process will start. At this time, the console on e2 studio will appear and display logs.

(4) When finishing the conversion, the result will be shown in the "Conversion result" area.
ainavi ai model conversion complete (v2h)


After the conversion is finished, you need to edit and build your AI application.
Click the above button on this page [Option3: Edit Application] and see the procedures.

Note Before moving to the next step, close the AI Model Conversion Tool.


Option3: Edit Application

You can modify your AI application source codes and build them on e2 studio.
Click [Edit Application] on the AI Navigator menu and follow the instructions below.

ainavi build

1. Configure the build environment
Click [Start Settings...], and then start configuring the build environment for the AI application.
Once the setup process is complete, the appropriate toolchain for the target e2 studio project is automatically registered.
Note After registering the toolchain, you need to specify the toolchain version for your AI application.
Click [Project] -> [Properties] -> [C/C++ Build] -> [Settings], and click the "Toolchain" tab.
Then, select the toolchain version in "Use integrated tool chain version" as shown in the figure, and click [Apply and Close].
*The toolchain version is (RZ/V2L) 3.1.21, (RZ/V2H) 3.1.26 in AI Navigator 1.1.0.
ainavi build set toolchain version

2. Edit Source codes
Click [Edit...] and edit a source code.
Note When you click [Edit...], a main source file of the target AI application appears.
If you want to edit another source file, select it in the src directory on the project explorer.

3. Build
Click [Build] and start building. The build result will be shown in the console on the e2 studio.


Step 4: Run on the Board

It’s time to run your AI application on the target board.
Click [Run on the Board] on the AI Navigator menu and follow the steps below.

ainavi run ai


1. Format SD card

Insert your micro SD card into your Linux host PC.
Click on [Create a bootable disk…] and you can automatically format your SD for booting the board.

Note For formatting procedure on Linux console, please refer to A2. Format SD card, 1. Setup RZ/V2L EVK or 1. Setup RZ/V2H EVK, and click on eSD Bootloader.
Note for RZ/V2H users bmap-tools is used to set up the RZ/V2H EVK. If you do not have it installed, the following message will appear when creating a bootable disk.
"bmaptool is required for this setup program. Do you want to install it now?"
If you accept it, type "yes" and the creation process will continue.


2. Boot the board

Start booting the board with the formatted SD card. If you are not sure how to boot the target board, click on [Boot instruction…] and the instructions for Boot RZ/V2L Evaluation Board Kit or Boot RZ/V2H Evaluation Board Kit will appear.

Note At this point, be sure to configure each IP address for the connection between your Linux host PC and the target board.
This may cause the later process (copying AI models and applications, and running the AI application) to fail if you have not configured it.

In addition, if you want to use IP addresses other than those set by default in AI Navigator, please see Settings required when using other IP addresses of target board.


3. Copy AI model and Application to the board

Once the board is configured, you are ready to transfer your AI models and AI applications to the board.

Click [Copy AI] to copy your AI models to the board.
Then, click [Copy App] to copy your AI applications to the board.

4. Run AI application

Click [Run the AI] and start running your AI application on the board.

Note Please read the readme.md in each AI application project before using it. Each readme.md describes the specification, such as which input mode (such as MIPI or USB camera) is supported, how to run the application, and so on.

Also, read "RZV_AI_App_mode_switching_guide_en.pdf" in your imported AI application project. This document describes how to change some configurations when changing the input mode of AI application.


This is the end of AI Navigator Quick Start Guide.

Please refer to AI Navigator Quick Start Guide Appendix for the additional information.