DRP-AI TVM on RZ/V series

DRP-AI TVM is Machine Learning Compiler plugin for Apache TVM with AI accelerator DRP-AI provided by Renesas Electronics Corporation.

What's new

2025.3.11

• DRP-AI TVM v2.5.0  is now available.
  • Newly support RZ/V2N device

For more details, see Release Note of DRP-AI TVM GitHub.

Getting Started

DRP-AI TVM is provided as a tool for deploying AI models to RZ/V MPUs. In addition, an optional tool for pruning AI models (DRP-AI Extension Pack) is provided. Please refer to the following documents for how to use these tools.

AI model compiler (DRP-AI TVM)

• Tool to convert wide-range AI models for RZ/V.
• How to Compile Sample Model : Tutorial with sample AI model (See here first).
• How to Compile Your Own Model: Tutorial with your own AI model.

AI model pruning support tool (DRP-AI Extension Pack) for RZ/V2H, RZ/V2N

• Tool to provide a pruning function optimized for RZ/V2H, RZ/V2N.
• How to Prune Your Own Model: Tutorial for Pruning with your own AI model.

Features of DRP-AI development environments

BYOM (Bring Your Own Model) support

• For engineers with experience in AI development, "DRP-AI TVM", an end-to-end tool is provided, which allow users deploy users' own AI models on the RZ/V.
  For details, see BYOM tutorial.
  
• For engineers with no experience in AI development, "Renesas AI applications" are provided, which allow users to use pre-trained models and apps as is.
  For details, see Renesas AI Applications.
  

Multiple AI framework support

• DRP-AI TVM is provided, which is optimized for DRP-AI and is based on Apache TVM, a widely used open source AI compiler.
• AI models from multiple AI frameworks (ONNX, Pytorch, Tensorflow) can be converted into object files that can be executed on the RZ/V series.(*)
(*) Pruning model generation support is available only for Pytorch and Tensorflow.
• By combining Renesas' AI accelerator "DRP-AI" with the ARM CPU, more AI models can be run on the RZ/V.

Optimized for the AI accelerator on RZ/V (DRP-AI)

• High power efficiency is achieved through cooperation between hardware (DRP-AI) and software (DRP-AI Tool).
• By using the DRP-AI Tool, you can easily deploy AI models that are optimized to maximize the performance of DRP-AI.
• In addition, for the RZ/V2H, RZ/V2N, it is possible to generate pruning models optimized for the DRP-AI architecture, resulting in higher power efficiency.
• For details, see below:
• Overview of DRP-AI hardware and tool optimization technology (whitepaper)
• Overview of optimization technology for lightweight models of RZ/V2H, RZ/V2N (V2H whitepaper, V2N whitepaper)

AI model weight reduction support(RZ/V2H, RZ/V2N)

• The built-in calibration function of DRP-AI TVM automatically generates INT8 models that are lighter than FP32 AI models.(*)
(*) A few dozen sample images are required.
• A pruning tool is provided to reduce the weight data of AI models by up to approximately 90%.(**)
(**) Patches and guides for modifying the AI ​​framework's training script for DRP-AI will be provided as an Extension Pack. The training script for Pytorch or Tensorflow must be prepared by the user.
• For details, see below:
• The AI model weight reduction

A function to check performance and accuracy before implementation on a device

• Supports estimating and outputting the inference time when implementing on a device when compiling an AI model in DRP-AI TVM.
• Supports "Interpreter mode" that outputs quantized AI models in ONNX format. The same scripts as for the pre-quantization AI model (**) can be used, making it easy to check the recognition accuracy, etc.
(**) The inference script must be prepared by the user.

Reference sample applications

To support the development of applications using DRP-AI TVM, source code for sample applications of representative AI tasks is provided. Please use them as a reference for how to implement the API of DRP-AI TVM, how to input images from a camera connected to an evaluation board, and how to output inference results to an HDMI display, etc.

Classification (ResNet50v1)
Source Code
Video

Object Detection (YOLOX-s)
Source Code
Video

Pose Estimation (HRNet)
Source Code
Video

Semantic Segmentation (DeepLabV3)
Source Code
Video

Demo

With the demo binary, users can try the Reference sample applications using GUI application.
Download the Demo Binary and refer to How to Use Guide for more details.

RZ/V2H	RZ/V2N	Description
Demo Binary (USB)	Demo Binary (USB)	SD card image file (USB camera version)
Demo Binary (MIPI)	Demo Binary (MIPI)	SD card image file (MIPI camera version)
Licence	Licence	OSS license list for software included in the SD card image file.
How to Use Guide	How to Use Guide	A quick guide to using SD card image.

Video

Following videos show the overview of DRP-AI TVM. Refer to them first to understand the overview.

DRP-AI TVM overview

How to run ONNX model (YouTube)

Target hardware

RZ/V series with dedicated AI accelerator (DRP-AI)

• Highly power efficient and flexible by co-operation between high-speed MAC unit for AI and Dynamically Reconfigurable Processor (DRP)
• RZ/V2H, RZ/V2N supports efficient calculation of compacted AI models (INT8 quantization and pruning (option))

	RZ/V2L	RZ/V2MA	RZ/V2M	RZ/V2H	RZ/V2N
CPU	Cortex-A55 Dual/Single Cortex-M33 Single	Cortex-A53 Dual	Cortex-A53 Dual	Cortex-A55 Quad Cortex-R8 Dual Cortex-M33 Single	Cortex-A55 Quad Cortex-M33 Single
AI accelerator	DRP-AI 1 TOPS	DRP-AI 1 TOPS	DRP-AI 1 TOPS	DRP-AI3 8 Dense TOPS 80 Sparse TOPS	DRP-AI3 4 Dense TOPS 15 Sparse TOPS
Data type	FP16	FP16	FP16	INT8	INT8
Compaction model support	-	-	-	Performance optimization for pruned model	Performance optimization for pruned model
Production page	Link	Link	Link	Link	Link