HMI Applications¶

There are several ways to explore running HMI applications using the HMI SDK on RZ/G EVK boards.

This page will guide you through:

• Trying the out-of-box Demo Applications
• Building and deploying Sample Applications

After completing these, you can also develop your own HMI applications by referring to the instructions provided in Wiki.

Hardware Setup¶

Necessary Equipment¶

• Please prepare the following required equipment.

EVK Peripheral Setup¶

Please set up the necessary equipment referring to the EVK peripheral diagram below.

EVK DIP Switch Setup¶

• eSD Boot Mode

• QSPI Boot Mode

Demo Applications¶

This section explains how to run the HMI demo applications and provides detailed introductions for each one.

The HMI SDK Demo Launcher starts automatically when the EVK boots. Currently, it includes the following prebuilt demo applications. If you want to customize them, their source code can be downloaded from the GitHub links listed in the table below.

Please follow the steps below to run the demo applications.

1. Prepare the necessary equipment and configure the EVK DIP switches by following the instructions in Hardware Setup.

2. Insert the bootable microSD card into the microSD card slot for eSD boot mode, and then power on the EVK board.

   Tip

   • Please refer to the EVK Peripheral Setup for the location of the microSD card slot.
   • Press and hold the power button (red button, POWER SW) for 1 second to turn on the EVK board, and for 2 seconds to turn it off.

3. The HMI SDK Demo Launcher launches automatically once the device is fully booted.
   

   Right after boot, you will see the launch window as shown below.

   

   After a few seconds, the HMI SDK Demo Launcher will appear.
   Click the corresponding button to try each demo application we provide.

   

   Tip

   If you want to exit a demo application and return to the HMI SDK Demo Launcher, you can either
   * restart your EVK board by press reset button (blue button, SW10) or
   * run the following command in your board terminal to relaunch it:

   /usr/share/demo-launcher/start_demo.sh 
   

   Note that this is temporary behavior, and we will improve it in future releases.

   Tip

   If you want to use the audio output function, open a terminal, and log in as root. This will enable audio output and set default audio volume.
   This behavior will be improved in a future release.
   For detailed information about logging in the system, see section "5.2 Power on the board and Startup Linux" of "RZ/G3E-EVKIT Linux Start-up Guide" included in RZ/G3E Board Support Package V1.0.0.

   
   

   • LVGL Demo Applications
   LVGL Benchmark DemoLVGL Home Panel Demo

   This demo application is implemented using LVGL, and it includes the following tools for performance testing.

   

   Please click each tool and follow the detailed instructions to try them.
   After you finish trying the tools, click the Quit button at the top-right corner to exit.

   

   This demo application is implemented using LVGL.
   Click the buttons or use the sidebar to explore the available functions.

   

   For example, by clicking the Image Gallery button, you will see the screen shown below.

   

   For example, by clicking the Home Automation button, you will see the screen shown below.

   

   
   

   • Chromium Demo Applications
   Chromium Home Panel Demo

   This demo application is implemented using Chromium, and it presents an HTML5 single-page home panel interface.

   When you start this demo, you may see the following window. In that case, click 'Don't sign in' button.

   

   Click the buttons or use the sidebar to explore the available functions.

   

   For example, by clicking the Video Player button, you will see the screen shown below.

   

   For example, by clicking the Home Automation button, you will see the screen shown below.

   

   The live camera function is also available. The image below is captured using USB Camera. Please switch the camera display mode using the control on top.

   

   Tip

   When you click UI parts repeatedly, the demo may crash. If you encounter this issue many times, change the "VBUS_SEL" rotary switch on the RZ/G3E EVK Carrier Board (RZ SMARC Series Carrier Board II) to the 40W setting (VBUS_SEL = 4). For detailed information about the switch settings, see section "2.3.1 VBUS_SEL – Main Power" of "RZ SMARC Series Carrier Board II User's Manual".

   
   

   • Flutter Demo Applications
   Flutter samples

   You can try three types of Flutter sample applications to see what features are provided. These samples are open-source software distributed on GitHub.

   Material 3 demo

   Animation demo

   Shopping demo

Sample Applications¶

This section explains how to build, deploy, and run the HMI sample applications, as well as provides detailed introductions for each one.

We have prepared the following sample applications.
Their source code can be downloaded from the GitHub links listed in the table below.

The executable binaries are also available in the bin directory of each sample application, so you can run the samples without building them yourself. If you choose to use the binaries, simply download them, skip Step 1, and start from Step 2: Deploy Sample Applications.

Step 1: Build Sample Applications¶

1. Install cross-compilation toolchain.

   • When Using the HMI SDK Package

     If you download the HMI SDK Package and follow the Getting Started setup instructions, you will find the toolchain installer script included in the package. Follow the steps below to install the toolchain for your board.
     

     RZ/G3E

     Target Board	Toolchain Installer Script	Location
     RZ/G3E	rz-vlp-glibc-x86_64-core-image-weston-cortexa55-smarc-rzg3e-toolchain-5.0.8.sh	toolchain-installer_rzg3e_hmi-sdk_v3.4.0.0.zip

     Unzip the package, and extract the installer.

     Navigate to the directory where the HMI SDK Package was extracted.

     cd RTK0EF0195F03400SJ_rzg3e/
     unzip toolchain-installer_rzg3e_hmi-sdk_v3.4.0.0.zip
     

     Then, install the toolchain.

     cd toolchain-installer_rzg3e_hmi-sdk_v3.4.0.0/
     sudo sh rz-vlp-glibc-x86_64-core-image-weston-cortexa55-smarc-rzg3e-toolchain-5.0.8.sh
     

     Tip

     Please set the directory for installing the toolchain.
     

     The default <path-to-toolchain> is /opt/rz-vlp/5.0.8, which is used in the instructions below.
     If you set the target directory manually, please remember to update the path accordingly in the next step.

     
     

   • When Using the HMI SDK Yocto Build Package

     If you download the HMI SDK Yocto Build Package, please ensure that you have completed Step 2-10: Create the SDK toolchain in Building the HMI SDK with Yocto. Then follow the steps below to install the toolchain for your board.

     Tip

     ${WORK} refers to the directory you set in Step 2-3. Set environment variables in Building the HMI SDK with Yocto.

     RZ/G3E

     Target Board	Toolchain Installer Script	Location
     RZ/G3E	rz-vlp-glibc-x86_64-core-image-weston-cortexa55-smarc-rzg3e-toolchain-5.0.8.sh	${WORK}/build/tmp/deploy/sdk/

     Install the toolchain.

     cd ${WORK}/build/tmp/deploy/sdk/
     sudo sh rz-vlp-glibc-x86_64-core-image-weston-cortexa55-smarc-rzg3e-toolchain-5.0.8.sh
     

     Tip

     Please set the directory for installing the toolchain.
     

     The default <path-to-toolchain> is /opt/rz-vlp/5.0.8, which is used in the instructions below.
     If you set the target directory manually, please remember to update the path accordingly in the next step.

2. Enable cross compiler.

   source <path-to-toolchain>/environment-setup-cortexa55-poky-linux 
   

   By default:

   source /opt/rz-vlp/5.0.8/environment-setup-cortexa55-poky-linux 
   

3. Get the sample application source code.

   Navigate to the directory where you want to store the source code.

   git clone https://github.com/renesas-rz/rzg_hmi_sdk
   

4. Go to the source code working directory.

   cd <path-to>/rzg_hmi_sdk/sample_app/<sample-application-directory>
   

   Tip

   Existing <sample-application-directory>entries are listed in the table below.

   <sample-application-directory>	Description
   lvgl/lvgl_sample_img_disp	LVGL Sample Application for Image Display
   lvgl/lvgl_sample_audio_playback	LVGL Sample Application for Audio Playback
   lvgl/lvgl_sample_video_playback	LVGL Sample Application for Video Playback

5. Build sample applications.

   MACHINE=smarc-rzg3e make
   

   The executable binaries for the sample applications are built as follows:
   rzg_hmi_sdk/sample_app/<sample-application-directory>/<sample-application-binary>.

   Tip

   The executable binaries file name (<sample-application-binary>) is the same as its directory name.

   For example, for lvgl_sample_img_disp, the path to the built executable is:

   <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_img_disp/lvgl_sample_img_disp
   

Step 2: Deploy Sample Applications¶

Option 1. Over Ethernet (using SCP)¶

1. Connect the EVK board to your Linux PC and boot it.

   Connect an Ethernet cable between the EVK’s port and your Linux PC.

   Note

   Refer to EVK Peripheral Setup for details.

   RZ/G3E

   Ethernet 0 and Ethernet 1 are available; either port can be used.

   Press and hold the power button (red button) for 1 second to turn on the EVK board.
   
   

2. Check your EVK board’s IP address.

   Run the ip command on your EVK board as follows.

   ip a
   

   Tip

   For example, if the RZ/G3E board is connected to the local network through the Ethernet 0 port:

   root@smarc-rzg3e:~# ip a | grep end
   5: end0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
       inet 192.168.20.99/24 brd 192.168.20.255 scope global end0
   6: end1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
   

   RZ/G3E board's IP address is 192.168.20.99 (at end0).

3. Copy the executable binary files and resource files to your EVK board (external SD card storage).

   Copy the executable binary files from your Linux PC (Ubuntu) using scp command.

   LVGL Sample Application for Image DisplayLVGL Sample Application for Audio PlaybackLVGL Sample Application for Video Playback

   scp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_img_disp/lvgl_sample_img_disp root@<your-EVK-IP>:/home/root/
   

   scp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_audio_playback/lvgl_sample_audio_playback root@<your-EVK-IP>:/home/root/
   

   scp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_video_playback/lvgl_sample_video_playback root@<your-EVK-IP>:/home/root/
   

   Tip

   • We recommend copying the file to the /home/root directory.
     
   • Don’t forget to replace <your-EVK-IP> with your EVK's IP address.

   
   Copy resource files from your Linux PC (Ubuntu) using scp command.

   LVGL Sample Application for Image DisplayLVGL Sample Application for Audio PlaybackLVGL Sample Application for Video Playback

   Notice

   Please make sure the /usr/share/images/ exists on your EVK before using scp.
   If it does not exist, create it using the following command:

   mkdir -p /usr/share/images/
   

   scp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_img_disp/sample/* root@<your-EVK-IP>:/usr/share/images/
   

   Notice

   Please make sure the /usr/share/sounds/sample/ exists on your EVK board before using scp.
   If it does not exist, create it using the following command:

   mkdir -p /usr/share/sounds/sample/
   

   scp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_audio_playback/sample/* root@<your-EVK-IP>:/usr/share/sounds/sample/
   

   Notice

   Please make sure the /usr/share/movies/ exists on your EVK board before using scp.
   If it does not exist, create it using the following command:

   mkdir -p /usr/share/movies/
   

   scp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_video_playback/sample/* root@<your-EVK-IP>:/usr/share/movies/
   

   Tip

   Don’t forget to replace <your-EVK-IP> with your EVK's IP address.

Option 2. Over SD Card¶

1. Insert your SD card into the Linux PC and mount it.

   Notice

   Make sure to turn off your EVK board before ejecting the SD card.
   Press and hold the power button for 2 seconds to turn it off.

2. Copy the executable binary files and resource files to your SD card.

   Tip

   Follow the instructions in Step 4 in Option 2 to check where your SD card is mounted using the lsblk command.

   For example, if the SD card is mounted at /media/user/rootfs.

   Copy the executable binary files.

   LVGL Sample Application for Image DisplayLVGL Sample Application for Audio PlaybackLVGL Sample Application for Video Playback

   cp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_img_disp/lvgl_sample_img_disp /media/user/rootfs/home/root/
   

   cp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_audio_playback/lvgl_sample_audio_playback /media/user/rootfs/home/root/
   

   cp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_video_playback/lvgl_sample_video_playback /media/user/rootfs/home/root/
   

   Tip

   We recommend copying the file to the /home/root directory.

   
   Copy resource files.

   LVGL Sample Application for Image DisplayLVGL Sample Application for Audio PlaybackLVGL Sample Application for Video Playback

   Notice

   Please make sure the /usr/share/images/ exists on your SD card before copying.
   If it does not exist, create it using the following command:

   mkdir -p /media/user/rootfs/usr/share/images/
   

   cp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_img_disp/sample/* /media/user/rootfs/usr/share/images/
   

   Notice

   Please make sure the /usr/share/sounds/sample/ exists on your SD card before using scp.
   If it does not exist, create it using the following command:

   mkdir -p /media/user/rootfs/usr/share/sounds/sample/
   

   cp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_audio_playback/sample/* /media/user/rootfs/usr/share/sounds/sample/
   

   Notice

   Please make sure the /usr/share/movies/ exists on your SD card before using scp.
   If it does not exist, create it using the following command:

   mkdir -p /media/user/rootfs/usr/share/movies/
   

   cp <path-to>/rzg_hmi_sdk/sample_app/lvgl/lvgl_sample_video_playback/sample/* /media/user/rootfs/usr/share/movies/
   

Step 3: Run the HMI Sample Applications¶

1. Prepare the necessary equipment and configure the EVK DIP switches by following the instructions in Hardware Setup.

2. Insert the bootable microSD card created in Step 2 into the microSD card slot, and then power on the EVK board.

   Tip

   • Please refer to the EVK Peripheral Setup to locate the microSD card slot based on your selected boot mode.
   • Press and hold the power button (red button, SW9) for 1 second to turn on the EVK board, and for 2 seconds to turn it off.

3. Set execute permission and run the sample applications on your EVK board.
   
   

   • LVGL Sample Applications
   LVGL Sample Application for Image DisplayLVGL Sample Application for Audio PlaybackLVGL Sample Application for Video Playback

   Grant execute permission to the binary files.

   cd /home/root
   

   chmod +x lvgl_sample_img_disp
   

   Run the executable binary files.

   ./lvgl_sample_img_disp
   

   The launched sample application is shown below.
   This sample application is implemented using LVGL and can display image files in BMP, JPG, PNG, and GIF formats.

   

   Click the button to display the image from the source file.

   

   To exit, simply close the window or press Ctrl+C in the EVK board terminal.

   Grant execute permission to the binary files.

   cd /home/root
   

   chmod +x lvgl_sample_audio_playback
   

   Run the executable binary files.

   ./lvgl_sample_audio_playback
   

   The launched sample application is shown below.
   This sample application is implemented using LVGL and can play audio files in mp3, AAC, and WAV formats.

   

   Click the button to start, stop, or pause the audio from the source file.

   To exit, simply close the window or press Ctrl+C in the EVK board terminal.

   Grant execute permission to the binary files.

   cd /home/root
   

   chmod +x lvgl_sample_video_playback
   

   Run the executable binary files.

   ./lvgl_sample_video_playback -i /usr/share/movies/sample_video_with_audio.mp4
   

   The launched sample application is shown below.
   This sample application is implemented using LVGL, and it can play movie files in MP4 format.

   

   Click the button to start, stop, or pause the video from the source file.

   

   To exit, simply close the window or press Ctrl+C in the EVK board terminal.