Getting started with STM MPUs: STM32MP1 - OpenGL - DRM

STM32MP1 Discovery Kit (STM32MP157F-DK2) with 800x480 Display Module.

The following article explains all necessary steps to create an Embedded Wizard UI application for STM32MP1 using OpenGL ES 2.0 and DRM. The description was tested on the STM32MP157F-DK2 Discovery Kit (STM32MP157F-DK2) from STMicroelectronics, but can also be used analogously on other STM32MP1 boards.

Please follow these instructions carefully and step by step in order to ensure that you will get everything up and running on your target. In case you are not familiar with Embedded Wizard, please read first the chapter basic concepts and the Quick Tour tutorial to understand the principles of Embedded Wizard and the GUI development workflow. Moreover, this article assumes that you are familiar with Linux.

Prerequisites

First, make sure you have all of the following items:

Hardware components

Discovery Kit with STM32MP157F MPU (STM32MP157F-DK2) from STMicroelectronics or other board based on STM32MP1

Micro-USB cable to connect the board with your PC

2 USB Type-C cables

Network cable to connect the STM32MP1 board with your network

Micro SD card with 16GB

Software components

Embedded Wizard Studio Free or Embedded Wizard Studio Pro

If you want to use the Free edition of Embedded Wizard Studio please register on our website and download the software.

As a customer, please visit the Embedded Wizard Download Center (login/password required) and download Embedded Wizard Studio Pro.

Embedded Wizard Build Environment for STM32MP1 with OpenGL and DRM (STM32MP1-OpenGL-DRM)

To evaluate Embedded Wizard on the mentioned target, you can download the suitable Build Environment from the following link:

STM32MP1-OpenGL-DRM-BuildEnvironment-V12.00.01.00.zip

As a customer, please visit the Embedded Wizard Download Center (login/password required) and download the latest version of the Build Environment and your licensed Platform Package libraries or source codes.

STM32MP15x OpenSTLinux Starter Package (SD Card image) version 4.0.0

Yocto SDK version 4.0.0

IMPORTANT

In order to build (cross-compile) the GUI applications for the STM32MP1 target, a Linux PC or virtual machine is required. Please prepare an adequate Linux environment (e.g. Ubuntu within a VMware virtual machine).

Preparing Target and Linux Build System

A step by step install guide is available at the STM32 MPU Wiki. Alternatively, please follow this brief install guide:

Prepare and connect the STM32MP1 target (here: STM32MP157F-DK2 Discovery Kit):

Insert the SD card on the STM32MP1 target platform.

Connect your PC to the ST-LINK/V2-1 port CN11 (Micro-USB).

Connect a power supply (or your PC) to PWR IN (USB Type-C).

Connect your PC to the OTG port CN7 (USB Type-C).

Connect the STM32MP1 target to your network (Ethernet)

Set the microswitches SW1 to Off, Off

Press reset (DFU mode).

Download SD card linux image to STM32MP1 target using STM32_Programmer_CLI on Windows or Linux:

Download the STM32CubeProgrammer and install it.

Unpack the STM32MP15x OpenSTLinux Starter Package (SD Card image en.FLASH-stm32mp1-openstlinux-...).

Open a terminal and enter to folder /images/stm32mp1.

Download the SD card image suitable for your target using STM32_Programmer_CLI on Windows or Linux:

STM32_Programmer_CLI -c port=usb1 -w flashlayout_st-image-weston/trusted/FlashLayout_sdcard_stm32mp157f-dk2-trusted.tsv

Set the microswitch SW1 to On, On

Press reset.

As a result, the STM32MP1 target shout boot up using the Linux system from the SD card and automatically log in as root.

Determine the IP address of your target (e.g. by using ifconfig) and note it (e.g. 192.168.0.xxx).

Open a SSH console on your PC and connect with the target using its IP address.

Login as root without password.

Stop the currently running Weston demo:

systemctl stop weston-launch.service

Prepare your Ubuntu 64 bit Linux system (e.g. as virtual machine on your PC) as follows:

Update packages and install new packages:

sudo apt-get update sudo apt-get install sed wget curl cvs subversion git-core coreutils unzip texi2html texinfo docbook-utils gawk python-pysqlite2 diffstat help2man make gcc build-essential g++ desktop-file-utils chrpath libxml2-utils xmlto docbook bsdmainutils iputils-ping cpio python-wand python-pycryptopp python-crypto sudo apt-get install libsdl1.2-dev xterm corkscrew nfs-common nfs-kernel-server device-tree-compiler mercurial u-boot-tools libarchive-zip-perl sudo apt-get install ncurses-dev bc linux-headers-generic gcc-multilib libncurses5-dev libncursesw5-dev lrzsz dos2unix lib32ncurses5 repo libssl-dev sudo apt-get install default-jre

Allow up to 16 partitions per mmc:

echo 'options mmc_block perdev_minors=16' > /tmp/mmc_block.conf sudo mv /tmp/mmc_block.conf /etc/modprobe.d/mmc_block.conf

Install the SDK:

Unpack the Yocto_SDK:

mkdir -p $HOME/STM32MPU_workspace/tmp cd $HOME/STM32MPU_workspace/tmp tar xvf en.SDK-x86_64-stm32mp1-openstlinux-5.15-yocto-kirkstone-mp1-v22.06.15.tar.xz

Change the permissions of the SDK installation script so that it is executable:

chmod +x $HOME/STM32MPU_workspace/tmp/stm32mp1-openstlinux-5.15-yocto-kirkstone-mp1-v22.06.15/sdk/st-image-weston-openstlinux-weston-stm32mp1-x86_64-toolchain-4.0.1-openstlinux-5.15-yocto-kirkstone-mp1-v22.06.15.sh

Execute the SDK installation script:

$HOME/STM32MPU_workspace/tmp/stm32mp1-openstlinux-5.15-yocto-kirkstone-mp1-v22.06.15/sdk/st-image-weston-openstlinux-weston-stm32mp1-x86_64-toolchain-4.0.1-openstlinux-5.15-yocto-kirkstone-mp1-v22.06.15.sh

Source the sdk (to be done each time the build console is opened):

. /opt/st/stm32mp1/4.0.1-openstlinux-5.15-yocto-kirkstone-mp1-v22.06.15/environment-setup-cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi

Optionally check the target architecture, toolchain binary prefix, C compiler version, SDK version:

echo $ARCH echo $CROSS_COMPILE $CC --version echo $OECORE_SDK_VERSION

Installing Tools and Software

Step 1: Install the latest version of Embedded Wizard Studio on your Windows PC.

Step 2: Unpack the provided Embedded Wizard Build Environment for STM32MP1 with OpenGL and DRM (STM32MP1-OpenGL-DRM) into a new directory on your Windows PC (e.g. C:\STM32MP1).

Step 3: Share the directory of the Embedded Wizard Build Environment for STM32MP1 with your Linux system (e.g. /mnt/hgfs/STM32MP1).

Exploring the Build Environment

The provided Embedded Wizard Build Environment for STM32MP1 with OpenGL and DRM (STM32MP1-OpenGL-DRM) contains everything you need to create, compile and link an Embedded Wizard UI application for the STM32MP1 target. After unpacking, you will find the following subdirectories and files:

\Application - This folder contains a ready-to-use project to compile and link an Embedded Wizard generated UI application. It is used for all provided examples and it can be used to build your own UI applications.

\GeneratedCode - This folder is used to receive the generated code from an Embedded Wizard UI project. The template project is building the UI application out of this folder. You can create your own UI project and generate the code into the subdirectory \GeneratedCode without the need to adapt the project.

\Project - This folder contains the prepared project for GCC (make).

\Source - This folder contains the files main.c and ewmain.c. There you will find the initialization of the system and the main loop to drive an Embedded Wizard GUI application. The file ewconfig.h contains general configuration settings for the target system, like memory ranges and display parameter and configuration settings for the Embedded Wizard Graphics Engine and Runtime Environment.

\Examples\<ScreenSize> - These folders contain a set of demo applications prepared for a dedicated screen size of 480x272 pixel, 800x480 pixel and 1280x720 pixel. Each example is stored in a separate folder containing the entire Embedded Wizard UI project. Every project contains the necessary profile settings for the STM32MP1 target. Depending on the screen size, the following samples are provided:

\HelloWorld - A very simple project that is useful as starting point and to verify that the entire toolchain, your installation and your board is properly working.

\ScreenOrientation - This demo shows, that the orientation of the UI application is independent from the physical orientation of the display.

\DeviceIntegration - This example shows the integration of devices into a UI application and addresses typical questions: How to start a certain action on the target? How to get data from a device?

\GraphicsAccelerator - This application demonstrates the graphics performance of the target by using sets of basic drawing operations that are executed permanently and continuously.

\BezierClock - The sample application BezierClock implements a fancy digital clock and timer application with animated digits. The application uses vector graphics to render dynamically the different digits for clock and timer. The change from one digit to another is handled by moving the vector points to get a smooth transition animation.

\BrickGame - The sample application BrickGame implements a classic "paddle and ball" game. In the game, a couple of brick rows are arranged in the upper part of the screen. A ball travels across the screen, bouncing off the top and side walls of the screen. When a brick is hit, the ball bounces away and the brick is destroyed. The player has a movable paddle to bounce the ball upward, keeping it in play.

\ChartsDemo - This application demonstrates the possibilities and the usage of the chart classes LineChart, BarChart and PieChart. The appearance of each chart diagram can be configured easily by using their corresponding properties. Additionally, this demo shows the usage of an outline box to manage scrollable content.

\ClimateCabinet - The ClimateCabinet demo shows the implementation of a control panel for a climatic exposure test cabinet. The user can define a heating time, a nominal temperature and humidity, a dwell time and the final cooling time.

\Dashboard - The sample application Dashboard shows the implementation of a car dashboard. The application demonstrates the usage of vector graphics created from svg path data and how transitions between two dashboard variants and further UI components can be achieved.

\PaperCutter - This demo shows the implementation of a paper cutting machine, where the user can define the pagination and format of the paper as well as the cutting speed and the amount of papers. The application contains many rect effects and fade-in/fade-out effects to change dynamically the layout of the screen.

\PulseOximeter - The sample application PulseOximeter shows the implementation of a medical device for monitoring a person's pulse frequency and peripheral oxygen saturation. The application demonstrates the usage of vector graphics within graphs and circular gauges.

\SmartThermostat - The SmartThermostat demo application shows the implementation of a fancy, rotatable temperature controller to adjust and display the nominal and actual temperature.

\WashingMachine - This demo shows the implementation of a washing machine with a couple of fancy scrollable list widgets to choose the washing program and parameters. The speciality of this sample application is the magnification effect of the centered list items and the soft fade-in/fade-out effects.

\WaveformGenerator - This WaveformGenerator demo application combines waveforms with different amplitudes and frequencies. The implementation shows the usage of vector graphics to draw a curve based on a list of coordinates.

\MasterDemo - This folder contains the binary file of the Embedded Wizard Master Demo. It combines a variety of examples within one huge demo application. It can be used for presentations and showcases.

\PlatformPackage - This folder contains the necessary source codes and/or libraries of the Linux OpenGL Platform Package: The Graphics Engine (in the subdirectory \RGBA8888) and the Runtime Environment (in the subdirectory \RTE).

\TargetSpecific - This folder contains all configuration files and platform specific source codes. The different ew_bsp_xxx files implement the bridge between the Embedded Wizard UI application and the underlying Linux drivers in order to access the display, the serial interface and the touch driver.

Creating the UI Examples

For the first bring up of your system, we recommend to use the example 'HelloWorld':

Example 'HelloWorld' within Embedded Wizard Studio.

The following steps are necessary to generate the source code of this sample application:

On your PC, navigate to the directory \Examples\<ScreenSize>\HelloWorld.

Open the project file HelloWorld.ewp with your previously installed Embedded Wizard Studio. The entire project is well documented inline. You can run the UI application within the Prototyper by pressing Ctrl+F5.

To start the code generator, select the menu items BuildBuild this profile - or simply press F8. Embedded Wizard Studio generates now the sources files of the example project into the directory \Application\GeneratedCode.

Building the Binary

The following steps are necessary to build the Embedded Wizard UI sample application using the cross-compiler on your Linux build system:

Open a terminal window and source the cross-compiler:

. /opt/st/stm32mp1/4.0.1-openstlinux-5.15-yocto-kirkstone-mp1-v22.06.15/environment-setup-cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi

Change into the subfolder /Application/Project of the shared directory of the Embedded Wizard Build Environment.

cd /mnt/hgfs/STM32MP1/Application/Project

Compile and link the example application:

make

The resulting binary of the application EmbeddedWizard-Linux-OpenGL-DRM will be stored in the current directory.

Downloading the Binary

Start your STM32MP target (as described above), determine the IP address of your target and open a SSH connection.

Login as root without a password.

To run an application based on DRM, the Wayland/Weston service has to be stopped:

systemctl stop weston-launch.service

Open the file /Application/Project/target_ip.txt with a text editor and insert the determined IP address (see above).

Copy the application binary to the target and start it:

make install

Alternatively, the application can be downloaded and started manually:

scp EmbeddedWizard-Linux-OpenGL-DRM root@<target-ip-address>:/usr/local

Open a SSH connection, log in as root without a password and start the application:

cd /usr/local/ ./EmbeddedWizard-Linux-OpenGL-DRM

Example 'HelloWorld' running on STM32MP1 target.

All other examples can be created in the same way: Just open the desired example with Embedded Wizard Studio, generate code, rebuild the application and download it to the target.

Creating your own UI Applications

In order to create your own UI project suitable for the STM32MP1 target, you can create a new project and select the STM32MP1 OpenGL project template:

As a result you get a new Embedded Wizard project, that contains the necessary Profile attributes suitable for the STM32MP1 target:

The following profile settings are important for your target:

The attribute PlatformPackage should refer to the Linux OpenGL Platform Package.

The attribute ScreenSize should correspond to the display size of the STM32MP1 target.

The attribute OutputDirectory should refer to the \Application\GeneratedCode directory within your Build Environment. By using this template, it will be very easy to build the UI project for your target.

The attribute CleanOutputDirectories should be set to true to ensure that unused source code within the output directory \Application\GeneratedCode will be deleted.

Now you can use the template project in the same manner as it was used for the provided examples to compile and link the binary.

After generating code, just build and execute your GUI application in the same way as described above.

Project settings related to the GUI application are taken directly out of the generated code - all other settings can be made directly within the file ewconfig.h, which contains general configuration settings for the target system.

Console output

The SSH client is very helpful to receive error messages or to display simple debug or trace messages from your Embedded Wizard UI application.

It can be used for all trace statements from your Embedded Wizard UI applications or for all debug messages from your C code.

Embedded Wizard GUI Demos

If you just want to run our GUI demos on your STM32MP1 target without building the different examples, you can simply start the binary file of the Embedded Wizard Master Demo.

The Embedded Wizard Master Demo combines a variety of examples within one huge demo application. It can be used for presentations and showcases. Each demo application can be activated from a common main menu. To return back from a demo application to the main menu, a small home button appears on top of every demo. Additionally, the Master Demo contains an auto-demo mode that presents one demo application after the other. The auto-demo starts automatically and stops as soon as the user touches the screen.

In order to copy the binary file to your target, please follow these steps:

Change into the subfolder /MasterDemo of the shared directory of the Embedded Wizard Build Environment.

Copy the binary to the target:

scp EmbeddedWizard-Linux-OpenGL-DRM root@<target-ip-address>:/usr/local

Start the binary via SSH connection:

cd /usr/local/ ./EmbeddedWizard-Linux-OpenGL-DRM

This is the resulting application on an STM32MP1 target:

'Master Demo' running on STM32MP1 target.

Display and target configuration

The file /Application/Source/ewconfig.h provides a variety of configuration options that can be adapted to your needs, like display size, autoscaling, viewport rotation (0, 90, 180, 270 degrees), touch calibration or memory configuration and footprint optimization.

For more details please see the inline documentation within the file ewconfig.h.

Release notes

The following section contains the version history of the Build Environment (including Graphics Engine and Runtime Environment) for STM32MP1 with OpenGL and DRM (STM32MP1-OpenGL-DRM). These release notes describe only the platform specific aspects - for all general improvements and enhancements please see the Embedded Wizard release notes.

Version 12.00.00.00

Enhanced system configuration: The set of macros used to configure the target system has been enhanced within the file ewconfig.h. All macros are now described in the article target configuration.

Operating system wrapper (ew_bsp_os.c): All operating system services used within the Build Environment are now grouped together and implemented within the file ew_bsp_os.c.

Multi-Thread, Multi-Task check functionality: All relevant top-level Runtime Environment and Graphics Engine functions perform now a test whether the function is called in context of the original GUI thread/task. For this purpose the test compares the current CPU stack address with the address area known as used for GUI thread/task stack. The stack memory area is provided to the Embedded Wizard application within the file main.c by using the function EwSetStackAddressArea(). The size of the stack area is defined by the macro EW_GUI_THREAD_STACK_SIZE in the file ewconfig.h. The test functionality can be disabled by the macro EW_DONT_CHECK_THREAD_CONTEXT in the file ewconfig.h.

Communication between threads and tasks: The Runtime Environment has been enhanced by new functions EwInvoke() and EwInvokeCopy(). These functions register user's own implemented C functions to be called in context of GUI thread/task. These functions are explicitly allowed to be called from any foreign thread or even from interrupt service routines. The new macro EW_INVOCATION_QUEUE_SIZE in the file ewconfig.h determines the size of the invocation queue in bytes. The initialization of the invocation queue is done in EwInit() by calling the function EwInitInvocations(). The deinitialization of the invocation queue by using EwDoneInvocations() can be found in EwDone(). The main loop EwProcess() that drives the entire Embedded Wizard GUI application is enhanced by the new function EwProcessInvocations().

System initialization: The configuration of Runtime Environment and Graphics Engine is now done in EwInit() by using the new functions EwConfigRuntimeEnvironment() and EwConfigGraphicsEngine().

System information: The function EwPrintSystemInfo() is now determining the current configuration of Runtime Environment and Graphics Engine by using the new functions EwGetRuntimeEnvironmentInfo() and EwGetGraphicsEngineInfo(). The version numbers are printed according to the new version numbering scheme. The layout of the entire message printed to the console is rearranged.

Immediate garbage collection: The test function EwTestImmediateReclaimMemory() performs a simple test for the correct function of the immediate garbage collection. If the application is compiled with the macro EW_USE_IMMEDIATE_GARBAGE_COLLECTION and the the stack area is specified by using EwSetStackAddressArea(), the function EwTestImmediateReclaimMemory() verifies the correct operation. This is added to EwInit() in the file ewmain.c.

Touch initialization: The initialization of the touch driver is improved to avoid potential division by zero errors during runtime in case that touch initialization has failed.

Color channel order: The default color channel order in compressed bitmap resources has changed from ABGR to ARGB. This avoids an additional color channel reordering during the loading (decompression) of the bitmap resources.

Device integration example: The example DeviceIntegration and the DeviceDriver.c/h files have to be adapted to show the usage of the new features: A separate worker-thread is created to provide simulated ADC data to the GUI. The functions EwInvoke() and EwInvokeCopy() are used from the work-thread.

Additional examples with other screen sizes added to the \Examples directory.

Changed to Graphics Engine (GFX) and Runtime Environment (RTE) V12.00.

Changed to STM32MP15x OpenSTLinux Starter Package 4.0.0.

Changed to Yocto SDK 4.0.0.

Known issue: The start of a GUI application will take about 4s on the target, due to long compilation time for the OpenGL shader programs. This will be optimized in future versions.

Version 12.00.01.00

Application launch time improvement: In the previous release, launching a GUI application on the target took up to 4 seconds due to the long compile time for the OpenGL shader programs. The problem is now solved.

Improved display initialization with more flexibility: The display interface (e.g. DSI, HDMI) and the display mode settings (e.g. resolution, frame rate) within the file ewconfig.h are now used as preferences. If theses settings cannot be fulfilled on the specific platform, one of the available interfaces and display modes are used, that fulfill the preferred settings as good as possible.

Changed to STM32MP15x OpenSTLinux Starter Package 4.1.0.

Changed to Yocto SDK 4.1.0.