Platform Integration Aspects: Framebuffer Concepts

This article describes the different framebuffer concepts and display integration scenarios, that are typically provided within our Build Environments and that are used within many products based on Embedded Wizard generated GUI applications.

First of all, let's clarify some wording - this helps to get a common understanding:

Framebuffer - This is a pixel buffer, containing the current graphical user interface that should be presented on the display. The content of the framebuffer is prepared by the Graphics Engine using software rendering (via CPU) or by using a graphics accelerator (GA). A framebuffer is a continuous memory area, typically located within the address space of the CPU. In case of complex graphics systems (e.g. using OpenGL ES 2.0) a framebuffer is located within a separated video memory.

Double-buffering - A technique where the framebuffer consist of two independent pixel buffers. The content of the currently non-visible buffer is prepared by the graphics software (or hardware), while the other buffer is currently visible. Once the content of the non-visible buffer is complete, the two buffers are exchanged (typically, just by exchanging the address of the buffer). This swapping of the buffers happens typically V-synchronous, so that no build-up effects are noticeable.

Frontbuffer - This is the currently visible framebuffer in case of a double-buffered framebuffer.

Backbuffer - This is the non-visible framebuffer in case of a double-buffered framebuffer.

Off-screen Buffer - This is an internal buffer, allocated by Embedded Wizard and used to prepare the graphical content (composition buffer). The result is then copied into a framebuffer

Scratch-pad Buffer - This is small framebuffer used for partial display updates on extremely memory constraint systems.

Display Controller (DC) - Hardware that drives the display and refreshes the content of the display continuously by reading the content of the graphics memory.

Display Interface (DI) - On-chip interface (e.g. SPI, 8/16-bit parallel, FMC) used to connect an external display controller in order to transfer the graphics content of the framebuffer.

The following integrations can be done - each with it's own advantages and disadvantages. Please note, that each Platform Package and Build Environment only supports a subset of these concepts - depending on the system architecture.

One Framebuffer (Single-buffer / Synchronous Single-buffer)

In principle it is sufficient to use just the visible framebuffer and to let Embedded Wizard prepare the graphical output directly within the visible framebuffer. This is the most efficient approach, because it only requires the memory for one buffer and there is no additional copy operation involved.

However, due to the fact, that Embedded Wizard will draw all graphical objects from back to top (according to their z-order), the complete graphics composition can be seen on the display! As a result a lot of flickering and build-up effects will be seen. So this is not really a good solution for most LCDs that read the screen content directly from the framebuffer.

In order to avoid this drawback, it is possible to synchronize the display composition with the current read position of the display controller. This means, the entire screen update is divided into horizontal stripes, that are redrawn while they are not accessed by the display controller. As long as the drawing operations are fast enough, the update happens without any flickering or tearing effects.

Double-buffered Framebuffer

In this solution, the framebuffer is used in real double-buffering mode. The graphics composition of Embedded Wizard will be done directly within the backbuffer. Once the update is complete, the two buffers (front and back) can be exchanged by hardware. This so called 'swapping' or 'flipping' of the two buffers has to be done on V-sync. This is a very popular solution - especially for games, where always the complete scene (= the complete screen) has to be redrawn. In order to avoid that Embedded Wizard has to redraw always the whole screen, the Mosaic class library is able to reduce the drawing operations to the affected areas of the current and the previous update.

The advantage of this solution is that the screen update is V-synchronous by using two buffers, which avoids all flicker or tearing effects. However, the disadvantage is, that the graphics composition has to be done within a larger area and that the V-synchronous update causes additional overhead and waiting time.

One Framebuffer and one Off-screen Buffer

In this solution, Embedded Wizard make the graphics composition within it's own off-screen buffer. Each time the content of the off-screen buffer is prepared, the visible framebuffer is updated. This means that the content of the off-screen buffer is copied into the framebuffer. Here it is important to know, that Embedded Wizard minimize the area that has to be drawn and takes care, that only the affected area (so called 'dirty area') is copied from the off-screen buffer into the framebuffer.

Overall, this solution is a good compromise - it requires only 2 buffers and the area that has to be transferred is minimized. The disadvantage of this solution is, that the update (copy operation) is not V-synchronous, which may lead to some jitter or tearing effects within animations.

This scenario is very powerful in case that the read access to the framebuffer is very slow - which is typically the case when framebuffer is configured as shared memory under Embedded Linux. In this case the framebuffer is only accessed with write operations from CPU or GA.

In case that the copy operation into the framebuffer is hardware accelerated and the graphics accelerator is able to make a color conversion, it is possible to use a lower color format for the framebuffer, which can save some memory.

Double-buffered Framebuffer and one Off-screen Buffer

In this solution, Embedded Wizard can make the graphics composition within its own off-screen buffer. Each time the content is prepared, it will be copied into the backbuffer. The swapping of front and backbuffer has to be done V-synchronous.

This solution requires the most memory (3 buffers), but it combines the advantages and use-cases of all above concepts.

One Framebuffer and an external Display Controller with Memory (GRAM)

This is the typical layout, when an external display controller with its own graphics memory (GRAM) is used. In this case, the content of the framebuffer is transferred to the graphics memory after the update of the GUI is completed.

In order to avoid tearing effects due to the update of the GRAM, the display controller (DC) has to provide line events that can be used to synchronize the transfer via the display interface (DI).

This solution is typically used in combination with MIPI-DSI displays.

Double-buffered Framebuffer and an external Display Controller with Memory (GRAM)

This solution is the improved version of the above scenario. The additional double-buffering makes it possible to prepare the next frame by CPU/GA within the backbuffer while the content of the framebuffer is transferred to the external display controller.

Scratch-pad buffer and an external Display Controller with Memory (GRAM)

This solution is intended to be used on extremely memory constraint MCUs without external SDRAM. In this solution only a small piece of memory ("scratch-pad") is used to prepare a rectangular area of the GUI. As soon as the area is drawn, it can be transferred to the external display controller with its own graphics memory (GRAM). As soon as the transfer is completed, the next area can be prepared. The number of iterations for a full-screen update depends on the size of the scratch-pad memory. The smaller the scratch-pad buffer, the more partial updates are needed and the more noticeable is the result.

This solution is great for systems that do not have enough memory for at least one framebuffer, but it is not the best choice when the GUI application contains a lot of animations and slide-effects.

Double-buffered Scratch-pad buffer and an external Display Controller with Memory (GRAM)

This is a combination of the scratch-pad buffer solution and double-buffering. Two small memory areas are used as scratch-pads, one as frontbuffer and one as backbuffer. The advantage of this solution is, that the transfer of a prepared scratch-pad (current frontbuffer) to the external display can be done in parallel while the next rectangular area of the GUI is prepared (current backbuffer). This solution is intended to be used on memory constraint systems which is able to update the display in parallel, e.g. by using a DMA.