Platform Integration Aspects: Integrating Video Content

Modern GUI applications increasingly require the ability to display video content - whether for demonstrating product features, playing instructional clips, showing live camera feeds, or streaming protected media. Unlike static bitmap resources, video content is dynamic by nature: it is decoded frame by frame at runtime, potentially from a file system, a network stream, or directly from a camera peripheral.

The Mosaic framework provides two complementary approaches for integrating video into a GUI application. Both rely on a platform-specific implementation that you provide in C. Which approach is right for your project depends on the capabilities of your target platform and the requirements of your application.

The two approaches for video integration in Embedded Wizard: ExternVideo delivers decoded frames as bitmaps into the GUI layer, while the VideoPlayer Applet renders video directly into a separate platform-managed surface or video layer.

The two approaches

ExternVideo Interface

The ExternVideo Interface approach integrates video directly into the GUI surface. Each decoded frame is delivered to Embedded Wizard as an XBitmap*, which the GUI engine renders like any other bitmap resource. This enables full GUI integration: the video can be scaled, rotated, warped, and overlaid with any GUI elements. It is the preferred approach whenever the platform supports it.

You implement a fixed set of C functions declared in EwExternVideo.h. The implementation is completely free in its choice of decoder — whether a software library, a hardware peripheral, or a multimedia framework like GStreamer.

VideoPlayer Applet

The VideoPlayer Applet approach renders video into a platform-managed surface that is composited independently of the Embedded Wizard GUI layer — for example a Wayland SubSurface. Embedded Wizard controls the position and size of the video area, but the video content itself never passes through the Embedded Wizard bitmap pipeline. This approach is suitable for specific use cases where ExternVideo is not applicable or not efficient enough.

Choosing the right approach

The following table compares both approaches across the most relevant aspects to help with the decision.

Aspect	ExternVideo	VideoPlayer Applet
Zoom, rotation, warping	Supported	Not supported
GUI elements overlaid on video	Supported	Supported if GUI layer is placed above the video layer
Separate hardware video layer	Not applicable	Supported
Secure Video (DRM)	Not applicable	Supported
YUV to RGB color conversion	Must be handled by decoder or graphics system — performance is platform dependent	Can be handled by the display controller during compositing — potentially more efficient
Live video / camera feed	Suitable if color conversion is hardware-accelerated	Suitable when CPU-based color conversion is not acceptable

Color conversion and platform performance

Many video decoders and most camera peripherals deliver frames in a YUV-based color format. Integrating such frames into an RGB-based GUI surface requires a color space conversion from YUV to RGB. Whether this conversion can be performed efficiently depends strongly on the platform:

•On platforms with hardware-accelerated color conversion (e.g. a GPU performing the conversion during OpenGL texture upload, or a dedicated image processing unit), ExternVideo works well even with YUV sources.

•On platforms where color conversion must be done by the CPU, the cost can be significant — especially for high resolutions or high frame rates. For continuous live video from a camera, this cost is incurred for every single frame with no opportunity to amortize it.

•The VideoPlayer Applet sidesteps this problem entirely: the display controller composites the YUV video surface directly onto the screen without any CPU involvement in color conversion.

If you are unsure which applies to your platform, start with ExternVideo. If profiling reveals that color conversion is a bottleneck, consider switching to the VideoPlayer Applet.