“Visualizing information can give us a very quick solution to problems. We can get clarity or the answer to a simple problem very quickly.”

David McCandless

Working with VLGL, the little graphics library for embedded systems, is an interesting process. Sometimes you want to capture what you see in order to use it for diagnosis, tests or documentation.

For example, here is a screen capture from the T-Embed:

LVGL provides a nice little built in to help us with this, lv_snapshot_take. The issue then becomes getting the resulting data off the device and onto the host. We can do that with a combination of the ESP-IDF component console which allows us to add an extensible REPL to our ESP32 firmware.

First, however, here is some code to grab the screen and output it to the serial connection as a stream of 2-byte (16 bit – the color depth the display uses) hexadecimal numbers.

static int snapshot(int argc, char **argv) {
    LOCK_GUI;
    lv_img_dsc_t *snap=lv_snapshot_take(lv_scr_act(), LV_IMG_CF_TRUE_COLOR);
    UNLOCK_GUI;

    ESP_LOGI(TAG, "Snapshot %dx%d", snap->header.w, snap->header.h);
    // Output image
    for(int x=0;x<snap->header.w;x++) {
        for(int y=0;y<snap->header.h;y++) {
            lv_color_t col=lv_img_buf_get_px_color(snap, x, y, lv_color_black());
            printf("%04x ",col.full);
        }
    }
    printf("\n");

    lv_snapshot_free(snap);

    return 0;
}

To use this with the ESP-IDF console, we need to initialize the console according to the example, then register an additional command

static void register_cmd_snapshot(void)
{
    const esp_console_cmd_t cmd = {
        .command = "snap",
        .help = "Take a snapshot of the screen",
        .hint = NULL,
        .func = &snapshot,
    };
    ESP_ERROR_CHECK( esp_console_cmd_register(&cmd) );
}

Now, once the firmware is flashed we can use idf.py monitor | tee snap.log to capture a log file of the serial port. Once the ESP is up and running, entering snap on the console triggers the new snapshot command and results in a single (very long!) line of hex being dumped to the monitor, and to the log file. After capturing the hex data in the log, extract the single line by deleting the rest of the log and we can turn that back into a block of binary data using the xxd -r -ps <snap.log >snap.raw command. This results in a raw format graphics file – just graphics data and no metadata. It’s more friendly to convert it to a PNG, which we can do as follows :

#!/usr/bin/python3
import argparse
import os
from PIL import Image, ImageOps
import struct
      
width = 170
height = 320
png = Image.new('RGB', (width, height))

with open('snap.raw', 'rb') as input_file:
   src = input_file.read()
stream = struct.unpack('<' + str(len(src) // 2) + "H", src)

for i, word in enumerate(stream):
    r = (word >> 11) & 0x1F
    g = (word >> 5) & 0x3F
    b = (word) & 0x1F
    png.putpixel((i % width, i // width), (r << 3, g << 2, b << 3))

png_r = png.rotate(270, expand=True)
png_m = ImageOps.mirror(png_r)
png_m.save('snap.png')

As this is a simple demo, I have hardcoded the image size and filenames. It is not, however, difficult to use python argparse to make them parameters.

This simple technique can be easily expanded upon to use as part of a testsuite.