Arduino and AnimatedGIF on Adafruit Qualia ESP32-S3 for TTL RGB-666 Display 2

Support Adafruit
1

[This same post was also made in the Arduino section because it is relevant to both: Arduino and AnimatedGIF on Adafruit Qualia ESP32-S3 for TTL RGB-666 Display ]

Hello. I have the following items:

  • Adafruit Qualia ESP32-S3 for TTL RGB-666 Displays
  • Round RGB 666 TTL TFT Display - 2.8" 480x480 - No Touch - TL028WVC01-B1621A

Using Arduino IDE, esp32 20.0.17 (by Espressif Systems) for the board manager, AnimatedGIF 2.2.0 (by Larry Bank), and GFX Library for Arduino 1.4.0 (by Moon on our nation), I uploaded the following code:

#include <Arduino_GFX_Library.h>
#include <AnimatedGIF.h>
#include "portal_image.h"

#define DISPLAY_WIDTH     480
#define DISPLAY_HEIGHT    480


Arduino_XCA9554SWSPI *expander = new Arduino_XCA9554SWSPI(
    PCA_TFT_RESET, PCA_TFT_CS, PCA_TFT_SCK, PCA_TFT_MOSI,
    &Wire, 0x3F); //I2C expander used for BACKLIGHT LED CONTROL

Arduino_ESP32RGBPanel *rgbpanel = new Arduino_ESP32RGBPanel(
    TFT_DE, TFT_VSYNC, TFT_HSYNC, TFT_PCLK,
    TFT_R1, TFT_R2, TFT_R3, TFT_R4, TFT_R5,
    TFT_G0, TFT_G1, TFT_G2, TFT_G3, TFT_G4, TFT_G5,
    TFT_B1, TFT_B2, TFT_B3, TFT_B4, TFT_B5,
    1 /* hync_polarity */, 46 /* hsync_front_porch */, 2 /* hsync_pulse_width */, 44 /* hsync_back_porch */,
    1 /* vsync_polarity */, 50 /* vsync_front_porch */, 16 /* vsync_pulse_width */, 16 /* vsync_back_porch */);

Arduino_RGB_Display *gfx = new Arduino_RGB_Display(
// 2.8" round
    DISPLAY_WIDTH /* width */, DISPLAY_HEIGHT /* height */, rgbpanel, 0 /* rotation */, true /* auto_flush */,
    expander, GFX_NOT_DEFINED /* RST */, TL028WVC01_init_operations, sizeof(TL028WVC01_init_operations));


AnimatedGIF gif;
int x_offset = 0;
int y_offset = 0;

// Draw a line of image directly on the TFT
void GIFDraw(GIFDRAW *pDraw){
    uint8_t *s;
    uint16_t *d, *usPalette, usTemp[320];
    int x, y, iWidth;

    iWidth = pDraw->iWidth;
    if (iWidth + pDraw->iX > DISPLAY_WIDTH)
       iWidth = DISPLAY_WIDTH - pDraw->iX;
    usPalette = pDraw->pPalette;
    y = pDraw->iY + pDraw->y; // current line
    if (y >= DISPLAY_HEIGHT || pDraw->iX >= DISPLAY_WIDTH || iWidth < 1)
       return; 
    s = pDraw->pPixels;
    if (pDraw->ucDisposalMethod == 2) // restore to background color
    {
      for (x=0; x<iWidth; x++)
      {
        if (s[x] == pDraw->ucTransparent)
           s[x] = pDraw->ucBackground;
      }
      pDraw->ucHasTransparency = 0;
    }

    // Apply the new pixels to the main image
    if (pDraw->ucHasTransparency) // if transparency used
    {
      uint8_t *pEnd, c, ucTransparent = pDraw->ucTransparent;
      int x, iCount;
      pEnd = s + iWidth;
      x = 0;
      iCount = 0; // count non-transparent pixels
      while(x < iWidth)
      {
        c = ucTransparent-1;
        d = usTemp;
        while (c != ucTransparent && s < pEnd)
        {
          c = *s++;
          if (c == ucTransparent) // done, stop
          {
            s--; // back up to treat it like transparent
          }
          else // opaque
          {
             *d++ = usPalette[c];
             iCount++;
          }
        } // while looking for opaque pixels
        if (iCount) // any opaque pixels?
        {
          gfx->draw16bitRGBBitmap(pDraw->iX+x+x_offset, y+y_offset, usTemp, iCount, 1);          
          x += iCount;
          iCount = 0;
        }
        // no, look for a run of transparent pixels
        c = ucTransparent;
        while (c == ucTransparent && s < pEnd)
        {
          c = *s++;
          if (c == ucTransparent)
             iCount++;
          else
             s--; 
        }
        if (iCount)
        {
          x += iCount; // skip these
          iCount = 0;
        }
      }
    }
    else
    {
      s = pDraw->pPixels;
      // Translate the 8-bit pixels through the RGB565 palette (already byte reversed)
      for (x=0; x<iWidth; x++)
        usTemp[x] = usPalette[*s++];

      gfx->draw16bitRGBBitmap(pDraw->iX+x_offset, y+y_offset, usTemp, iWidth, 1);
    }
} /* GIFDraw() */

void setup(void)
{
  Serial.begin(115200);

#ifdef GFX_EXTRA_PRE_INIT
  GFX_EXTRA_PRE_INIT();
#endif

  Serial.println("Starting RGB screen");

  // Init Display
  Wire.setClock(400000); // speed up I2C for expander
  if (!gfx->begin()) {
    Serial.println("gfx->begin() failed!!!");
    while (1) yield();
  }

  gfx->fillScreen(BLACK);

  expander->pinMode(PCA_TFT_BACKLIGHT, OUTPUT);
  expander->digitalWrite(PCA_TFT_BACKLIGHT, HIGH); //turn on backlight led


  //TEST CODE TO MAKE SURE DISPLAY IS OK BEGINS
  Serial.println("Hello!");
  gfx->fillScreen(BLACK);
  gfx->setCursor(100, gfx->height() / 2 - 75);
  gfx->setTextSize(5);
  gfx->setTextColor(WHITE);
  gfx->println("Hello World!");

  Serial.println("Hello in RED!");
  gfx->setCursor(100, gfx->height() / 2 - 25);
  gfx->setTextColor(RED);
  gfx->println("RED");

  Serial.println("Hello in GREEEN!");
  gfx->setCursor(100, gfx->height() / 2 + 25);
  gfx->setTextColor(GREEN);
  gfx->println("GREEN");

  Serial.println("Hello in BL;UE!");
  gfx->setCursor(100, gfx->height() / 2 + 75);
  gfx->setTextColor(BLUE);
  gfx->println("BLUE");
  delay(1000);
  gfx->fillScreen(BLACK);
  //TEST CODE TO MAKE SURE DISPLAY IS OK ENDS

}


void loop(){


  // put your main code here, to run repeatedly:
  if (gif.open((uint8_t *)gif_data, sizeof(gif_data), GIFDraw)){
    Serial.printf("Successfully opened GIF; Canvas size = %d x %d\n", gif.getCanvasWidth(), gif.getCanvasHeight());
    if( gif.getCanvasWidth() < DISPLAY_WIDTH || gif.getCanvasHeight() < DISPLAY_HEIGHT){
      x_offset = (DISPLAY_WIDTH - gif.getCanvasWidth()) / 2;
      y_offset = (DISPLAY_HEIGHT - gif.getCanvasHeight())/ 2;
    }

    /*
    while (gif.playFrame(true, NULL))
    {      
    }
    gif.close();*/

          Serial.println("Starting GIF playback...");
        while (gif.playFrame(true, NULL)) {
            Serial.println("Playing frame...");
            delay(10); // Optional delay to control frame rate
        }
        Serial.println("Finished GIF playback.");
        gif.close();
    } else {
        Serial.println("Failed to open GIF.");
    }

}

The ā€˜portal_image.hā€™ is an animated GIF 480x480 pixel that has already been converted to binary.
The current code prints the following information to the Serial Monitor:

Hello!

Hello in RED!

Hello in GREEEN!

Hello in BL;UE!

Successfully opened GIF; Canvas size = 480 x 480

Starting GIF playback...

And finally for what it actually displays on the screen, it successfully prints out Hello World and a number of other coloured words. Then shortly after, the device crashes and reboots when attempting to load the GIF.

Please note: I have already attempted to ask this question on the official Adafruit and Arduino forums months ago and they were thoroughly unhelpful. The code youā€™re seeing above is actually provided to me by a programmar on Fiverr who is currently trying to solve the issue. Every online tutorial Iā€™ve found online hasnā€™t been very helpful. The animated GIF in question, as far as Iā€™m aware from the uploading output, successfully transferred to the device.

Here is the output:

Sketch uses 959245 bytes (45%) of program storage space. Maximum is 2097152 bytes.
Global variables use 54756 bytes (16%) of dynamic memory, leaving 272924 bytes for local variables. Maximum is 327680 bytes.
esptool.py v4.5.1
Serial port COM7
Connecting...
Chip is ESP32-S3 (revision v0.2)
Features: WiFi, BLE
Crystal is 40MHz
MAC: 74:4d:bd:9d:41:84
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Erasing flash (this may take a while)...
Chip erase completed successfully in 3.4s
Compressed 21696 bytes to 13543...
Writing at 0x00000000... (100 %)
Wrote 21696 bytes (13543 compressed) at 0x00000000 in 0.3 seconds (effective 510.5 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 136...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (136 compressed) at 0x00008000 in 0.1 seconds (effective 224.2 kbit/s)...
Hash of data verified.
Compressed 8192 bytes to 47...
Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.1 seconds (effective 600.2 kbit/s)...
Hash of data verified.
Compressed 959616 bytes to 811640...
Writing at 0x00010000... (2 %)
Writing at 0x0001409f... (4 %)
Writing at 0x00018099... (6 %)
Writing at 0x0001c094... (8 %)
Writing at 0x0002008f... (10 %)
Writing at 0x0002408a... (12 %)
Writing at 0x00028085... (14 %)
Writing at 0x0002c080... (16 %)
Writing at 0x0003007b... (18 %)
Writing at 0x00034076... (20 %)
Writing at 0x00038071... (22 %)
Writing at 0x0003c06c... (24 %)
Writing at 0x00040067... (26 %)
Writing at 0x00044062... (28 %)
Writing at 0x0004805d... (30 %)
Writing at 0x0004c058... (32 %)
Writing at 0x00050053... (34 %)
Writing at 0x0005404e... (36 %)
Writing at 0x00058049... (38 %)
Writing at 0x0005c044... (40 %)
Writing at 0x0006003f... (42 %)
Writing at 0x0006403a... (44 %)
Writing at 0x00068035... (46 %)
Writing at 0x0006c030... (48 %)
Writing at 0x0007002b... (50 %)
Writing at 0x00074026... (52 %)
Writing at 0x00078021... (54 %)
Writing at 0x0007c01c... (56 %)
Writing at 0x00080017... (58 %)
Writing at 0x00084012... (60 %)
Writing at 0x0008800d... (62 %)
Writing at 0x0008c008... (64 %)
Writing at 0x00090003... (66 %)
Writing at 0x00093ffe... (68 %)
Writing at 0x00097ff9... (70 %)
Writing at 0x0009bff4... (72 %)
Writing at 0x0009ffef... (74 %)
Writing at 0x000a3fea... (76 %)
Writing at 0x000a8b3b... (78 %)
Writing at 0x000b795e... (80 %)
Writing at 0x000c0da4... (82 %)
Writing at 0x000c6080... (84 %)
Writing at 0x000cb943... (86 %)
Writing at 0x000d0ff8... (88 %)
Writing at 0x000d6437... (90 %)
Writing at 0x000db6d0... (92 %)
Writing at 0x000e1a4a... (94 %)
Writing at 0x000ebf43... (96 %)
Writing at 0x000f180a... (98 %)
Writing at 0x000f734e... (100 %)
Wrote 959616 bytes (811640 compressed) at 0x00010000 in 9.6 seconds (effective 802.2 kbit/s)...
Hash of data verified.
Compressed 157728 bytes to 101221...
Writing at 0x00410000... (14 %)
Writing at 0x00417b41... (28 %)
Writing at 0x0041d0ff... (42 %)
Writing at 0x0042404f... (57 %)
Writing at 0x0042ae12... (71 %)
Writing at 0x00430367... (85 %)
Writing at 0x004354e5... (100 %)
Wrote 157728 bytes (101221 compressed) at 0x00410000 in 1.3 seconds (effective 943.8 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

What Iā€™m after is:
-Advice on why it is crashing when attempting to load the GIF.
-What should I do to fix this?

2

Im more used to the ESP32 idf and its debug output. It would normally show the exact reason why it rebooted.

So looking your your serial output, that seems to be your output and no debug. Is there an option to compile with full debug ?

Things I would be looking for that would case a boot loop would be around stack overflows, memory issues and watchdog timers.

Since these are not your code it may take some work to track down the base cause.

The ESP32 has limited ram, so the graphic library may fail to reserve/mem_alloc memory for a frame buffer, but then assume it was allocated, that would cause a crash.

leaving 272924 bytes for local variables. Maximum is 327680

I assume the 666 is 6 bits for RGB this 12 Bits per pixel.
So based on your 480x480, ā€œifā€ it trys to run a full frame buffer, that would be 480x480x12/8 = 345,600 which looks bigger then the est free memory.

Not if may not be doing a full buffer, so this is just a guess. If the S3 as PSRAM and the above is true, you should be able to force the frame buffer into PSRAM..

Please note, this is just a guess based on the reported symptoms. A good trawl over all the source code is need to track down the issue. But my best guess will be a buffer overflow somewhere in the gif code.

1 Like
3

What I donā€™t understand about this is why would they even develop a 480x480 pixel screen and a board to go with it that isnā€™t capable of doing anything on it in the first place? The animatedGIF is only 12 frames. Really disappointing hardware.

Iā€™ll send your advice to the programmer helping me. Thank you.

4

Note there are smart ways they can do things. My feed back was more just a guess based on your overview; i.e. a rough place to start looking.

Just had a very quick look at the device and it does have PSRAM so I will assume the code base is using that to hold one or more frames.

Displays can be fun/frustrating e.g. you can have much smaller buffers that only cover part of the screen, fill that with data and send to a part of the screen and repeat. This works with small memory models, but does not allow full frame buffers. A full frame buffer is much easier and faster to manage when dealing with full screen changes. smaller buffers are faster for just part of the screen change.

So there are pros/cons.
But I could see something around this causing your issue as I have been there, done that with my own screen drivers.
even a simple thing where the bug could be the frame buffer is 1 or 2 bytes too short, that would only cause an issue when its sending that part of the bufferā€¦

But code analysis is whats needed to track it down.

1 Like
5

Can you find (or create) a smaller GIF with fewer frames? If you find that and run it succesfuly then that would indicate that the problem is associated with memory (either the amount available or the way it is being used) or possibly the encoding for your GIF. If you canā€™t run a small gif then the problem is more likely in the code or the library.

2 Likes
6

image
image722Ɨ112 4.72 KB

I tried another GIF, only 629 KB instead of 3876 KB. Works!
But the smaller version is only 320x320 pixels instead of the full 480x480, and so doesnā€™t fill the screen.

Am I correct in believing then the 480x480 will work if I can somehow severely reduce the file size?

image
image742Ɨ782 68.2 KB

Itā€™s a real shame these boards were limited so severely, yet still advertised to work with the 480x480 boards.

So I suppose I could just reduce the number of frames in the 480x480 animation. That would the size down, right? Is simply scaling this 320x320 animation up programmatically an option instead, or would that bring us back to the same problem all over again?

7

I would still be looking for a bug with the way the library uses the memory. It should be able to do this, but if there is a bug right on a limit, that should be an easy code fix, once found.

The fact that the small gif worked is very promising

1 Like
8

Itā€™s quite possible they work just fine as a 480x480 display, just not with display formats and procedures that have the particular memory requirements that your program has.

It depends on exactly what the problem is and how the GIF library is storing and accessing that data. For instance, if itā€™s a program memory problem then if the frames are being paged in only as required then the size of the frames is probably the issue. if the problem is that it is trying to load them all into main memory at once then the number of frames will be the issue.

Again - it depends. Whether or not loading a smaller image and scaling it up would help depends on how that scaling is done and how much memory it uses. Itā€™s all determined by details buried deep in the library code.

1 Like
9

Thanks guys, this has been extremely helpful.

Iā€™m going to start through trial and error and a number of different quality versions of the GIF to find the max size it can handle (seeming as the smaller one worked). And then after that Iā€™ll try to look for other ways to optimize.

Related question, anyone know a free and easy way to convert the GIFs to the correct format? Found this one but it has a max file size of 300kb:

10

image
image710Ɨ125 5.69 KB

Unfortunately the newest version of the 480x480 GIF didnā€™t work. Even though I severely shaved the file size down to 1221KB.

In the image provided, the original portal 480x480 GIF, almost 4MB, crashes the device.
And the little one is only 320x320 and works! But I must have it fill the 480x480 screen.

11

I would still be looking for a bug with the way the library uses the memory. It should be able to do this, but if there is a bug right on a limit, that should be an easy code fix, once found.

How would I find this bug? The device crashes before I can even connect it to the serial monitor.

It depends on exactly what the problem is and how the GIF library is storing and accessing that data. For instance, if itā€™s a program memory problem then if the frames are being paged in only as required then the size of the frames is probably the issue. if the problem is that it is trying to load them all into main memory at once then the number of frames will be the issue.

How would I find this out?

Has anyone worked with these specific devices before and been able to pull off a 480x480 animated GIF?

12

Normally you can connect to the debug comport, theb press the reset button, if its using an external uart to usb chip.

When you say it crashes, does it auto reboot and re run the bad code, or is it just freezing?

To help debug, you could try to add a wait for some sort of input befor loadimg the gif, then ensure debug is working then provide the inout to continue and load the gif.

For testing see how you go increasing the size of the gif bit by bit to see if you van find a size that always works and the size at which it will fail.

13

Your original post indicated that you are getting the message ā€˜Hello!ā€™ on the serial monitor, so it is not crashing before then. You can add more messages in the main loop until you track down the exact point where it shows the last message and crashes. I suspect that you will get one message only right at the start of the loop. That would be a good point at which to print the available memory. I think the PSRAM figure will be the interesting one. You can also create an empty sketch for memory size to compare so that the difference indicates how much your display sketch is taking.

  Serial.print("Total heap: ");
  Serial.println(ESP.getHeapSize());
  Serial.print("Free heap: ");
  Serial.println(ESP.getFreeHeap());
  Serial.print("Total PSRAM: ");
  Serial.println(ESP.getPsramSize());
  Serial.print("Free PSRAM: ");
  Serial.println(ESP.getFreePsram());
2 Likes
14

Any chance you can post your code ?
I had a quick scan over the gif library source and I have a few things floating around in my head, the main one being if you have an ā€œempty frameā€, that will exit with 0 which will exit the show frame while (in the examples i saw).

1 Like
15

That will be in ā€˜portal_image.hā€™.

1 Like
16

The PSRAM functions may or may not work if the PSRAM has not been initialized. I could not see anything in the gif library about using PSRAM, but it was allocating a few buffers via an abstract alloc call.

1 Like
17

The MCU comes with 8Mb of PSRAM, which I presume is required for large displays. If itā€™s not being used that could possibly be the problem!

1 Like
18

the gif code tries to allocate a few buffers. Some for speed reasons some for basic use. it states it can get away with as little as 22K.

// The goal of this code is to decode images up to 480x320
// using no more than 22K of RAM (if sent directly to an LCD display)

Which I also noted with the smaller buffer of 22k it was limited to 480x320. I did not really see if it would auto crop or just not plot at all (need more time to look over the code).

But lets wait to see what the memory looks like from the OP

1 Like
19

If itā€™s not going to use 16MB of flash and 8MB of PSRAM then itā€™s the wrong library for this device.

1 Like
20

Thanks for sticking with this, guys, these are some interesting insights. I get home at about 4pm after work today and will go through them one by one.

1 Like