The ILITEK ILI9488 is one of the larger and cheaper SPI displays available to the maker community,, available in 3.5" and 4". However, there are a few workable issues that prevent this display from being great.
Specifications
What's called ILI9488 is actually the LCD controller with an optional touch panel, you can mostly find it with XPT2046 resistive touch controller.
5v to 3.3v regulator, please note that you should short J1 if you're using 3.3v
General Issues
The ILI9488 can be bought in two versions, one with a diode and one without, I've yet to determine the functionality of the diode, but it seems that others think the diode can prevent the display from releasing the MISO line, unfortunately I didn't keep the diode so I can't validate this claim.
The schematics are available if you want to explore it further.
LVGL Issues
When I first started working with my ILI9488 the colors were a bit off but I attributed it to cheap and low quality display which was probably defective. but then I've started wondering if its possible to fix since the controller can be configured with other voltages it pushes to the panel.
I wanted to see if the display will work at 80Mhz, unfortunately its not, but seeing some of the graphics I guess that it can be fixed with a logic analyzer and some patience.
80Mhz
ESP32 Specific Issues
While it might not be specifically ESP32 issues, its issues that you might encounter while integrating it with ESP32. The most prominent issue is the way CS works in ESP32, it seems that CS issues are common in the embedded world, the STM32 has a similar issue with NSS not properly controlled by the cube's code.
To support multiple transactions with multiple devices on the same SPI bus, the ESP32 switches off the CS signal between transactions which is great, however, the way ILI9488 works is that if you switch off CS after you've sent a read request, it switches from 4-wire SPI to 3-wire SPI.
There are a few ways to solve this issue:
1. Use software CS, set to low before a transaction and set to high after you're done receiving.
But wait, what's the problem with working half duplex (3-wire)?
Well, if you share the same SPI bus with the touch panel it locks the MISO line on LOW and won't allow the touch panel from transmitting touch data.
Solutions?
Well, I've given up on getting data from the display and for most uses its good enough, so you can disconnect the MISO line from the display and keep it working for the touch panel.
Another possibility is to put a 1k resistor in series to the display MISO. This way the panel won't lock the touch and you can keep the buggy code until you can get it fixed to your satisfaction.
Unfortunately working in half-duplex is not currently possible if you're using the LVGL driver since it will attempt to set the bus to 4-wire mode for the touch panel to work.
UI for Embedded is always a hassle, find the right MCU, find the right Display, connect the right wires and that's even before writing the first line of code that actually shows anything on the display, drivers, graphic libraries and input libraries can be a pain to use, not to mention a pain to write.
Fortunately, we no longer live in a cave, we have PlatformIO, LVGL and drivers for many of the LCDs available for commercial use and maker community.
I propose a way to start quickly so we can save some of the bring up time for a new setup, configuration is done in the kconfig way, so its really go through the menu, change a setting, compile and test.
There is a getting started for ESP32 on LVGL github, Gabor Kiss-Vamosi wrote a tutorial and we have some documentation, Espressif event got an example repository, but I've discovered my way is a bit more flexible and easier to use once its set-up, since you can develop on the desktop (LVGL refers to it as Simulator) and don't have to upload each revision, plus you don't have to do it, just fork my project and you're good to go. In any case, the full instructions are below, so you can mix and match versions and for me it provided a pretty much consistent experience.
Hardware
There are many kits you can buy, each one with its own quirks but my goal was to test what I had and unfortunately I had none of the kits.
Please note that I have yet to test any of them, so do your own research before purchasing, the following are affiliate links.
ESP32-LCDKit
The ESP32-LCDKit looks like its the most versatile for Graphic development, the schematic is available and you can use which ever ESP32 you want (as long as its a 38 pins)
ESP-WROVER-KIT
Next in line is the ESP-WROVER-KIT, like the previous one, its schematics are also available, but one major drawback is that the kit does not include a touch screen, only a display so its less suitable for interactive UI.
Last but not least, The WT32-SC01 seems like it has great potential for having all the hardware on a single board, mounting holes and capacitive touch screen, schematics and code samples are available.
Source Control
We can't really start a project without source control, how can we track changes? how can we go back to a stable state?
Than take library.json from our example project and copy it into lib/lvgl root, what this library.json file actually does is allow you to select which parts of lvgl gets compiled.
Native / Desktop Drivers
There are multiple ways of working with LVGL but one of the better ways is getting your UI completely disconnected from your business logic and running the UI on your PC, this way its easier to design, debug and verify, on top of it, you can use LVGL's snapshot API to automatically validate your views so they can stay consistent no matter which changes you do.
The way LVGL works on the desktop is by using SDL2.
Then we'll modify library.json to include SDL dependency, otherwise PlatformIO dependency detection won't work properly due to the way SDL is included through a #DEFINE macro.
Lastly, we'll update include/native/lv_drv_conf.h in appropriate place and copy our menu configuration keys to SDL configuration keys, otherwise our menu won't control SDL (Desktop) display properly.
#include "lvgl_native_drivers.h"
#define USE_SDL 1
#define SDL_HOR_RES CONFIG_SDL_HOR_RES
#define SDL_VER_RES CONFIG_SDL_VER_RES
Since our native environment will need to use SDL, we should also add SDL2 to lib, there are different libraries for different environments, such as Windows and Linux.
In Windows, we'll need to download SDL for mingw, extract it into our lib folder and copy library.json from this project to your SDL library
Now that we have our desktop setup, we also want our hardware setup so we can flash our device and see how our design looks on the real hardware.
Please note that the drivers are not always configured ideally, if the colors seems a bit off, you should read the datasheet and make sure everything is configured properly.
Some drivers are not working properly with PlatformIO's scons configuration and needs to be enabled/disabled on a per-file basis, you should look in library.json as an example.
Another thing we want to tell our library.json is which framework it should work with, for example, in our setup we have native and esp32 environments and the esp32 drivers should not be compiled on the native environment since none of Espressif's libraries exist or even needed for desktops.
Then we'll modify lvgl_helper.c to include "lv_conf.h" right after "sdkconfig.h", the vanilla setup assumes your lvgl is part of esp32 components which can make desktop configuration a problem.
These two scripts and their setting enables platformio.ini to use the target scripts for easy configuration. To see which scripts are installed for each environment:
esp32 Platform size Program Size Calculate program size
esp32 Platform upload Upload
esp32 Platform uploadfs Upload Filesystem Image
esp32 Platform uploadfsota Upload Filesystem Image OTA
Now that we've assigned each configuration output to a different folder under include, we should tell platformio to include headers from these folders so each environment will get a differentset of configuration files.
LVGL Configuration
LVGL uses configuration files separate from the driver configuration files to configure some aspects of it, such as fonts, widgets, colors and layouts, but we need to tell it where to take the configuration from.
We do it by adding these flags to build_flags for relevant environments in platformio.ini:
-DLV_LVGL_H_INCLUDE_SIMPLE
-DLV_CONF_INCLUDE_SIMPLE
-DLV_CONF_PATH=lv_conf.h
Environment Hardware Abstraction
There is a small library in the demo project called lvgl_hal, it contains the setup for the drivers, obviously different from native to ESP32, you may need to modify it to your environment / programming.
Lets start with ESP32 configuration, LVGL can run lean or resource intensive, larger buffers may help with rendering speed, caching images and data can also help, my usual setup is 240Mhz CPU speed and 80Mhz PSRAM speed.
To make these configuration, you'll need to modify ESP32 configuration by running:
pio run -e esp32 -t menuconfig
We can configure ESP32 drivers:
pio run -e esp32 -t lvgl-esp32-drivers-config
And native drivers, which at the time of this writing is only resolution:
pio run -e native -t lvgl-native-drivers-config
And lastly we'll want to configure our lvgl, using the same configuration for both desktop and embedded can help you to find bugs quicker.
pio run -e esp32 -t lvgl-config
pio run -e native -t lvgl-config
Runner
The runner library is intended as an abstraction of the main function, on a desktop its int main(argc,argv), on ESP32 its appmain() and on Arduino its setup() and loop(), instead of writing the same ifdefs everywhere, just copy the runner library, include it in your main file and use it:
MAIN(){
}
Unfortunately my ILI9488 is not configured properly (more on that later)
If you're looking for a solution to the ILI9488 configuration issue, you can read about it here.
I've been playing with the idea of running linux on ESP32 since the first days I've met its more robust module, the WROVER-B, on paper it seem possible since its a dual core 240Mhz and has 16MB flash and 8MB RAM, compared to our antique machines that could run linux, it seems like a beast.
Doing some research on it, I've understood that its MMU is insufficient for running Linux on it. during the past few years I've been looking into it to see if anyone else found the time to implement it and eventually I've decided its going to be a good opportunity to learn a bit more about RISCV and Buildroot. Two subjects I've been putting off for longer than I'd like to admit.
I've decided to start with something rather to write it all from scratch, which I didn't have time or energy to do for this project, I've looked into QEMU emulation for RISCV but taking this project apart and getting only a few components out of it to run on an embedded system seemed like too much work. Eventually I've found out about Fabrice Bellard's TinyEMU (demo).
RISC-V
RISC-V is the new ISA kid in the block, well, not really a kid and not really new, but it becomes more and more popular, Espressif got out the ESP32-C3 at 2020.
Previous Successes
Max Filippov patched the kernel to support ESP32 back at 2019, I'm pretty sure it runs a lot faster since its not an emulation.
Li XiongHui wrote the juiceVM which implemented RISCV ISA and runs on ESP32, he wrote about it on whycan and reddit and has video of it booting on YouTube, the video does state x30 speedup, which means the system booted in about 6 hours. Li never released the source code so the only improvements that can be done is by him and judging from my own life, you never have enough time for these things.
TinyEMU
"TinyEMU is a system emulator for the RISC-V and x86 architectures. Its purpose is to be small and simple while being complete."
The ESP32 is a dual core 240Mhz MCU, it was released at September 2016 and its still one of the best value for money MCU you can get, one of its versions has 8MB or RAM and 16MB of FLASH. That amount of RAM on any of its competitors takes more than "I want it" to get it working, at the time it came out especially so. Espressif did an amazing job with esp-idf and one of their best features is listening to their customers and with the help of the maker community they've built an amazing framework.
TinyEMU on ESP32
Will it even compile?
Apparently yes, making it compile was very easy, some tweaks here and there and a missing standard library and it was compiled perfectly.
But what can I do about memory? the ESP32 only has 8MB and half of it is not even accessible as a standard but rather bank switched with its own APIs (himem).
My thinking at the time was that it doesn't matter so much since the kernel will probably not need so much memory once it starts, for example, if I don't access files, the memory holding the file system and the file system functions will be left alone other than a periodic flush.
So something like a swap file will probably be good enough for this experiment, right?
Then I've discovered that 3MB for the virtual pages is too slow due to the PSRAM, using 80Mhz only improved a bit, so I went on a journey to find the fastest search trees, I've tried splay tree and eventually rested on AVL tree which was good enough.
But I could squeeze more out of the ESP32, I've implemented a rudimentary direct-mapped-cache so most memory accesses won't even search for their page (Professor Luis Ceze has a great presentation on the subject).
I was still not happy enough, my SD card is slow and no matter what I did, it slowed things down. I've measured how many page faults I had and decided that dirty pages should only be written once they are abandoned, so my LRU cache pushed dirty pages into himem and only when these himem pages reclaimed they got pushed to the page file.
At that point I was content enough, my kernel booted in 1:35 minutes.
UPDATE 2023-04-13: Improved Speed by 40%
Buildroot
"Buildroot is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation."
I've always wanted to learn how the big guys do it, how embedded cameras, kiosks and perhaps even satellites gets their OS build without all the bloat and package managers.
So once I got my basic emulator working (without all the optimizations), it was time to start learning buildroot.
Apparently its simple, you just download the archive, work through a few menus, read some documentation, modify the rootfs with overlays and you're done. issue a make command and you have your kernel and your rootfs.
In the old days (or so I've heard), you've had to build RISC-V toolchain and patch the kernel to get things going, These days buildroot comes with a precompiled toolchain from bootlin, so the whole experience was fun and easy to learn.
Thoughts for the future
Using the emulator to run embedded RISC-V code, implement SiFive GPIO and provide more flexible VM for running code on multiple embedded platforms. Communicating over the console is implemented in TinyEMU patches, or see here or through virtio.
Closing
This was a wonderful journey of learning, I've learned more than I wanted about: