Is ESP32 Ready for some AI?

IoT is a passion of mine for quite some time, so imagine how happy I was to receive a gift from Semix, the all new ESP-EYE v2.1!

Semix specializes in representation and distribution of world leading manufacturers of Electronic Components, Modules and Integrated solutions in Israel, in this case Espressif and Manica.

While the board looks like it was pretty thought out (ferrite beads all over it!), it does lack GPIO connections, looking like it was directly made to demonstrate the ESP32 capabilities rather than a maker Swiss army knife. In the end of this article there are some other options if you're curious about combining these capabilities with your other crazy ideas. :-)

I've started by looking up some information, videos, design reference and anything I can find on that module and eventually I've cloned the esp-who project.

I've followed a few getting started examples but I really love what PlatformIO did with Visual Studio Code so I had to set it up to compile in PlatformIO. BTW, PlatformIO already has the esp-idf framework, which makes it very easy to use with ESP32!

You'd be surprised how much faster a good IDE can help you understand a project structure!

Eventually I got curious enough to see how they did it so I've begun to dig up a bit. the face recognition part seems to be based on MTCNN, where it's actually 3 separate networks integrated with algorithm glue.

source

The audio keyword recognition seems to be very similar to TensorFlow demo I've seen, but during my research I've seen a few other examples that gave me the impression Espressif did not use TensorFlow.

However, TensorFlow lite could be used for another project I was doing research for, so I've decided to take the plunge and see if I can compile it for ESP32.

The getting started is pretty straight forward, download, compile, run and of-curse learn. but to really get started you need to get your feet wet and test the hardware compatibility since TensorFlow lite was not specifically ported to ESP32. So what do you do? you run the suite tests on the ESP32.

Let me assure you, all the tests passed, some did take some time to complete but that's because it wasn't optimized for tensilica yet.

I've also run the micro speech demo, but since it wasn't optimized, it took 360ms to process 100ms of audio. I did find some optimizations for the inference engine and boom, this thing is fast! (80ms for FFT + inference!!)

If you'd like you can find more pretrained models and examples in TensorFlow website.

To me this little exploration opened a whole new world and ideas of AI on ESP32, if you had any doubts, you should definitely check out TensorFlow lite on ESP32!

source

Please note that there are other variations of the ESP-EYE (or ESP-CAM) with different capabilities:

1. ESP32-CAM

The ESP32-CAM also has 4MB of external PSRAM, it exposes some GPIOs for extensibility and even has an SD-CARD slot but no Mic.
Notice there's no USB plug, so you'll need an external USB-TTL adapter to program this device.

2. M5Stack Official ESP32 Camera Development Board

Almost looks like a copy,I couldn't find any reference of external PSRAM, so if anyone knows, leave your comments please.

There are empty footprints for MPU6050, BME280, Mic and lithium battery connection, so it can be easily used for your wearable projects.

Notice it has USB-C connector.

3. TTGO T-Camera Plus

That thing is sweet!
8MB of PSRAM(!!!)
1.3 inch LCD
Microphone
SD card slot
Battery connection/charger
and a USB connection.

4. TTGO T-Camera ESP32-WROVER-B

Another notable module, very similar to the T-Camera Plus.
We all know that video/imaging takes power, how are we expected to write low power applications when our MCU takes most of our power? well, if your particular application doesn't require you to always scan your camera, you can put your MCU to sleep and wake it up only when there's movement with a simple PIR sensor.

Read More

Tags: Camera, esp32, TENSORFLOW

Dror Gluska July 17, 2019  6 comments