ESP32 as Professional Grade/Industry 4.0 Device

According to Forbes, IoT is a $264 Billion market while Industry 4.0 will be $152 billion by 2022, these predictions can lead us to believe in IoT as more than a fleeting trend and should prepare us for the future to come where embedded developers will be plucked from other software development fields, as micro-controllers get more powerful they can also host a more modern programming paradigms and we’ll explore some of these for the sake of making embedded development more robust as well as faster time to market development.

Many words have been written about the advantages and the future that Industry 4.0 devices can achieve for manufacturing, factories, plants and countless ideas have been in the open, hackathons have been performed to find the next killer device, services and applications. This article is not about that, its about how to achieve the fastest way to develop such applications for wide scale deployment.

Industry 4.0 plays a significant role in industrial optimization, by monitoring systems and analyzing the sensor inputs either by statistics or by learning algorithms, failure prediction, load analysis and and optimizations can be achieved. In turn cost cutting and growth are two byproducts that every business needs.

While some of the big time players offer ready made solutions, all the parts have been in the open source community for quite a while, IoT no longer a toy for the geeks but a growing movement to improve a person's life and can be applied in industrial setting.

In my humble opinion, IoT and Industry 4.0 are not there to take jobs but to take the mundane and boring work and allow a better working condition, more interesting and less erosion and by that increase engagement and productivity.

Industrial Usage

While the ESP32 is not a replacement for commercial PLC/SCADA devices, it can achieve that goal pretty fast, sensors and relay modules or SSR modules can make connectivity relatively easy task and if you’re missing a few inputs or outputs, a multiplexer can be a solution as long as your switching speed is within the tolerances.

But what about software?

https://github.com/node-red/node-red

At this moment, node-red is the most popular IoT control software, it is very easy to use and many tutorials have been done in both video and articles. Other options are Eclipse Kura and Project Flogo.

If your needs are more industrial, a few SCADA open source project:

http://rapidscada.org

RapidSCADA - Rapid SCADA is free, open source, full featured SCADA software.

Eclipse NeoSCADA™ is flexible. It is not an out of the box solution, but a set of tools that can be combined in many different ways. It provides development libraries, interface applications, mass configuration tools, front-end and back-end applications.

IndigoSCADA - A small footprint SCADA system entirely developed in C and C++ with multiple OS support and multiple front end protocol drivers.

ScadaBR is free, open source free software for the development of Automation, Data Acquisition and Supervisory Control applications.

SZARP is a full-featured SCADA system designed for monitoring slowly changing industrial processes, for example operations of urban heating systems. It is completely free software, published under terms of the GNU General Public License 2.0 (or any later version).

Why ESP32

While all of the ideas in this article are common to most of the IoT devices with a decent hardware, this article is about ESP32, which is even more suitable than most of the other micro-controllers I’ve encountered lately.

While Espressif is well known in the hobbyist community, it is less used at the professional and industrial sectors, in my opinion not rightfully so.

Espressif ESP32 gives a fair fight against many micro-controllers. Maxim, Texas Instruments, Microchip, Nordic Semiconductor, NXP, Silicon Labs, ST and Atmel all provide microprocessors that can compete, some more popular than others, some provide a free IDE, some provide very expensive SDKs and most of them has a sort of RTOS. What Espressif did right is opening up their SDK and development environments so others like PlatformIO can make development for their hardware accessible, their choice of FreeRTOS allowed them to open source the entire framework and by that help anyone willing to learn about embedded development to get the information, details and insights that only a source code can provide.

Around accessibility to hardware and software a community can be created and in turn the hardware can get support, software, components and drivers like no other, essentially by giving the community esp-idf, design documents and specs, this gift keeps on giving back, volunteers contribute back their time and effort while Epressif can sit, relax and plan their next killer chips.

To give a good example for how not to push a product we can take Intel Curie, Galileo, Joule and Edison, while Intel did make an effort to push their maker chips into the market, most of them were way overpriced and under-performing and in spite of their efforts by giving many many of these devices as gifts, prizes, participating in hackathons and more, their downfall was the lack of documentation and community support. The companies that did try to build a business around Intel maker chips are now left hanging and need to redesign their products.

https://www.cnx-software.com/

ESP32 Ecosystem

Some of the components that compose the esp32 ecosystem are:

esp32 microprocessor - A dual core 240Mhz microprocessor, 520kb ram, RTC, ULP, 34 GPIOs, network connectivity and multiple UARTs, I2Cs, I2Ss, SPIs, CANBUS, ADCs and DACs with external SPI EEPROM that can be from 1-16MB.

esp-idf – Espressif made a great effort to build a Software Development Kit (SDK) which exposes most if not all of the hardware’s capabilities.

Networking - WiFi/BT/Ethernet connectivity, lwIP TCP/IP Stack.

Security/Cryptography – Encryption acceleration instructions (AES (128, 192, 256) / SHA (1,256,384,512) Acceleration), Networking encryption, Flash encryption, Secure Boot and JTAG Disabling. As well as mbedtls library (formally known as PolarSSL), OpenSSL API translation and libsodium.

Cloud Ready – both AWS IoT and Azure IoT components exists and tested.

FreeRTOS – A full implementation including Tasks and real-time task scheduler, Timers, Queues, Interrupt handling, critical sections and events.

Modern C++ 11/14 GCC Compiler – Modern C++11 and up offers many advantages over C++98, among them is increased productivity, speed, stability, safer code and portability are the right buzzwords but smart pointers, improved collections, lambdas and threads and more are the most common advantages anyone should start looking into.

Industrial/Professional Grade

By combining all these components, Espressif made sure that esp32 will fit many applications, including industrial and professional grade, but what does it mean to be able to write industrial or professional grade applications?

First, lets define industrial and professional grade applications.

Industrial/Professional	Hobbist
Paid by corporations/factories	Paid by private consumers/Free/Open Source
Multiple teams/developers are working for a common goal	One or more developers are working on their spare time or with very little budget
Mission-Critical Systems, essential for a business to grow	Occasional downtime is expected, nothing extreme will happen if the system is down.
Reliable, Robust application, should prove business value	Reliability/Robustness is not guaranteed.
Fault Tolerant/High Availability/Recoverability	Repair/Support if any, can be at business hours
Predictable/Repeatable Performance and Timing	Performance is not defined and may not have a negative effect
Multi Layered, Discreet Components, Separation of concerns	Boundaries are not clear, programming paradigms are flexible.
Queued Work, Interrupt Handling	Slow Response and inconsistent results might be acceptable.
Connectivity Essential	Connectivity Optional
Higher Costs	Medium Costs

Lets get things straight, industrial/enterprise applications do not mean higher quality of code and hobbyist software does not mean lower quality, all it means is that they both have different requirements, for example, a hobbyist application developer might spend more time on user experience and UI, a commercial developer can and probably will spend more time on stability, load testing and making sure the software won't fail when a 30,000 RPM drill is coming at an aluminum block at a very fast speed. different requirements, that's all. More over, these days the boundaries between industrial and hobbyist products are getting more and more fuzzy because there is more awareness.

Now, lets imagine you need to write a program that almost never fail, imagine you need it to be up and running all the time, imagine you need to know what its doing, how its doing, its health, what happened to it when it failed, you need to update it when you find a new problem or find a more efficient way to do something, imagine its as secure as possible.

“a program that almost never fail”

While it is a nice dream, software does fail. But what can we do to solve the most common failures and improve stability?

• Use Modern Compilers and programming paradigms, object oriented development, RAII, smart pointers, collections, lambdas, anything that can help you get your job done in a safer manner. Raw pointers have been around since C, but smart pointers are safer for use. Prefer dynamic casts and static casts over reinterpret cast and c casts.
• Tests are essential to a stable application, while running tests for xtensa processors on a PC inside QEMU could get complicated, you can use abstraction layers and mocks to run and test most of your code on your development OS, esp-idf and PlatformIO have testing facilities to ease the pain. Also, as the code is a standard C++ code, you can use cppcheck for static code analysis.
• Always use source control, even for POCs.
• Build Automation, Versioning.
• Crash Dump Analysis/Stack Overflow Detection are part of esp-idf, use them.
• Logs are important for diagnosing a problem or a program misbehave, while esp-idf provides a logging mechanism which can be used for tracing, its also good to be able to save these messages to files and later transmit for analysis and post mortem or use a network syslog.
• Enable OTA updates, use versioning, certificate signing and a staging environment that contains all the hardware this firmware will be deployed.

What can we do to make the software more reliable? robust? secure?

• Input Validation - always verify your inputs, buffer overrun is still one of the major problem that leads to hacking, SQL injection is still not dead.
• Tests - test your code with invalid input, long WiFi names, external hardware glitches. disconnect power, reconnect power. worse case errors should lead to crashes, not compromise security.
• Use more advanced paradigms, like queues, events, callbacks, exceptions, micro components, layers, inversion of control, control weaving.
• Logs should have everything you need to diagnose a problem from remote, you shouldn't fly someone across an ocean just to find out a configuration problem.

What can we do to make the software fault tolerant, highly available and recoverable?

• Fail Fast - failing fast leads to development finding the problem at development time, never swallow errors, if you must, log them first.
• Exception Handling - an exception happens at an exceptional state, it should not happen, if it did, find out why. don't use exceptions for non-exceptional errors.
• Reboot upon failure - esp-idf is configured to reboot on panic, invalid pointers access will lead to reboots, . Stack overflow leads to a reboot.
• Watchdog - sometimes failure does not mean a crash, sometimes it means that a software no long responds at the designed time, sometimes a task enters an infinite loop and waste all its time on doing nothing. watchdogs and heartbeats are designed to detect these situations, use them.
• High CPU usage detection - while not part of FreeRTOS failures, if you see a poorly performing software, add it, it's not hard. 
• Brownout detection - brownout is a state of too low power for the hardware to work properly, the system should reboot on brownout detection. Fault injection could be just another way to hack the hardware.
• If appropriate, use more than 1 device, perform election to select the better device, if it fails, failover to the redundant device, alternatively use heartbeat to select the master device. An example usage would be a redundant controller on a CCR breathing system.

What can we do the to make the software more predictable? High performance? Accurate Timing?

• RTOS - RTOSs provide multi tasking capabilities, queues, interrupt handling in a safe way.
• Multicore, faster microcontroller - if your core is busy all the time, it won't be able to react to changes in time, leading to reduced performance in the best case and dependent hardware crash/accident in the worse case.
• Use Interrupts and Timers - its the age old Polling vs Push.

A Technical Drill Down

esp-idf is a very interesting SDK, Espressif made a huge effort to make it as easy to use as possible, you can find APIs controlling the hardware, drivers, pthread compatible APIs and C++ stdlib, even glob, regex and tar and miniz, all of this makes programming for the ESP32 relatively easy with many features that makes porting existing code a breeze.

esp-idf includes a few interesting features and components that aids building a more robust application, among them is SPI filesystems, both FAT and SPIFFS (spoiler: FAT is faster), you can use the filesystems to store log and data files, the SPI EEPROM include a wear leveling layer which helps to prolong the amount of writes possible. Settings can be stored in NVS so formatting the filessytem won't affect it, lastly if you need more than the SPI memory, you can always use the SD-Card Library.

Cloud Connectivity is essential to large scale monitoring and control, esp-idf includes AWS IoT connectivity and Microsoft Azure IoT connectivity works as well, TelegramBot API, CoAP and MQTT are available as well.

lwIP is a IPv4/IPv6 TCP/IP Stack which have a few interesting features, among them is DNS/mDNS resolution, SNMP, DHCP, AUTOIP, PPP and L2TP , all of them are the basics of networking, among the things you can do with it are DHCP servers, SNTP, PING, HTTP(s) Client/Server, CoAp Client/Server, OpenSSL Client/Server.

nghttp2 is an http2 client/server.

mbedTLS/libsodium are cryptographic libraries, using them is mostly preference, both are very similar, and have their own set of features and quirks, you should make an informed decision by researching them.

FreeRTOS APIs include Task management, including a scheduler and monitoring, data structrures that incude Queues/Stacks, Ring Buffer, Locking Mechanisms like Semaphores and Mutexes. Interrupt Aware APIs

But free is not always good, free means you can use it without paying but sometimes you need it with support, sometimes you need it with a guarantee, FreeRTOS have a commercial alternative called OpenRTOS, it’s the exact same RTOS just with a different licensing. Another Alternative is the SAFERTOS. For more information you can see a comparison here.

Power Management is an important features for low power applications, the ESP32 have multiple sleep levels with an ULP processor that can reduce power requirements even further.

Debugging – while personally I don’t believe in straight up debugging as its usually reducing the quality of logging and on production you can mostly work with logs and crash dumps, Espressif got OpenOCD with JTAG working. esp-idf comes with a unit test app based on Unity and  If you choose to use PlatformIO, you can use its testing facilities as well.

RTOS

Other RTOSs are not bad, on the contrary, but SDKs have a very steep learning curve, while getting an example up and running could take anywhere from 5 minutes to a few hours, being proficient in that SDK takes time, learning its quirks and overcoming its limitations takes study, but if you take a look at other RTOSs , most of them have almost the same structure and APIs.

But why use a popular RTOS? What does it mean?
Learning about a device from its datasheet could take a long time, especially those 100 pages datasheets for complex devices, on the other hand, using a library that is already working and tested can take significantly less time.

One of the advantages of being a part of the Arduino community is that its a huge community, so many developers already read the datasheet and wrote drivers and sample codes, some of them are very low quality but like every kind of community, some are very good.

ESP32 can be used two additional RTOS that I know of:

Simba http://simba-os.readthedocs.io

NuttX http://www.nuttx.org/

For more information, you can look at this comparison.

Licensing

While the ESP32 SDK is Apache License, always check license for other components and libraries when you use them. FreeRTOS have a different license, some LCD display drivers have another etc' etc'

Always consult a license lawyer when using for commercial applications

Rapid Prototyping

SDKs are a great way to explore the device’s capabilities but although C++ these days is very easy to learn, sometimes the developers who do the proof of concept and the exploration come from a different field, electronics, mathematics, physics and biology to name a few and C++ might not be their preferred language, a few other languages have been implemented on the ESP32

- LUA - Lua RTOS is a real-time operating system designed to run on embedded systems, with minimal requirements of FLASH and RAM memory.
- MicroPython - aims to put an implementation of Python 3.x on microcontrollers and small embedded systems.
- Basic - Basic interpreter hidden in esp32 silicon

A Few Interesting Projects

Web Radio - MP3 web radio project.

Camera Demo - With OV7725 camera module.

Alexa Voice Service (AVS) client for ESP32- https://github.com/GOLDELEC/Huan

WebSocket example project - https://github.com/ThomasBarth/WebSockets-on-the-ESP32

WiFiManager - WiFi Connection manager with fallback web configuration portal

libesphttpd is a HTTP server library. It supports integration in projects running under the non-os and FreeRTOS-based SDK. Its core is clean and small, but it provides an extensible architecture with plugins to handle a flash-based compressed read-only filesystem for static files, a tiny template engine, websockets, a captive portal, and more.

CAN Bus Driver

Low Power - deep-sleep functionality if the ESP32

Console component - create an interactive shell.

ESP32 QEMU - https://github.com/Ebiroll/qemu_esp32/

ADNS3080 - Is a mouse optical flow, while you can find example code at ArduPilot, I couldn't find a source code but its a standard SPI and should be relatively easy to port, here's an example use.

Ethernet Connection - Using LAN8720

Development Environment

Eclipse https://exploreembedded.com/wiki/Setting_up_Eclipse_for_ESP-IDF

VSCode http://docs.platformio.org/en/latest/ide/vscode.html

Visual Studio http://docs.platformio.org/en/latest/ide/visualstudio.html

ATOM http://docs.platformio.org/en/latest/ide/atom.html

ESP32 Books, Tutorials and Samples

Neil Kolban has invested a lot of time writing a book, samples and tutorials and should get a special thank you due to the amount of effort he made.

https://github.com/nkolban/esp32-snippets

https://leanpub.com/kolban-ESP32 (https://www.robolinkmarket.com/datasheet/kolban-ESP32.pdf)

https://www.youtube.com/channel/UChKn_BlaVrMrhEquPNI6HuQ

Espressif Technical Reference: http://espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_en.pdf

Development Kits

ESP-WROVER-KIT looks very interesting at $47.00, comes with external 4GB RAM, LCD and JTAG .

ESP32 is also available in many flavors and configurations on the cheap:

A Kit, In a simple breadboard form, An alternative breadboard form, With OLED, In an easy way to change the SPI EEPROM, or this.

In theory you can use this chip to expand the EEPROM to 16MB (untested).

With a Battery, With a LoRa module and OLED display, With Soil, Temperature and humidity sensors, Battery and display in a compact form.

If you need a Flashing Fixture.

Further Reading and References

You can always explore other options and find the best suitable for your needs, you can find a list here (http://www.softwareforenterprise.us/list-of-free-and-open-source-scada-m2m-systems/)

http://esp32.net/

https://www.slideshare.net/KaiWaehner/iot-open-source-integration-comparison-kura-nodered-flogo-apache-nifi-streamsets

I'm not claiming china subsidizing Espressif, but they do with SOC development (https://www.cnx-software.com/2017/07/15/office-factory-business-model-and-ambitious-plans-of-shenzhen-xunlong-software-orange-pi-maker/ - http://www.minimachines.net/a-la-une/orange-pi-steven-zhao-52858)

https://www.i-scoop.eu/industry-4-0/

ESP32 Microcontroller Hardware and Software

IoTuz software design challenges and ESP-IDF (Iot Development Framework)

Summary

In my opinion 3 things drive the Arduino community, cost, accessibility and open source. While an Enterprise or a Factory might not care so much to spend $10,000 - $15,000 on a development environment, they will start caring when they can easily find software developers that already know the hardware being used, in financial terms, the cost of training a new employee are far greater than their hardware.

While I did not bring any real world examples of using ESP32 for industrial purposes, I think I did bring a lot of information on how to do it, the future is inevitable, many software developers are not familiar with hardware but their knowledge and expertise in an professional software practices is mostly far greater than the average hardware programmer, but different requirements will make that previously clear boundary a thing of the past....

Tags: esp32, industry 4.0, iot