View Dror Gluska's profileI've started doing electronics and other technical stuff since I was very little but I've left this hobby in the late 90's. Since then it all went under the radar for me, I would still buy the occasional Elektor magazine or browse the kits section in an electronics shop but it lost my serious attention.
About 10 years ago I've wanted to try building something and so bought the immortal Hakko 936 and a Power Supply from CircuitSpecialists and later made an order of a few transistors and ICs from DigiKey but that project did not really work as I forgot too much and needed to learn too much but without the proper books and it caused me to lose interest again.
At 2012 I've had another idea how to build an all budget house automation system and so found myself browsing AliExpress and it rekindled my love for soldering smells and building things.
So why am I telling you this?
Around that time the Arduino ecosystem started flourishing and everyone could start doing microprocessor work easily and at home and although my first microprocessor was a Microchip PIC, it was around 2005, the IDE sucked, you could do either assembler only or limited C code, I've lost interest pretty fast and I bought about 3 processors for about $25...
 |
| Attractive eh? |
So again, why am I telling you this?
Like Steve Ballmer immortal words "Developers Developers Developers!", a good development environment is one of the primary reasons a product can get popular.
While Arduino IDE is a perfect for beginners, you quickly reach the limit of what you can do without a modern IDE, such as Visual Studio, Eclipse or any real code parsing, autocompletion, quickinfo IDE, the fun part is discovering that Atmel AVR (the processor many Arduinos are based on) already has an IDE, Atmel Studio.
Its even better to find out someone has already made the Arduino IDE compatible with Visual Studio IDE, which began to be my favorite since Visual Studio 2013 when the C++ parser was improved tremendously, that plugin is Visual Micro.
 |
| http://www.visualmicro.com/forums/YaBB.pl?num=1462975225/10 |
But lately I've found myself limited with what I can do in Visual Micro. I've started programming the ESP32, which is my favorite microprocessor at the moment. The more I learn about it, the more I realize this is an enterprise grade, industry 4.0, IIoT, whichever buzzword you want to stick on it, it can probably do it with its dual core, 520kb memory, 4MB flash, Wifi, BT, 18 channel ADC, 4SPI, 2 I2C, 3 UARTs CAN bus, IR, PWM, sleep modes, encryption and cryptographic acceleration. A thing of beauty for $5 module or $8 development board with USB Programmer.
So why am I telling you this?
I've recently decided to take another look at PlatformIO, while their github started at about 2014, I'm only taking interest now since it can probably do what I want it to do, give me access to esp-idf with a decent editor and make the noise go away until I need it (build environment).
I'll be blunt, the IDE is still rough around the edges, Atom is not as fun as Visual Studio, VS Code does not have a C++ parser as good as Visual Studio's and overall development experience is about 7 out of 10 but that thing does its magic and enables me to explore the entire FreeRTOS and esp-idf framework so I'll bear the quirks.
In any case, I've installed the plugin about a week ago, since then I've patched and did a pull request for allowing custom partitions and I've even found a way to measure task CPU usage in FreeRTOS which eluded me in-spite of all my efforts in Visual Micro/Arduino.
Do I recommend it?
Only if you've surpassed the capabilities Visual Micro/Arduino gave you since there's still no replacement for Visual Studio C++ parser in my opinion. and Yes, I did try Eclipse some time ago but was not satisfied with the results so I've abandoned it.
But wait, there's good news, I've found a way to use Visual Studio with PlatformIO. At first I saw issue 543, it did not work so well for me, so I combined it with my own solution and it worked, I now have a project that works with PlatformIO! :-)
PlatformIO Use
Lets continue working on an example of what I could do with PlatformIO that I couldn't do with Visual Micro/Arduino, first lets discuss how projects are built in PlatformIO as far as I could tell in this early stage.With PlatformIO the do-it-all command is... you guessed it - platformio (or platformio.exe in windows) or pio.
In our case, for esp32 with esp-idf:
platformio init --board esp32dev --project-option "framework=espidf"
Now the project is ready for development, you can open it with VSCode or Atom and start working.
But there are two enhancements we can do right now, first we should edit platformio.ini and add the following line in the end of env:esp32dev so the "platformio device monitor" will not crap out:
monitor_baud = 115200
Then we can create our Visual Studio project (not vscode!):
platformio init --ide visualstudio
But we're not done yet, Visual Studio intellisense is designed to work with Microsoft Visual C++, we'll need to add a file and set it in Project Properties -> NMake -> Forced Includes
At first I've attempted to use the solution in Issue 543, but it did not deliver, so I've modified it to this but its a work in progress, so most of the intellisense is working but there are still some quirks.
#define _ALLOW_KEYWORD_MACROS
#ifndef __GNUC__
#define __GNUC__ 2
#endif
#ifndef __STDC__
#define __STDC__
#endif
#ifdef _WIN32
#define __attribute__(A) /* do nothing */
#endif
#ifdef _MSC_VER#define __asm__(x)
#define __extension__(x)
#define __attribute__(x)
#define __builtin_va_list int
#define __extension__
#define __inline__
#define __builtin_constant_p
#define _Bool bool
typedef int __INTPTR_TYPE__ ;
typedef unsigned int __UINTPTR_TYPE__;
#endif
So now that we have a good environment, we can start working.
Custom Partition Table
One of the first features I wanted to try and implement with PlatformIO which I couldn't with Visual Micro is a custom partition table. We add a new partitions_table.csv and follow the instructions.You will also need to modify sdkconfig.h and update these defines:
#define CONFIG_PARTITION_TABLE_FILENAME "coredump_partitions.csv"
#define CONFIG_PARTITION_TABLE_CUSTOM_FILENAME "coredump_partitions.csv"
In this case I wanted to try a partition table with ota, a fat filesystem and a coredump.
# Name, Type, SubType, Offset, Size
# Note: if you change the phy_init or app partition offset, make sure to change the offset in Kconfig.projbuild
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x140000,
app1, app, ota_1, 0x150000,0x140000,
eeprom, data, 0x99, 0x290000,0x1000,
fat, data, fat, 0x291000,0x15F000,
coredump, data, coredump,0x3F0000,0x10000
I've wrote a small snippet of code to display the partition table once the esp boots up and now we can upload the code to esp32:
platformio run -t upload
If we're successful, we'll see:
Leaving...
Hard resetting...
==== [SUCCESS] Took 20.79 seconds ====
Now we can monitor esp32:
platformio device monitor
And see the results:
- label type/subtype address:size encrypted
- app0 0x00/0x10 0x00010000:0x00140000 0
- app1 0x00/0x11 0x00150000:0x00140000 0
- nvs 0x01/0x02 0x00009000:0x00005000 0
- otadata 0x01/0x00 0x0000e000:0x00002000 0
- eeprom 0x01/0x99 0x00290000:0x00001000 0
- fat 0x01/0x81 0x00291000:0x0015f000 0
- coredump 0x01/0x03 0x003f0000:0x00010000 0
Great! First success with the esp-idf and PlatformIO!
You can find the sources for this POC here:
https://github.com/drorgl/esp32-poc
Core Dumps
One of the features that an enterprise grade product needs is to be able to analyze crashes, not just the ones that a developer is able to attach a debugger to or the ones that can be reproduced but the ones that seldom happen, the ones that happen only in production, the hard ones.To be able to do that, Alexey Gerenkov implemented saving the core dumps to a special flash partition, but I wanted to do something a bit different, I wanted to send the core dumps back to my server for monitoring, so whenever the device would start it will go to the core dump partition, check if it not empty, then be able to send it to the server, save it to the SPI fat file system or anything else that might be needed.
Documentation:
https://github.com/espressif/esp-idf/blob/master/docs/api-guides/core_dump.rst
I had a good starting point for a crash, I've used this code.
You need to select a partition table with a core dump partition, this is where the previous POC came into play as the PlatformIO esp-idf framework can only use partitions_singleapp.csv as far as I can tell without my modification.
I then started to look for a way to retrieve the core dump partition, you will need to use
esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_COREDUMP, NULL);
I've then used esp_partition_mmap to easily access the partition and check its length.
I think it might be better to check for the start and end markers instead of a "not empty" (0xFF) characte, but its good enough for the POC.
COREDUMP_FLASH_MAGIC_START
COREDUMP_FLASH_MAGIC_END
So now we have the start address and length of the core dump, we can do whatever we want with it, dump it to screen as base64, there's also a built in function that does that on crash, but you need to configure it:
#define CONFIG_ESP32_ENABLE_COREDUMP 1
#define CONFIG_ESP32_ENABLE_COREDUMP_TO_NONE 0
#define CONFIG_ESP32_ENABLE_COREDUMP_TO_FLASH 1
#define CONFIG_ESP32_ENABLE_COREDUMP_TO_UART 1
Again, this is not what I wanted as there will be no one connected to the device to grab that core dump from the console.
Eventually we can assume we have a core dump, make sure you're uploading the core dump along with the firmware version it happened in otherwise you won't know where it crashed and you'll get the wrong results.
We can now run the command to get the core dump in text form:
espcoredump.py info_corefile -t b64 -c test_crash.txt -g toolchain-xtensa32\bin\xtensa-esp32-elf-gdb.exe ".pioenvs\esp32dev\firmware.elf"
And the result:
===============================================================
==================== ESP32 CORE DUMP START ====================
================== CURRENT THREAD REGISTERS ===================
pc 0x400d0901 0x400d0901 <recur_func()+73>
lbeg 0x400014fd 1073747197
lend 0x4000150d 1073747213
lcount 0xffffffff 4294967295
sar 0x0 0
ps 0x60f20 397088
threadptr <unavailable>
br <unavailable>
scompare1 <unavailable>
acclo <unavailable>
acchi <unavailable>
m0 <unavailable>
m1 <unavailable>
m2 <unavailable>
m3 <unavailable>
expstate <unavailable>
f64r_lo <unavailable>
f64r_hi <unavailable>
f64s <unavailable>
fcr <unavailable>
fsr <unavailable>
a0 0x400d08e0 1074596064
a1 0x3ffc7f20 1073512224
a2 0x2 2
a3 0x3f403578 1061172600
a4 0x3ffc7f60 1073512288
a5 0x3ffc271c 1073489692
a6 0x0 0
a7 0x0 0
a8 0x5 5
a9 0xffffffad -83
a10 0x20 32
a11 0x3ffc656c 1073505644
a12 0x1 1
a13 0x80 128
a14 0x1 1
a15 0x0 0
==================== CURRENT THREAD STACK =====================
#0 0x400d0901 in recur_func () at src\ est_core_dump.cpp:80
#1 0x400d08e0 in recur_func () at src\ est_core_dump.cpp:73
#2 0x400d08e0 in recur_func () at src\ est_core_dump.cpp:73
#3 0x400d0924 in unaligned_ptr_task (pvParameter=0x0) at src\ est_core_dump.cpp:90
======================== THREADS INFO =========================
Id Target Id Frame
9 process 8 xQueueGenericReceive (xQueue=0x3ffc2ecc, pvBuffer=0x0, xTicksToWait=4294967295, xJustPeeking=0) at .platformio\\packages\\framework-espidf\\components\\freertos\\queue.c:1452
8 process 7 xQueueGenericReceive (xQueue=0x3ffc2904, pvBuffer=0x0, xTicksToWait=4294967295, xJustPeeking=0) at .platformio\\packages\\framework-espidf\\components\\freertos\\queue.c:1452
7 process 6 prvTimerTask (pvParameters=0x0) at .platformio\\packages\\framework-espidf\\components\\freertos\ imers.c:445
6 process 5 failed_assert_task (pvParameter=0x0) at src\ est_core_dump.cpp:103
5 process 4 0x400d0c12 in init_analysis_task (pvParameter=<optimized out>) at src\ est_core_dump.cpp:130
4 process 3 bad_ptr_task (pvParameter=0x0) at src\ est_core_dump.cpp:51
3 process 2 0x400d1838 in esp_vApplicationIdleHook () at .platformio\\packages\\framework-espidf\\components\\esp32\\freertos_hooks.c:52
2 process 1 0x400d1838 in esp_vApplicationIdleHook () at .platformio\\packages\\framework-espidf\\components\\esp32\\freertos_hooks.c:52
* 1 <main task> 0x400d0901 in recur_func () at src\ est_core_dump.cpp:80
======================= ALL MEMORY REGIONS ========================
Name Address Size Attrs
.rtc.text 0x400c0000 0x0 RW
.iram0.vectors 0x40080000 0x400 R XA
.iram0.text 0x40080400 0x7860 R XA
.dram0.data 0x3ffc0000 0x1d20 RW A
.flash.rodata 0x3f400010 0x3e64 RW A
.flash.text 0x400d0018 0xed20 R XA
.coredump.tasks 0x3ffc6504 0x164 RW
.coredump.tasks 0x3ffc7e60 0x1d4 RW
.coredump.tasks 0x3ffc559c 0x164 RW
.coredump.tasks 0x3ffc5430 0x160 RW
.coredump.tasks 0x3ffc5028 0x164 RW
.coredump.tasks 0x3ffc4eb0 0x16c RW
.coredump.tasks 0x3ffc6398 0x164 RW
.coredump.tasks 0x3ffc76e0 0x14c RW
.coredump.tasks 0x3ffc622c 0x164 RW
.coredump.tasks 0x3ffc6ed0 0x154 RW
.coredump.tasks 0x3ffc8848 0x164 RW
.coredump.tasks 0x3ffc86f0 0x14c RW
.coredump.tasks 0x3ffc6004 0x164 RW
.coredump.tasks 0x3ffc5ea0 0x158 RW
.coredump.tasks 0x3ffc2d60 0x164 RW
.coredump.tasks 0x3ffc2bd0 0x184 RW
.coredump.tasks 0x3ffc3328 0x164 RW
.coredump.tasks 0x3ffc3190 0x18c RW
===================== ESP32 CORE DUMP END =====================
===============================================================
CPU Utilization
This is a bit of a hack, since I would rather not change esp-idf code without commiting the change back to the main repo, I've tried many workarounds, eventually the last nail in the coffin was the fact there's a definition for portGET_RUN_TIME_COUNTER_VALUE in freertos/portmacro.h, making it impossible to override from outside the framework.So.... I had to modify portmacro.h, a change I plan to ask to push back into esp-idf:
#ifndef portGET_RUN_TIME_COUNTER_VALUE
#define portGET_RUN_TIME_COUNTER_VALUE() xthal_get_ccount()
#endif
Then I needed to add the functions for initializing the counter and getting the counter values in sdkconfig.h:
#ifndef __ASSEMBLER__
#ifdef __cplusplus
extern "C"{
#endif
#ifdef ESP32
extern unsigned long get_run_time_counter_value();
extern void init_run_time_counter();
#endif
#ifdef __cplusplus
}
#endif
#define portGET_RUN_TIME_COUNTER_VALUE() get_run_time_counter_value()
#define portCONFIGURE_TIMER_FOR_RUN_TIME_STATS() init_run_time_counter()
#endif
#define configGENERATE_RUN_TIME_STATS 1
#define configUSE_STATS_FORMATTING_FUNCTIONS 1
#define configUSE_TRACE_FACILITY 1
I then went ahead and used system_get_time to get the value for the counter as it was the only stable one for now, the others are core dependent so every time a task would switch core due to context switching, the counter value will be different showing skewed results.
I know system_get_time is deprecated but I've yet to find a better function with stable results that won't crash, I've tried gettimeofday but it crashed the device.
In any case, this should not be used in production software according to FreeRTOS.
After running for a few cycles, we get these results:
Tasks currently running:
Printing Stats:
high_load1 502167712 49
medium_load2 15551453 1
display_stats 235900 <1
medium_load1 18987019 1
low_load2 384767 <1
low_load1 422990 <1
IDLE 34476 <1
IDLE 977132 <1
high_load2 505107417 50
Tmr Svc 60 <1
ipc0 340010 <1
ipc1 22441 <1
iterations low: 70 medium: 3156 high: 100096
What I would like you to notice is the amount of iterations each function executed, bear in mind this is an RTOS, so higher priority tasks run first, if the high and medium tasks are not in a wait section such as vTaskDelay, the low priority won't get any time, taskYIELD might not be enough here so make sure you don't have high priority tasks that never release CPU time.
Summary
This was a short demonstration of a few things the ESP32 can do in terms of enterprise grade devices, the more I read esp-idf code, the more I realize its potential.I think this development board is one of the most fun ones I played with, in the upcoming weeks I'll see if its possible to apply enterprise grade architecture and design with it, I've already implemented a lean dependency injection for it and did a POC controlling it with Azure IoT Hub.
You must have seen that:
In my opinion ESP32 + FreeRTOS + Azure IoT Hub + PlatformIO is overlooked and has the potential of realizing many Industry 4.0 projects.
To understand the potential, this is a small section from Microsoft Build 2017 First day Keynotes, you will notice what analyzing seemingly naive data can give you in the long run a change in power consumption, vibrations, temperatures can all point to an imminent failure, predicting maintenance, short product life and much much more.
I think the next best step will be to learn about PlatformIO unit testing capabilities.
I'm looking forward to see what espressif will think of next, just imagine the PC we had about 20 years ago is now the size of a coin and can be programmed by anyone.
0 comments:
Post a Comment