Startup flow refactoring
This set of commits changes the startup code in a way that lets the application choose if/when to initialize WiFi/BT.
Application entry point is now a more familiar `main()` function. This function is executed in its own task. Application may choose to do some initialization from main function, create some tasks and then return from `main`. Simple applications may choose to do all their work from `main`.
Additionally this MR splits event handling code into two parts.
- One part is a set of standard handlers for WiFi and DHCP events. Most applications will use this set of handlers, and it is made available via new `esp_event_process_default` function.
- Another part is the default implementation of event handling loop. Some applications may choose to use default event loop through `esp_event_loop_` set of APIs, which start an event handling task and call user-provided event callback from this task. Other applications may create an event queue and implement event loop themselves. In this case application has to provide `esp_event_send` function. In this case the implementation provided by `esp_event_loop_` module is unused.
esp-idf-template has been updated to match this set of changes: https://github.com/espressif/esp-idf-template/tree/feature/init_refactoring
BT example has also been updated.
We need to provide examples of both event handling approaches. This will be done in a separate follow-up MR.
See merge request !112
* master:
components/spi_flash: remove stray level of indentation
components/nvs: fix broken sentences in comment blocks
Roll back submodule version
Spinlocks already come initialized. Remove the code that would essentially re-initialize them at runtime
Remove all references to prvLockQueue / prvUnlockQueue
components/esp32: clean up unused function warnings in single core mode
clean up warnings
components/nvs: fix build, use log library instead of printf
components/spi_flash: add flash operation counters option to Kconfig
components/nvs: add erase function
components/nvs: fix formatting
components/nvs: batch writes when possible
components/spi_flash: add performance counters
components/nvs: maintain item hash list at page level
components/nvs: avoid reading just-erased page
# Conflicts:
# components/esp32/cpu_start.c
# components/esp32/event_default_handlers.c
nvs: speed improvement and erase APIs
NVS initialization speed issue raised in https://ezredmine.espressif.com/issues/6881 has been addressed for single core mode through optimization of `spi_flash_*` routines. This MR addresses same issue for dual core mode.
- An index table is added at Page level, it allows doing fast search of item index based on key name and namespace. This heavily reduces the number of flash reads.
- Write operations are batched when writing/erasing long variable-length values (str and blob). This gives a 33% reduction in the number of writes for a 64-byte value, 50% reduction for a 96-byte value.
- Add optional (configurable via menuconfig) perfomance counters for spi_flash APIs
With these changes, total init time for NVS goes down from 2 seconds to 140 ms.
Additional RAM usage is 128-640 bytes per page, depending on content. Typical RAM usage for current use case (storing wifi configuration parameters) is 256 bytes.
This change also exposes APIs to erase single key or the whole namespaces, needed for https://ezredmine.espressif.com/issues/6769.
See merge request !105
- put contents of a few headers into c++ guard blocks
- fix off-by-one error in do_global_ctors
- remove system_init from startup code (should be called from main)
This change separates definitions in esp_event.h and functions in event.c into several parts:
- event structure definitions (esp_event.h)
- default implementations of event handlers (event_default_handlers.c)
- default implementation of event loop (event_loop.c, esp_event_loop.h)
Purpose of this change is to allow applications choose their own poison:
- full control of event loop at the expense of more bootstrap code
- pre-defined event task firing event callbacks, but less code in app_main.c
For third party components (lwip and expat), compilation flags are adjusted to silence existing warnings (i have manually checked that all warnings are benign).
In components/esp32, replaced use of WIFI_DEBUG with ESP_LOG functions.
Only remaining warning is in FreeRTOS queue.c, and it may be a useful one.
This change removes implicit WiFi/BT initialization from startup code.
"main" task is started once essential part of startup code is complete.
This task calls application-provided "int main(void)" function, which can call WiFi/BT init functions if necessary.
For third party components (lwip and expat), compilation flags are adjusted to silence existing warnings (i have manually checked that all warnings are benign).
In components/esp32, replaced use of WIFI_DEBUG with ESP_LOG functions.
Only remaining warning is in FreeRTOS queue.c, and it may be a useful one.
Introduces new internal function, Page::alterEntryRangeState, which gathers changes to multiple elements of entry state table into a single write, provided that these changes fall into a single word. This allows changing state of up to 16 entries in a single write.
Also adds new function, writeEntryData, which writes the whole payload of SZ and BLOB type entries in one go, instead of splitting it into multiple 32-byte writes.
This reduces number of writes required for SZ and BLOB entries.
lwip: add debug code to show udp/tcp pcbs
Add code to show all tcp/udp pcbs, these kind of debug info is helpful for lwip issue debugging.
See merge request !98
* master: (57 commits)
components/lwip: fix grammar
components/lwip: make SO_REUSE configurable via menuconfig
bootloader: remove trailing newlines from log messages
components/freertos: override per-task __cleanup handler to close stdin, stdout, stderr
components/esp32: move peripheral symbols to separate ld script
components/log: regroup macros for better readability
gitlab-ci: allow running tests for branches, triggered via API
components/log: fix timestamp calculation
components/log: set default runtime log level to ESP_LOG_VERBOSE
components/log: fix error when using ESP_LOGx from C++ code
components/log: fix bugs, add options to override log level for files, components, and bootloader
fix ledc and spi typo
remove prefix and postfix
Enable SO_REUSEADDR in LWIP
freertos: fix memory debug macro issue Define configENABLE_MEMORY_DEBUG according to CONFIG_ENABLE_MEMORY_DEBUG
peripheral structure headers: move volatile keyword from members to typedef
Adding -fstrict-volatile-bitfields to the CFLAGS/CXXFLAGS. Without this, gcc tries to access bitfields using the smallest possible methods (eg l8i to grab an 8-bit field from a 32-bit). Our hardware does not like that. This flag tells gcc that if a bitfield is volatile, it should always use the type the field is defined at (uint32_t in our case) to size its access to the field. This fixes accessing the hardware through the xxx_struct.h headers.
add peripheral module struct headers
build system docs: Add note about no spaces in component names
Docs: Add note about unusual submodule messages when cloning on Windows
...
# Conflicts:
# components/esp32/cpu_start.c
# components/esp32/include/soc/cpu.h
Add logging library
Logging library, intended to be used across other components.
Uses design similar to Android NDK logging APIs and allows for both compile time and run time filters, similar to logcat.
Also includes cleanup of cpu_startup.c — i was changing some logging output in this file so decided to re-format it and reduce code duplication.
Ref TW6703
See merge request !68
make SO_REUSE configurable via menuconfig
SSC expects SO_REUSE to be off by default.
Trivial change, moving this option to menuconfig.
This should also (finally!) fix failing tests in master.
See merge request !99
* branch 'master':
components/freertos: override per-task __cleanup handler to close stdin, stdout, stderr
gitlab-ci: allow running tests for branches, triggered via API
allow running test suite for branches, triggered via API
One line change to run testing step for branches, if the build is triggered via API.
See merge request !91
override per-task __cleanup handler to close stdin, stdout, stderr
Default _cleanup_r function (called from _reclaim_reent) calls _fclose_r for all file descriptors related to a particular instance of struct _reent.
It does that by calling _fwalk_reent, which iterates over __sglue list in struct _reent and calls _fclose_r for each member. But _stdin, _stdout, and _stderr are not in this list, so _fclose_r is not called for them. Which leads to a memory leak.
The easy way to fix this is to reset __cleanup member of struct _reent to our new “patched” version, which first calls the original version (_cleanup_r) and then does _fclose_r for each of the stdin, stdout, and stderr.
See merge request !94
freertos: fix memory debug macro issue
When enable memory debug via make menuconfig, we got compile error.
Define configENABLE_MEMORY_DEBUG according to CONFIG_ENABLE_MEMORY_DEBUG
See merge request !89
fix deploy to GitHub
Previous change to gitlab-ci.yml moved setting of Gitlab ssh key to global before_script.
This together with the fact that we used `echo >>` instead of `echo >` to write to `id_rsa_base64` file caused wrong key to be used while pushing to GitHub.
Therefore:
- don't run default before_script before push_master_to_github job
- replace echo >> with echo > to avoid mistakes in the future
See merge request !82
Move CPU region protection setup into soc/cpu.h
change tabs to spaces
remove unused extern declarations
use RTC_WDTCONFIG0 instead of numeric address (still need to fix BB reg)
This also removes logging implementation from bootloader and replaces it
with the one provided by the log component. Some occurrences of printf
and ets_printf have been changed to ESP_LOGx APIs.
Note that with WiFi stack enabled, system_init will reset frequency to 240MHz.
To make this setting useful, esp32-wifi-libs submodule needs to be updated.
* branch master: (51 commits)
fix typos
callback: add a void* pointer with each callback
header: callbacks use "void *" as arguments
header: format and fix typos
add wifi api comments
add wifi api comments
components/spi_flash: call SPIUnlock only once
components/spi_flash: improve comments and readability
esp32 cpu_start: Include rom/uart.h, remove inline ROM function declarations
bootloader: Add bootloader Kconfig
Fix bug where nesting vTaskEnterCritical calls would not re-enable interrupts after vTaskExitCritical sections.
installed gitlab key in global before_script, don't need to overwrite in template before_script
ci: Use github repo for wifi libraries on master branch, gitlab repo otherwise
components/esp32,spi_flash: update some comments
components/esp32: remove dependency of WIFI_ENABLED on FREERTOS_UNICORE
esp32/lib: Bump WiFi revision
update ci: build ssc with idf submodule wifi lib
components/esp32: add inter-processor call API and implement spi_flash through it
add MACRO to get flash pad number from efuse value
add comments to register read/write operations
...
components/spi_flash: call SPIUnlock once
This fixes the performance impact for spi_flash_write and spi_flash_erase.
With this change, NVS init in single core mode takes about 50ms (compared to >2seconds before that).
For dual core mode, we still spend on average 2ms for each spi_flash_ operation due to synchronization between CPUs, so NVS startup time is about 2 seconds in dual core mode.
See merge request !80
This fixes the performance impact for spi_flash_write and spi_flash_erase.
With this change, NVS init in single core mode takes about 50ms (compared to >2seconds before that).
Dual core compatible spi flash APIs
Implementation of spi flash APIs which work in dual core mode.
This also includes other changes:
- fixes to FreeRTOS tasks.c made by Jeroen and me — these were originally made to fix issues with high priority tasks seen in WiFi stack, but were required to get stuff in ipc.c working as well
- initial base idea with two high-priority tasks was separated into a separate API — esp_ipc.h. That API allows one to run a function on the other CPU from a high-priority task. Because we don't have preemption across CPUs, this is currently not very fast: it requires on average half a tick until the second CPU switches to the high priority task. Currently that API can be either blocking, or non-blocking with queue length of 1. I haven't yet found a use case for arbitrary sized queue of IPC requests. Anyone who needs that can trivially add that later by replacing a binary semaphore with a generic queue.
With these changes i can run application with WiFi stack and NVS on both cores.
See merge request !25
update ci config file with 2 changes:
1. add build SSC and do sanity test config
2. add tags to each job as sanity test need to be executed on special runner
Test is now only arranged for master branch
See merge request !69
* master: (130 commits)
lwip: Define LWIP_ESP8266 in port lwipopts.h not gcc command line
CI: Build the esp-idf-template with the matching branch name, if it exists
README: Add Resources section with some links
Rename README.buildenv to docs/build_system.rst and ReST-ify it
Eclipse docs: Easier to just replace entire PATH, msys32 has everything we need to build/flash
test_build_system: Print ESP_IDF_TEMPLATE_GIT for easier debugging
Name component makefiles component.mk instead of Makefile
Eclipse doc: Add troubleshooting note about Makefile directories
eclipse_make.sh: Fix printing of make directory
Move bin/eclipse_windows_make.sh to tools/windows_eclipse_make.sh
Eclipse docs: Prepend IDF paths to beginning of PATH
Set default SPI flash access mode to DIO
FreeRTOS: temporary solution for memory canaries and memory debug
tcpip_adapter: fix dhcp client work flow
event: not post got ip event if static is invalid
tcpip_adapter: typedef clean up
event: post got ip event when use static ip
tcpip_adapter: use dhcp callback to post got ip event
dhcp: add dhcp callback
lwip: remove netif_reg_addr_change_cb
...
Gitlab CI will cache build directory per branch and per build step. Also it doesn't try to update submodules.
We have to do this manually because otherwise we end up with old version of submodule in cached directory.
gitlab-ci: update submodules before build
Gitlab CI will cache build directory per branch and per build step. Also it doesn't try to update submodules.
We have to do this manually because otherwise we end up with old version of submodule in cached directory.
See merge request !47
Gitlab CI will cache build directory per branch and per build step. Also it doesn't try to update submodules.
We have to do this manually because otherwise we end up with old version of submodule in cached directory.
[Github] Fix prototype for ets_aes_set_endian
https://github.com/espressif/esp-idf/pull/3
Original description:
>
> found small mistake in it
>
> we have:
>
```
PROVIDE ( ets_aes_crypt = 0x4005c9b8 );
PROVIDE ( ets_aes_disable = 0x4005c8f8 );
PROVIDE ( ets_aes_enable = 0x4005c8cc );
PROVIDE ( ets_aes_set_endian = 0x4005c928 );
PROVIDE ( ets_aes_setkey_dec = 0x4005c994 );
PROVIDE ( ets_aes_setkey_enc = 0x4005c97c );
```
See merge request !29
Fix dualcore startup
This MR includes:
- a workaround for a hardware bug with cache initialization
- fix of `#if CONFIG_WIFI_ENABLED` block in cpu_start.c for the case when WiFi is disabled.
- removal of miscellaneous delays in cpu_start.c
See merge request !22
fix DRAM_ATTR attribute
Variables marked with DRAM_ATTR were not placed in any specific section and were appended to the last section, causing some errors at binary file generation stage.
This change places contents of .dram1 section into .dram0.data.
Also removes .iram1pro/.iram1app/.dram1pro/.dram1app which weren't used anywhere.
See merge request !23
Make: fix incremental builds, add build tests
Four semi-related build changes in one:
* Add basic tests for make system sanity (can be expanded as we find corner cases)
* Fix incremental building of bootloader when source files change
* Don't regenerate component libraries or re-link the ELF file if nothing changed
* Rename `$(vecho)` to `$(summary)` and add a new `$(details)` macro to echo some helpful build details when V=1.
See merge request !20
This is a workaround for a hardware bug with cache initialization.
Only two lines of code related to DPORT_APP_CACHE_MMU_IA_CLR were added around mmu_init(1); call,
and two lines at the end of comment block.
I reformatted surrounding lines to use spaces for indentation, like the rest of the code in this file does.
This change removes various delays in PRO and APP CPU startup paths.
Also some unused code and ets_printf statements are cleaned up or removed.
2nd CPU now communicates that it has started through a global variable.
Fix for noninteractive builds
This MR fixes the case when new Kconfig configuration options were added
and you run `make menuconfig`. Previously this would first show console
prompt to set values for new options, and then launch menuconfig.
Now this will just launch menuconfig. New items are marked as `(NEW)`.
New target is added, defconfig, which may be useful for non-interactive
environments to set default values for all new configuration options.
Gitlab CI uses `make defconfig` to update sdkconfig in case any new
options were added in ESP-IDF.
See merge request !15
Currently this is mostly a verbatim copy of macOS guide with some obvious changes.
Exact lists of dependencies for Debian and Arch need to be specified.
Also made minor fixes to Windows and macOS guides:
- changed http urls to https
- added --prefix=$PWD to crosstool-NG's configure
- added chmod u+w for the build output directory
In general, except for prerequisites, Linux and macOS guides are virtually identical.
It might make sense to refactor this into single document with prerequisites being
described in separate docs.
This commit fixes the case when new Kconfig configuration options were added and you run `make menuconfig`.
Previously this would first show console prompt to set values for new options, and then launch menuconfig.
Now this will just launch menuconfig.
New target is added, defconfig, which may be useful for non-interactive environments to set default values for
all new configuration options.
Ivan Grokhotkov