Docs: Document flash encryption disable steps, clean up flash encryption & secure boot docs
Generate clean up of flash encryption & secure boot docs
Including steps for disabling flash encryption (for people who accidentally enable it).
See merge request !500
phy_init: don’t rewrite valid calibration data
In the default PHY init routine, calibration data is loaded from NVS.
Most of the time the incremental changes to calibration will be fairly
small, so we don’t need to rewrite the existing calibration data stored
in the NVS.
Possible enhancement to be done in the future: expose a function in PHY
library to tell how big was the change in calibration data. If the
change was significant, then calibration data stored in NVS should be
updated.
See merge request !487
freertos: fix protection issue in freertos queue event list
When functions in queue.c calls listLIST_IS_EMPTY() to check queue event list, the queue list is
protected by queue mutex, on the other hand, when xTaskIncrementTick() modify the queue list, the
queue list is protected by xTaskQueueMutex, this may cause xTaskRemoveFromEventList operate on
the empty queue list and cause problem.
This change works around the problem by reducing the window where the race condition can happen.
See merge request !465
CI: get test env config from gitlab
Previous design was put test env config on local runners. It's not easy to manage as test runners count growing.
Now we'll put config files for test runners to a Gitlab repository. Test runners will get its config from Gitlab every time before running.
See merge request !480
In the default PHY init routine, calibration data is loaded from NVS.
Most of the time the incremental changes to calibration will be fairly
small, so we don’t need to rewrite the existing calibration data stored
in the NVS.
Possible enhancement to be done in the future: expose a function in PHY
library to tell how big was the change in calibration data. If the
change was significant, then calibration data stored in NVS should be
updated.
esp32: Fixes watchdog problem when printing core dump to uart
1) Fixes watchdog problem when printing core dump to uart
2) Also fixes generation of core dumps when flash cache is disabled
See merge request !472
CI: add job to check build esp-at in esp-idf
Add a job to check if latest IDF breaks `esp-at` build.
As `esp-at` is an application and has its own CI, we don't save binary as artifacts.
See merge request !462
ci: deploy code and docs for tags
Currently deploy happens only for the master branch.
This change is needed to support tags in release branches.
See merge request !458
docs: use custom roles to generate GitHub links
This change replaces direct links to GitHub master branch with
auto-generated links using docutils custom roles.
These auto-generated links point to the tree or blob for the git commit
ID (or tag) of the repository. This is needed to ensure that links don’t
become broken when files in master branch are moved around or deleted.
The following roles are introduced:
```
- :idf:`path` - points to directory inside ESP-IDF
- :idf_blob:`path` - points to file inside ESP-IDF
- :idf_raw:`path` - points to raw view of the file inside ESP-IDF
- :component:`path` - points to directory inside ESP-IDF components dir
- :component_blob:`path` - points to file inside ESP-IDF components dir
- :component_raw:`path` - points to raw view of the file inside ESP-IDF
components dir
- :example:`path` - points to directory inside ESP-IDF examples dir
- :example_blob:`path` - points to file inside ESP-IDF examples dir
- :example_raw:`path` - points to raw view of the file inside ESP-IDF
examples dir
```
A check is added to the CI build script, which searches RST files for
presence of hard-coded links (identified by tree/master, blob/master,
or raw/master part of the URL).
This check can be run manually: cd docs && make gh-linkcheck
Additionally, Sphinx linkcheck build type is used to create new CI test,
which check for broken links. This test has to be triggered explicitly,
because including it in normal build process (when the commit is not yet
deployed to Github) will not work. It can be triggered in a regular
fashion using a combination of cron and Curl, similar to stress tests.
See merge request !455
This change replaces direct links to GitHub master branch with
auto-generated links using docutils custom roles.
These auto-generated links point to the tree or blob for the git commit
ID (or tag) of the repository. This is needed to ensure that links don’t
become broken when files in master branch are moved around or deleted.
The following roles are introduced:
- :idf:`path` - points to directory inside ESP-IDF
- :idf_blob:`path` - points to file inside ESP-IDF
- :idf_raw:`path` - points to raw view of the file inside ESP-IDF
- :component:`path` - points to directory inside ESP-IDF components dir
- :component_blob:`path` - points to file inside ESP-IDF components dir
- :component_raw:`path` - points to raw view of the file inside ESP-IDF
components dir
- :example:`path` - points to directory inside ESP-IDF examples dir
- :example_blob:`path` - points to file inside ESP-IDF examples dir
- :example_raw:`path` - points to raw view of the file inside ESP-IDF
examples dir
A check is added to the CI build script, which searches RST files for
presence of hard-coded links (identified by tree/master, blob/master,
or raw/master part of the URL).
This check can be run manually: cd docs && make gh-linkcheck
Additionally, Sphinx linkcheck build type is used to create new CI test,
which check for broken links. This test has to be triggered explicitly,
because including it in normal build process (when the commit is not yet
deployed to Github) will not work. It can be triggered in a regular
fashion using a combination of cron and Curl, similar to stress tests.
CI: fix bug that test result always fail
CI test report jobs always fails because of 2 bugs:
1. merge/rebase error: the variable used to log test result is modified during merge/rebase
2. path of unit test module definition file is not correct
See merge request !454
Move PHY options out of WiFi config, improve descriptions
- move PHY-related settings into new menu, make it dependent on WIFI_ENABLED || BT_ENABLED
- improve descriptions of Ethernet Kconfig options
See merge request !443
ESP32_PHY_MAX_TX_POWER option is only meaningful for WiFi, so previous
change made it depend on WIFI_ENABLED. However if WiFi is not enabled,
but BT is, this option becomes undefined which breaks phy_init_data
generation.
This change turns ESP32_PHY_MAX_TX_POWER into a hidden parameter, which
depends on PHY_ENABLED. New user-visible parameter,
ESP32_PHY_MAX_WIFI_TX_POWER is introduced which depends on WIFI_ENABLED
and is used as default value for ESP32_PHY_MAX_TX_POWER if WIFI_ENABLED
is set. Otherwise ESP32_PHY_MAX_WIFI_TX_POWER is set to 20.
Readme and links to example category folders
- Added README.md to example category folders, for BT ref. https://github.com/espressif/esp-idf/issues/262
- Added links from api category TOC to example category folders
See merge request !450
newlib: fix register used for DPORT/RTC bug workaround
While there was no register at DR_REG_FRC_TIMER_BASE + 0x60, due to
peripheral address space wraparound this write actually affected one of
FRC2 registers, which is used by WiFi stack to implement legacy
ets_timer APIs.
This change uses FRC_TIMER_LOAD_REG(0) instead, which can be set to
known value safely.
See merge request !449
fixes for issues observed when using spi_flash
This MR fixes three unrelated issues:
- Race condition in spi_flash_enable_interrupts_caches_and_other_cpu
when operations on unpinned tasks are performed.
The issue is reported in https://github.com/espressif/esp-idf/pull/258
- esp_intr_noniram_disable doesn’t disable interrupts when compiled in
release mode. This issue manifested itself with an illegal instruction
exception when task WDT ISR was called at the time when flash was
disabled.
Fixes https://github.com/espressif/esp-idf/issues/263.
- Tick hooks on CPU1 were not called if CPU0 scheduler was disabled for
significant amount of time (which could happen when doing flash erase).
The issue manifested itself as “INT WDT timeout on core 1” error.
Fixes https://github.com/espressif/esp-idf/issues/219.
See merge request !441
While there was no register at DR_REG_FRC_TIMER_BASE + 0x60, due to
peripheral address space wraparound this write actually affected one of
FRC2 registers, which is used by WiFi stack to implement legacy
ets_timer APIs.
This change uses FRC_TIMER_LOAD_REG(0) instead, which can be set to
known value safely.
Organize examples and API documentation - supplement to !430
This supplement to !430, both intended to replace bulky !421.
1. Fixed broken links
2. Added links to available examples in API documentation, where missing
3. Updated links to point directly to example folder (to show README.md with example description)
4. Added README.md in docs folder info to point reader to ReadTheDocs instead of doc sources, ref. https://github.com/espressif/esp-idf/issues/243
5. Fixed some typo errors / Sphinx warnings
6. Fixed memory leak in example, ref. https://github.com/espressif/esp-idf/issues/209
See merge request !440
The block which dispatches ticks on CPU1 was a copy of the code block
for the normal path (CPU0). It used to check uxPendedTicks, with the
logic that uxPendedTicks can be 0 iff the scheduler is not suspended.
On CPU1 however, uxPendedTicks is not related to the state of the
scheduler (as uxPendedTicks is updated on CPU0). Due to this, if CPU0
scheduler was suspended, and uxPendedTicks happened to be nonzero,
tick hooks on CPU1 didn’t run, even though CPU1 scheduler was working.
This change removes the check for uxPendedTicks in CPU1 code path,
so that the tick hooks on CPU1 always get called (as for the CPU0 code
path).
Allow writes to encrypted partitions
There is a size alignment requirement but it is checked by
spi_flash_write_encrypted. However, this check flat-out bans encrypted
writes.
Original PR on Github: https://github.com/espressif/esp-idf/pull/249
See merge request !432
When compiling in release mode, compiler was choosing same register for
oldint and intmask variables, so INTENABLE was never modified.
This effectively broke disabling of non-IRAM interrupts during flash
operations, observed in the existing tests if task watchdog is enabled.
This change adds an extra constraint tells the compiler that output
operand should not be placed into the same register as an input one.
Parse unit test cases from test files
As of now, we need to sync three different places when adding new unit test cases. The plan is to add a python script in tools/unit-test-app. This script would parse unit test cases from the C files with tests and generate the needed YAML files in components/idf_test/unit_test. CI would run this script in build stage to create the files and unit-test stage would use the generated files to run tests.
To get the needed test metadata, we need to add some more tags in the test case functions. For example:
```
TEST_CASE("SYS_LIB_0102 test time functions", "[stdlib][fails][env=UT_T1_APC]")
{
//...
}
```
Reason for this kind of implementation is that we need the YAML files for generating documents.
See merge request !275
spi_flash_enable_interrupts_caches_and_other_cpu function used to enable
non-IRAM interrupts after giving up flash operation lock, which would
cause problems if another task was waiting on the lock to start a flash
operation. In fact, non-IRAM interrupts should be re-enabled before the
task scheduler is resumed. Otherwise non-pinned task can be moved to the
other CPU due to preemption, causing esp_intr_noniram_enable to be
called on the other CPU, causing an abort to be triggered.
Fixes the issue reported in
https://github.com/espressif/esp-idf/pull/258
Delete option to select core timer 2 as a FreeRTOS tick source
Core timer 2 is unusable for FreeRTOS ticks because it triggers a high-level interrupt. This MR deletes the option to select it.
Ref https://github.com/espressif/esp-idf/issues/234
See merge request !418
In some cases (when RTC register reads are performed from the APP CPU), a write to FRC_TIMER_INT_REG may be lost on the bus.
Writing to another DPORT register immediately before or after that works around the issue.
We write one dummy value to an address which doesn’t have any register associated with it.
Fixes https://github.com/espressif/arduino-esp32/issues/120
One common pattern of using assert function looks as follows:
int ret = do_foo();
assert(ret == 0); // which reads as: “do_foo should never fail here, by design”
The problem with such code is that if ‘assert’ is removed by the preprocessor in release build,
variable ret is no longer used, and the compiler issues a warning about this.
Changing assert definition in the way done here make the variable used, from language syntax perspective.
Semantically, the variable is still unused at run time (as sizeof can be evaluated at compile time), so the compiler
can optimize things away if possible.
These functions are marked as inline and are called from functions which are in IRAM.
In release (-Os) builds, the compiler may decide not to inline these functions.
Placing these functions into IRAM explicitly works around this.
vfs: implement reading from UART
This change adds support for reading from the UART FIFO via the `stdin`.
Useful for simple cases when handling lots of data is not a requirement (e.g. command prompts operated by humans).
`\r\n` -> `\n` conversion is guided by the existing `CONFIG_NEWLIB_STDOUT_ADDCR` configuration option.
See merge request !393
bootloader_random_disable: Restore all SARADC/I2S registers to defaults
Fix for issue with I2S0 not being usable after bootloader_random_enable()
See merge request !415
fixes for github issues
This MR includes commits from several Github PRs, as well some fixes for issues reported on Github and on the forum.
- compile PHY-related code only when WiFi is enabled
- fix macOS setup instructions
- fix some of the Sphinx warnings
- update docs index, improve structure of the readme-
- wifi: fix typos, rename ESP_ERR_WIFI_NOT_START to ESP_ERR_WIFI_NOT_STARTED
- sdmmc: fix explanation of flash voltage regulator configuration efuses
- sdmmc: change idx num of example
- fix i2c_get_period name
- fix esp_log_level_set function name in docs
See merge request !420
Fix panic register dump format
Fix misplaced a printf somewhere causing an enter to be printed after every register in the panic regdump instead of every 4th register.
Fixes https://github.com/espressif/esp-idf/issues/236
See merge request !419
Most of the packages mentioned are only needed for building the toolchain using crosstool-NG, not for the normal ESP-IDF environment.
Mention that pyserial needs to be installed (fixes https://github.com/espressif/esp-idf/issues/229).
Feature/new mdns
This is a written from scratch mDNS implementation that supports compression, service and host lookups and service advertisement.
See merge request !365
esp32 core dump to flash
1. menuconfig option to select where to store core dump: flash, uart or disable
2. Saving of core dump to flash
3. Partition table definitions files with core dump partition
4. Python scripts to support core dump generation from GDB command line
See merge request !341
deep sleep related fixes
A few fixes for deep sleep behavior:
- Fix a regression that deep sleep stub crashed if `ESP32_DEEP_SLEEP_WAKEUP_DELAY` option was enabled — the stub called `ets_update_cpu_frequency`, which was recently redefined to be an IRAM function, rather than a ROM one. Fixed by restoring the ROM definition as `ets_update_cpu_frequency_rom`, and using that in the deep sleep stub.
- By default, don't power down RTC_SLOW_MEM if there is any data in .rtc.data or .rtc.bss sections. This provides better out-of-the box experience, as variables attributed to .rtc.data are actually preserved during deep sleep.
- Store boot time in RTC_STORE registers instead of using a variable with RTC_DATA_ATTR. This allows RTC_SLOW_MEM to be powered down in deep sleep if no variables with RTC_DATA_ATTR are present in the program.
Fixes https://github.com/espressif/esp-idf/issues/182
See merge request !409
fix mbedtls unit tests, fix IDF_VER when building a project outside ESP-IDF tree
- mbedTLS test needs about 5% more time to run (in debug mode) when building with 5.2.0, compared to 4.8.5. Increase the timeout to let the test pass.
- when doing `git describe` in IDF_VER definition, `-C $(IDF_PATH)` is needed to get the version of ESP-IDF, and not the project being built (which may not even be in git)
See merge request !411
Clean up WiFi & Bluetooth config options
* Splits "WiFi" configuration out from ESP32 configuration submenu to its own menu.
* Renames "BT" to "Bluetooth", enabling Bluetooth is now in this option not the ESP32 submenu.
* Also disables compiling/linking of BT stack if Bluetooth is disabled, saves some build time.
See merge request !397
update toolchain version, rebuild newlib
- This MR updates the toolchain version used in the ESP-IDF
- Compiler version check and warning is added to project.mk
- Newlib is rebuilt to include wcsftime function
- libstdc++ build uses correct newlib headers (so we won't get undefined reference to `__impure_ptr` anymore)
See merge request !387
Place cross-core interrupt into IRAM, sanity check handler address passed into esp_intr_alloc
Cross-core interrupt handler was not marked with IRAM_ATTR.
The reason why this caused an exception is probably due to some tasks running at highest priority (these are not blocked by spi_flash_disable_caches_interrupts_and_other_cpu mechanism).
This change puts the interrupt handler into IRAM and adds a sanity check into `esp_intr_alloc`.
Reported on Github: https://github.com/espressif/esp-idf/issues/211
See merge request !404
build system: add IDF_VER environment variable and preprocessor define
This adds an `IDF_VER` preprocessor define which is obtained from `git describe`.
Looks like `v1.0-275-g0efaa4f` for non-release version or `v2.0` for a tagged (release) version.
See merge request !403
bootloader: export ets_update_cpu_frequency
MR !363 added linking librtc into bootloader. At the same time, MR !373 removed ets_update_cpu_frequency from ROM exports ld script, and replaced it with a dual CPU aware version. Combining these two MRs caused linker error in bootloader due to the unresolved `ets_update_cpu_frequency`.
One option to fix this is to move the definition of ets_update_cpu_frequency into bootloader_support component. This does however look a bit odd because this function is not something that is specifically needed for bootloader support.
Another option is to export the ROM version of ets_update_cpu_frequency for the bootloader only. This means that when bootloader changes CPU frequency, it only updates the tick period variable for PRO CPU. This isn't an issue though, because app startup code still calls the dual CPU aware version of ets_update_cpu_frequency early on, before any non-trivial code can run on the APP cpu.
This MR uses the second approach. Looking forward, we may move this definition of ets_update_cpu_frequency into the rtc_clk component, which will be used both by the bootloader and the app, and placing ets_update_cpu_frequency there would be logical.
See merge request !384
SDMMC peripheral driver, SD protocol driver, FATFS library, VFS integration
This MR contains:
- SDMMC host peripheral driver
- SD protocol driver in sdmmc component (can be extended to support MMC/eMMC and SPI based hosts)
- ChaN's FATFS library v0.12b
- VFS integration
- FAT access via VFS is thread-safe (unless same file is read/written/unlinked/renamed from different tasks)
- Support for POSIX directory-related functions in VFS (and in vfs_fatfs.c)
- unit test for the above
- Example
- API documentation
Will be done in other MRs:
- Support for spi_flash IO driver for FatFs
- SPI host driver and support for SPI mode commands in sdmmc component
- MMC/eMMC support in sdmmc component
- Support for slightly higher 53/26.6MHz clocks (currently I'm using 20MHz for DS and 40MHz for HS, instead of 25MHz/50MHz per standard), and arbitrary low clocks (e.g. 4MHz).
See merge request !353
C++ support
This change adds necessary support for compiling C++ programs:
- linking against libstdc++
- implementation of static initialization guards using FreeRTOS primitives: since we don't have condition variables at our disposal, and we don't want to allocate a synchronization primitive for every guard variable generated by the compiler, we imitate condition variables using a combination of a mutex, counting semaphore, and a counter (based on [Microsoft Research paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2004/12/ImplementingCVs.pdf), albeit because we don't need *arbitrary* code to use these CVs, implementation gets simpler).
Note that libstdc++ also contains an implementation of `__cxa_guard_{acquire,release,abort}` functions. These implementations come from an `#ifndef GXX_THREADS` branch, i.e. are not aware of multthreading. There are three ways of replacing these libstdc++ functions with our implementation:
1. Move our code into gcc. Pros: cleanest solution. Cons: Such changes are unlikely to be merged by any upstream, so we end up maintaining our own forks of {gcc,crosstool-ng}.
2. Use library as it is built by crosstool, use `ar` to delete one object file (`guards.o`), add this library to ESP-IDF. Pros: easy to implement. Cons: libstdc++ is a 15MB binary 😯
3. Keep using libstdc++ from crosstool, force our implementation to be linked using a `-u` linker flag. Pros: no impact on repo size, easy to implement. Cons: somewhat less clean than 1 (and about as hacky as 2).
For the reasons mentioned, option (3) looks like the best tradeoff.
Ref. TW6702
See merge request !364
Enable bootloader entropy source for RNG
Enables an entropy source when bootloader starts up, which both seeds the RNG for use before WiFi/BT stack is enabled and provides an adequate RNG for secure boot & flash encryption key generation.
A prerequisite was enabling 80MHz operation, so the CPU is now set to 80MHz as soon as second stage bootloader starts running.
See merge request !363
add menuconfig option to enable SO_RCVBUF
This option is required by Arduino and enables netconn connection to be queried for amount of data available in the rx buffer.
See merge request !372
esp32: ets_update_cpu_frequency should set tick scale for both CPUs
ets_update_cpu_frequency ROM function updates g_ticks_per_us, which is has two copies, one for each CPU.
The APP CPU copy of g_ticks_per_us never got updated, resulting in shorter delays produced by ets_delay_us on the APP CPU.
This MR replaces ROM ets_update_cpu_frequency with a copy in IRAM which updates scaling factors on both of the CPUs.
So now we get expected delays (in microseconds):
```
ets_delay_us core=0 expected=50000 actual=50014
ets_delay_us core=1 expected=50000 actual=50015
vTaskDelay core=0 expected=50000 actual=49428
vTaskDelay core=1 expected=50000 actual=50000
```
Reported on the forum: http://esp32.com/viewtopic.php?f=2&t=713#p3722
See merge request !373
modify bootloader dram start address, swap app bss and data segments
Modify bootloader dram_seg from address 0x3ffc0000 to 0x3fff0000, len from 0x20000 to 0x10000.
Also put app .data before .bss, to reduce the chance .data collides with ROM bootloader stack.
See merge request !375
This change reduces chances that a large .bss segment will push .data all the way into
0x3ffe1320 — 0x3ffe5320 range where the bootloader stack is, creating a problem when
bootloader will be loading application into memory.
With this change, .data would need to be at least 200k big to cause problems.
GPIO driver: add per-pin interrupt handlers
1. add ISR handler apis so that users of different layers can hook their own isr handler on different GPIO.
Audio project has different software layers, they need different gpio isr handler for layer instead of processing all GPIO interrupts in one handler.
If this kind of calling a handler from isr is not proper, please kindly point out.
2. add gpio example code.
3. update gpio doc.
See merge request !345
Clarification on documenting examples
Improve documentation of examples with two basic actions:
- include README.md in each example project
- add a link with synopsis in API documentation
This was not clear reading original API documentation template.
See merge request !349
Ivan Grokhotkov