change(build): drop BUILD_COMPONENTS support and migrate test_apps to not use it for buildv2
Closes IDF-15859
See merge request espressif/esp-idf!51695
pytest.ini enables log_cli, so logging the full ninja stdout after a
failed first-time build is one multi-MB ERROR. That hung shard 3/6 for
hours after the hints test. Log the last 80 lines; keep the full output
on the exception and at DEBUG.
git worktree add materializes submodules as gitlink files. rmtree() cannot
remove those, and exists()+iterdir() on a source gitlink raises and falls
back to shutil.copytree, which leaves mbedtls/include as a file. Unlink
dest gitlinks and only copy populated source directories.
sdkconfig.ci.memprot_esp32s3 was built but listed in no CONFIGS_MEMPROT_*, so
the S3 PMS panic path was never exercised.
expect_gme() takes an optional core, for panics with no attributable core.
Reading the BUILD_COMPONENTS build property aborts the build with a bare
message stating that the property is unsupported, without naming a
replacement. Add a hint that points at the $<TARGET_EXISTS:idf::component>
generator expression and at the documentation section describing the
migration, so the guidance lives with the other build hints instead of
being spelled out in the build system sources.
The compatibility shim populated BUILD_COMPONENTS from the library
interface's linked-components list so that consumers reading it kept
working. Those consumers now query the library interface directly, so
drop the shim population and reject reads of the property
unconditionally.
The BUILD_COMPONENTS build property only exists when building through the
Build system v1 compatibility shim. Query the executable library
interface's linked-components list instead, so the app does not depend on
compatibility-only properties.
The BUILD_COMPONENTS build property only exists when building through the
Build system v1 compatibility shim. Query the executable library
interface's linked-components list instead, so the app does not depend on
compatibility-only properties.
The BUILD_COMPONENTS build property only exists when building through the
Build system v1 compatibility shim. Query the executable library
interface's linked-components list instead, so the app does not depend on
compatibility-only properties.
The bootloader programs only unlocked PMA entries (no PMP), so the layout is
not a bootloader<->application ABI; the application resets all entries, programs
them in ascending order from IRAM, and locks everything. Verified on ESP32-H4 v0.1.
idf.py mcp-server printed startup, shutdown, and error messages on
stdout. On the stdio MCP transport, stdout is the JSON-RPC channel, so
those non-JSON lines can confuse or break strict clients. Route them to
stderr like the other diagnostics in mcp_ext.py.
Co-authored-by: Cursor <cursoragent@cursor.com>
idf.py mcp-server tool handlers were spawning idf.py without redirecting
stdin, so the child inherited the long-lived MCP JSON-RPC transport and
could hang indefinitely on tools/call. Pass stdin=subprocess.DEVNULL on
every spawn of idf.py.
Closes https://github.com/espressif/esp-idf/issues/18961
Both PSRAM layouts were mapped as a single RWX window, so everything in external
RAM - the heap included - was executable.
PSRAM used as data only is now RW, and under XIP-from-PSRAM it is split per
section as ESP32-P4 does: .text RX, .rodata read-only, and the MMU-page
alignment gaps and the reclaimed heap RW, so neither is executable.
Both describe the layout that esp_psram_init() produces, and the entries are
locked, so - again as on ESP32-P4 - they are only narrowed when
CONFIG_SPIRAM_PRE_CONFIGURE_MEMORY_PROTECTION says that layout applies. Without
it the application owns the region and PSRAM stays RWX.
The per-section entries cost one PMP entry more than the 16 available, so the CPU
subsystem and peripheral windows are chained as TOR entries, taking one entry
instead of three.
soc.h is corrected against the S31 bus address map: the peripheral window base
was 1 MB too low, and the LP peripheral top, derived from a register base plus a
size rather than from the map, was 16 KB short. SOC_NON_CACHEABLE_OFFSET_FLASH
is added.
ESP-IDF compiles every component with -ffunction-sections/-fdata-sections and
links with -Wl,--gc-sections, so that code and data nothing references does not
reach the image. With CONFIG_COMPILER_LTO_LINKTIME the object files carry GIMPLE
instead of machine code and code generation is deferred to the link, where those
two options no longer apply. The LTO partition is therefore emitted as a single
.text/.rodata, and since --gc-sections works at section granularity it can only
keep or drop that section as a whole. Some function in it is always live, so
nothing is dropped.
The options are recorded in the object file (in the .gnu.lto_.opts section), but
they do not reach the code generator. Code generation at link time runs as a
separate compiler invocation (LTRANS) whose command line lto-wrapper
reconstructs from those recorded options, and append_compiler_options() in
gcc/lto-wrapper.cc forwards only CL_TARGET options plus a small hard-coded list:
-fPIC/-fpic/-fPIE/-fpie, -fcommon, -fgnu-tm, -fopenmp/-fopenacc, -fcf-protection=,
(-fasynchronous-)unwind-tables, -g, -O/-Os/-Og/-Ofast/-Oz and the diagnostics
formatting options. Everything else hits the default arm and is dropped.
-ffunction-sections/-fdata-sections are neither. They are plain Common options in
common.opt without the Optimization marker, so they are not part of the
per-function state that is streamed with each function (which is why -O2 and -Os
do survive per translation unit under LTO), and they are not target options
either. They reach LTRANS only if they are repeated on the link command line.
Pass them next to -flto=auto, in both build systems.
This can be verified on any LTO build by adding -save-temps to the link options
and inspecting the generated <output>.ltrans.mk, which contains the literal
LTRANS command line:
grep -o -- "-ffunction-sections\|-fdata-sections" build/*.ltrans.mk
Measured on esp32c3 with -Os and CONFIG_COMPILER_LTO_COMPILETIME, application
binary size in bytes:
no LTO LTO LTO+fix fix saves
hello_world 116112 119792 115808 -3984 (-3.33%)
wifi/getting_started 734720 792960 732976 -59984 (-7.56%)
Without this change, enabling LTO produces a larger image than not using LTO at
all; with it, LTO is size neutral to slightly positive.
Two effects contribute. The dead code inside the partition itself stays, which
scales with how much code was compiled with LTO. On top of that, every function
retained this way keeps whatever it references alive as well, transitively and
across object boundaries: constant data, other functions, and sections of objects
that were not compiled with LTO at all. That second effect is not bounded by the
size of the LTO partition and can dominate. In the wifi/getting_started case a
single retained function, esp_crt_bundle_attach(), is the only referrer of the
mbedTLS certificate bundle, so a 55 KB .rodata blob stayed in an application that
never uses TLS. A plain non-LTO build collects all of it, which is what this
change restores.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
