Move 7 trace-related examples (app_trace_basic, app_trace_to_plot,
esp_trace_custom_library, function_tracing, gcov, sysview_tracing,
sysview_tracing_heap_log) from examples/system/ into a dedicated
examples/system/tracing/ subdirectory for better organization.
Update all path references across documentation (en + zh_CN),
build-test-rules, CI configs, Kconfig, and component READMEs.
Validate linked RISC-V executables when
CONFIG_COMPILER_ENABLE_RISCV_ZCMP is enabled on affected chips.
Disassemble each function and reject mstatus.MIE clears that lack an
earlier mintthresh write of 0xff.
Handle csrrci, csrrw, and register-mask csrrc patterns while ignoring
csrrs. Add build-only coverage for valid and invalid sequences with
CMake v1 and v2.
Stop the hardware stack guard before switching stacks during restart,
and keep ZCMP disabled for TEE test apps that still require the
workaround.
Allow specifying extra compiler flags (warning flags, preprocessor
definitions, etc.) via a new 'compile_options' entry in the
idf-build-file frontmatter. The options are applied to the source
file via target_compile_options in the generated main component
CMakeLists, with CMake quoting so that flags like -DMSG="hello world"
reach the compiler exactly as written.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a new idf.py extension that allows building standalone C files
without requiring full project boilerplate. Source files can include
optional YAML frontmatter in block comments to specify sdkconfig
options, component dependencies, and target configuration.
When the frontmatter configuration changes, the stale sdkconfig is
removed so the new defaults are applied, and a target change also
clears the build directory since IDF_TARGET is pinned in the CMake
cache.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move the following modules from the hal component into a new dedicated
esp_hal_debug_assist component, following the esp_hal_timg pattern:
- assist_debug (hal + target-specific LL headers)
- debug_probe (types + target-specific LL headers)
- riscv_trace (hal + types + source + target-specific LL headers)
- trace_ll (esp32/esp32s2/esp32s3 target-specific LL headers)
Update the following components to depend on esp_hal_debug_assist:
esp_system, riscv, bootloader_support, esp_hw_support, esp_riscv_trace
Remove riscv_trace_hal.c from the hal component.
Migrate idf.py and its actions to the shared esp-pylib library:
- add esp-pylib (and esp-pylib[ide]) to the core/ide requirements
- base FatalError on esp_pylib.errors.FatalError, keeping the idf.py ctx cleanup
- replace the local raw-ANSI helpers (red_print/yellow_print/print_warning/
color_print) with esp_pylib.logger.log (warn/err/note/hint) across the
idf.py actions
- install esp_pylib exception reporting at the idf.py entry point and silence
the logger during shell completion
- update affected tests for the new prefixes/line wrapping
- add normalize_output() helper and unify terminal env in conftest to handle
Rich line-wrapping in CI assertions
Co-authored-by: Cursor <cursoragent@cursor.com>
Keep legacy ULP apps on the CMake v1 build path while buildv2 coverage
is limited to full_subproject ULP apps.
Also add the missing esp_driver_gpio dependency to the esp_pm test app
so buildv2 dependency checks see driver/rtc_io.h explicitly.
Build ULP full subprojects through a dedicated entry file,
components/ulp/cmake/ulp_project.cmake, that wraps tools/cmakev2/idf.cmake
and layers a small ULP API on top, mirroring the cmakev2 layering:
ulp_project_init like idf_project_init (init, detect the ULP type,
reset the compile/link options inherited from the app)
ulp_build_executable like idf_build_executable, plus the embeddable
.bin/.h/.ld artifacts
ulp_project_default like idf_project_default (single-executable case)
A child project now includes this one file instead of idf.cmake and calls
these helpers directly, so idf_build_executable is used as-is for the
multi-binary case and the module-path indirection (include(IDFULPProject)
resolved via -DCMAKE_MODULE_PATH) is gone.
As a result:
- IDFULPProjectv2.cmake is removed; its setup moves into the wrapper.
- IDFULPProject.cmake becomes the CMake v1-only entry point.
- The ULP component no longer registers a POST_ELF callback; the binary
artifacts are produced by ulp_build_executable.
- The v2 full-subproject examples (lp_core, riscv, fsm, multi_binary,
combined) are updated to the new API.
- The ULP subproject API is documented in build-system-v2.rst.
Also fix a latent bug this exercises: idf_build_library emitted linker
scripts as "-T <name>" relying on a following "-L" search directory. GNU
ld only searches -L directories that precede -T, so the direct
esp32ulp-elf-ld link used for ULP FSM failed to open the script. Emit the
absolute path instead, matching what the CMake v1 ULP build already does.
Finally, replace the parent-argument bypass loop that used to live in
IDFULPProjectv2.cmake with --no-warn-unused-cli on the child configure,
and stop passing the unused IDF_PARENT_BUILD_DIR.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Register ULP memory linker scripts with target_linker_script so CMake v2 handles preprocessing and attachment through the component graph.
Keep the generated legacy outputs named .ld by stripping only the .in suffix, and pass full linker script paths to support direct ld invocation.
Resolve #include lines in linker script templates against the linked
component graph: idf_build_library appends every linked component's
INCLUDE_DIRS to the C preprocessor invocation for each .in linker
script, so a template can include any component header (e.g. a ULP
memory-layout template including soc/soc.h) without the owning
component having to know about it.
Also:
- Always pass -I<config_dir> so templates can include sdkconfig.h.
- Add a FLAGS option to target_linker_script for explicit preprocessor
flags (e.g. -D__ASSEMBLER__ or ld-snippet include dirs); when given,
FLAGS replaces the parent-dir==target include heuristic for that
script.
- Add a MEMORY option to target_linker_script that emits the marked
linker script as -T before all others, so section-placement scripts
in one component can reference MEMORY regions and REGION_ALIASes
declared by a memory-layout script in another component.
- Make C-preprocessor comment keeping (-C) part of the default flag set
rather than hardcoded, so FLAGS can drop it. linker_script_preprocessor
no longer forces -C; both the cmakev1 and cmakev2 preprocessors add it
to their default CFLAGS (output unchanged for existing scripts). A ULP
template that includes soc/soc.h omits -C so ld does not choke on the
header's // comments.
- Store per-script metadata (generated output, flags) in MD5-keyed
component properties instead of parallel lists.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add an opt_os_lto configuration to the startup test app that enables size
optimization together with link-time and compile-time LTO, and run it across
the supported targets so the LTO build is exercised on real hardware in CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the link-time optimization support into the cmakev2 build system so
that both build systems behave identically.
- project.cmake (__init_project_configuration): emit -flto=auto as a link
option when CONFIG_COMPILER_LTO_LINKTIME is set, except for bootloader and
ESP-TEE builds, otherwise keep -fno-lto.
- build.cmake (idf_build_library): when CONFIG_COMPILER_LTO_COMPILETIME is set,
compile each linked component with -flto=auto unless it has linker fragments,
is placed by another component's fragment (see tools/cmake/lto.cmake), has
opted out via NO_LTO, or is not a static library.
- project.cmake: when CONFIG_APP_REPRODUCIBLE_BUILD is also enabled, apply
the same three flags as the legacy build system to keep LTO output
reproducible: pass the prefix-map options to the linker (so link-time code
generation remaps DW_AT_comp_dir), add -save-temps (stable LTRANS object
names instead of random $TMPDIR paths in the .map), and pin -frandom-seed
(byte-identical LTO GIMPLE bytecode). See the commit message of
"feat(build): add options to enable link-time optimization (LTO)" for the
full analysis.
The gcc-ar / gcc-ranlib selection and the NO_LTO component property are shared
with the legacy build system through tools/cmake/toolchain.cmake and the common
component registration code, so no cmakev2-specific changes are needed there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Link-time optimization (LTO) lets the compiler inline and optimize across
translation units. ESP-IDF relies heavily on linker-script placement rules
that match object files by name, which LTO does not preserve, so LTO cannot
be enabled for the whole framework. This change adds two opt-in options that
side-step that conflict:
- CONFIG_COMPILER_LTO_LINKTIME tells the linker to perform LTO on any object
files that carry LTO information (compiled with -flto). On its own this is
safe: users can add -flto to specific components (e.g. their own libraries)
to shrink them, without affecting components that use linker fragments.
- CONFIG_COMPILER_LTO_COMPILETIME automatically compiles most
components with -flto. A component is excluded when it has its own linker
fragments, when it opts out via the NO_LTO component property, or when its
object code is placed by *another* component's linker fragment (matched by
archive name). The last case is handled by tools/cmake/lto.cmake, which
scans linker fragments for explicit "archive: libNAME.a" placement and
excludes those components. Without it, functions that must run from IRAM
while the flash cache is disabled (e.g. the spi_flash / GDMA HAL routines,
placed in IRAM by spi_flash/esp_driver_dma fragments) would be moved to
flash by LTO and the device would panic with a cache error at run time.
Both options are disabled for the bootloader and ESP-TEE builds, which depend
on object-file-name based placement. LTO is also gated off for Clang
(needs LLD, IDF-8286) and host builds.
LTO works together with CONFIG_APP_REPRODUCIBLE_BUILD, but needs extra
flags: LTO defers code generation and most debug-info emission from compile
time to link time, where the reproducible-build path remapping (applied to
compile_options only) does not take effect. When both options are enabled,
three extra flags keep the .elf, .bin and .map byte-identical across build
directories (verified on esp32 / GCC 16.1):
- the -f*-prefix-map options are passed to the linker as well, so the LTO
code generator remaps DW_AT_comp_dir (otherwise the build dir leaks into
.debug_str, and cascades into esp_app_desc_t.app_elf_sha256 in the .bin);
- -save-temps makes lto-wrapper use stable LTRANS object names in the build
dir instead of random $TMPDIR paths that leak into the .map;
- -frandom-seed=1 makes LTO GIMPLE bytecode objects byte-identical (a
shared seed was verified not to collide, including for C++ file-local
static variables and anonymous namespaces promoted by LTO).
The gcc-ar / gcc-ranlib wrappers are selected in the GCC toolchain file so
that the LTO plugin is loaded when creating and indexing static archives;
plain ar/ranlib do not record LTO symbols in the archive index.
Note: LTO, like other inlining, can also increase binary size. Enable it
together with CONFIG_COMPILER_OPTIMIZATION_SIZE to get a code-size benefit.
Related: IDF-71, IDF-8286
Closes https://github.com/espressif/esp-idf/issues/18741
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Use a non-deprecated bootloader support API in the mock build test.
Keep the test focused on validating the generated bootloader support mock.
Co-authored-by: Cursor <cursoragent@cursor.com>
Sub-projects that grow their own configuration (e.g. the ULP cmakev2
flow, where the sub-project runs its own kconfgen and writes its own
sdkconfig) need a way to expose that configuration via `idf.py
menuconfig` without forcing every caller to know per-sub-project target
names.
Add a small registration + dispatcher mechanism in cmakev2:
* `idf_register_menuconfig(NAME <label> TARGET <cmake-target>)` records
a menuconfig target with the build. It is intentionally decoupled
from `idf_create_menuconfig` so that any custom target may be
registered, including the parent-side proxy targets that sub-projects
add via `externalproject_add` (e.g. ULP's `menuconfig-<app_name>`).
* `__finalize_menuconfig()` at project finalization creates the
user-facing `menuconfig` target:
- With a single registration, `menuconfig` is a thin alias for that
target. Behaviour is identical to before for all existing single
-menuconfig projects (bootloader/TEE share the main configuration,
so they do not register).
- With two or more registrations, `menuconfig` runs a small Python
dispatcher (`tools/kconfig_new/menuconfig_dispatcher.py`) that
lists the registered configurations and re-invokes the selected
target. The prompt surfaces the direct target name (e.g.
`idf.py menuconfig-ulp_app`) so users learn the per-sub-project
target after first use.
`__project_default()` is updated to register the main application as
`menuconfig-app` and then call `__finalize_menuconfig`. The component
side change is one line in `components/ulp/project_include.cmake`,
registering the existing `menuconfig-<app_name>` proxy target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The GENERATE_SDKCONFIG build property was unconditionally set to 1 in
__init_kconfig(). Change it to use __get_default_value() so that a
value passed via -DGENERATE_SDKCONFIG=0 (e.g. from a parent build's
externalproject_add) is respected. This allows sub-projects like the
bootloader to skip writing back to the parent's sdkconfig file without
needing to explicitly set the property in their CMakeLists.txt.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Sub-projects like ULP are built as external CMake projects via
externalproject_add() and may use toolchains that are completely
different from the IDF target toolchain. For example, the ULP RISC-V
co-processor on an xtensa target (ESP32-S2/S3) uses
riscv32-esp-elf-gcc while the main project uses xtensa-esp-elf-gcc.
This causes problems in cmakev2:
1. __init_toolchain() in idf.cmake validates that CMAKE_TOOLCHAIN_FILE
matches IDF_TARGET and then overrides it. For sub-projects with
custom toolchains (passed via -DCMAKE_TOOLCHAIN_FILE from the
parent), this validation fails because the toolchain doesn't follow
the IDF naming convention (toolchain-<target>.cmake).
2. Several component project_include.cmake files use functions from
the IDF toolchain response file machinery (idf_toolchain_add_flags,
idf_toolchain_remove_flags, idf_toolchain_rerun_abi_detection) that
are not available in sub-projects with custom toolchains, or perform
validation that assumes the IDF target toolchain is in use.
Introduce IDF_CUSTOM_TOOLCHAIN, a CMake variable passed via -D from
the parent build to indicate that the toolchain was provided externally
and should not be resolved or validated by the IDF build system.
When IDF_CUSTOM_TOOLCHAIN is set:
- __init_toolchain() records the pre-set CMAKE_TOOLCHAIN_FILE and
skips IDF_TARGET-based resolution and validation.
- Component project_include.cmake files that manipulate IDF toolchain
flags (xtensa, esp_libc, esp_psram, soc) return early, as those
operations are not applicable to custom toolchains.
This is a generic mechanism usable by any sub-project that provides
its own toolchain (ULP, or any future sub-project with a non-IDF
toolchain).
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
