Build ULP full subprojects through a dedicated entry file,
components/ulp/cmake/ulp_project.cmake, that wraps tools/cmakev2/idf.cmake
and layers a small ULP API on top, mirroring the cmakev2 layering:
ulp_project_init like idf_project_init (init, detect the ULP type,
reset the compile/link options inherited from the app)
ulp_build_executable like idf_build_executable, plus the embeddable
.bin/.h/.ld artifacts
ulp_project_default like idf_project_default (single-executable case)
A child project now includes this one file instead of idf.cmake and calls
these helpers directly, so idf_build_executable is used as-is for the
multi-binary case and the module-path indirection (include(IDFULPProject)
resolved via -DCMAKE_MODULE_PATH) is gone.
As a result:
- IDFULPProjectv2.cmake is removed; its setup moves into the wrapper.
- IDFULPProject.cmake becomes the CMake v1-only entry point.
- The ULP component no longer registers a POST_ELF callback; the binary
artifacts are produced by ulp_build_executable.
- The v2 full-subproject examples (lp_core, riscv, fsm, multi_binary,
combined) are updated to the new API.
- The ULP subproject API is documented in build-system-v2.rst.
Also fix a latent bug this exercises: idf_build_library emitted linker
scripts as "-T <name>" relying on a following "-L" search directory. GNU
ld only searches -L directories that precede -T, so the direct
esp32ulp-elf-ld link used for ULP FSM failed to open the script. Emit the
absolute path instead, matching what the CMake v1 ULP build already does.
Finally, replace the parent-argument bypass loop that used to live in
IDFULPProjectv2.cmake with --no-warn-unused-cli on the child configure,
and stop passing the unused IDF_PARENT_BUILD_DIR.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Register ULP memory linker scripts with target_linker_script so CMake v2 handles preprocessing and attachment through the component graph.
Keep the generated legacy outputs named .ld by stripping only the .in suffix, and pass full linker script paths to support direct ld invocation.
Resolve #include lines in linker script templates against the linked
component graph: idf_build_library appends every linked component's
INCLUDE_DIRS to the C preprocessor invocation for each .in linker
script, so a template can include any component header (e.g. a ULP
memory-layout template including soc/soc.h) without the owning
component having to know about it.
Also:
- Always pass -I<config_dir> so templates can include sdkconfig.h.
- Add a FLAGS option to target_linker_script for explicit preprocessor
flags (e.g. -D__ASSEMBLER__ or ld-snippet include dirs); when given,
FLAGS replaces the parent-dir==target include heuristic for that
script.
- Add a MEMORY option to target_linker_script that emits the marked
linker script as -T before all others, so section-placement scripts
in one component can reference MEMORY regions and REGION_ALIASes
declared by a memory-layout script in another component.
- Make C-preprocessor comment keeping (-C) part of the default flag set
rather than hardcoded, so FLAGS can drop it. linker_script_preprocessor
no longer forces -C; both the cmakev1 and cmakev2 preprocessors add it
to their default CFLAGS (output unchanged for existing scripts). A ULP
template that includes soc/soc.h omits -C so ld does not choke on the
header's // comments.
- Store per-script metadata (generated output, flags) in MD5-keyed
component properties instead of parallel lists.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Mirror the link-time optimization support into the cmakev2 build system so
that both build systems behave identically.
- project.cmake (__init_project_configuration): emit -flto=auto as a link
option when CONFIG_COMPILER_LTO_LINKTIME is set, except for bootloader and
ESP-TEE builds, otherwise keep -fno-lto.
- build.cmake (idf_build_library): when CONFIG_COMPILER_LTO_COMPILETIME is set,
compile each linked component with -flto=auto unless it has linker fragments,
is placed by another component's fragment (see tools/cmake/lto.cmake), has
opted out via NO_LTO, or is not a static library.
- project.cmake: when CONFIG_APP_REPRODUCIBLE_BUILD is also enabled, apply
the same three flags as the legacy build system to keep LTO output
reproducible: pass the prefix-map options to the linker (so link-time code
generation remaps DW_AT_comp_dir), add -save-temps (stable LTRANS object
names instead of random $TMPDIR paths in the .map), and pin -frandom-seed
(byte-identical LTO GIMPLE bytecode). See the commit message of
"feat(build): add options to enable link-time optimization (LTO)" for the
full analysis.
The gcc-ar / gcc-ranlib selection and the NO_LTO component property are shared
with the legacy build system through tools/cmake/toolchain.cmake and the common
component registration code, so no cmakev2-specific changes are needed there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sub-projects that grow their own configuration (e.g. the ULP cmakev2
flow, where the sub-project runs its own kconfgen and writes its own
sdkconfig) need a way to expose that configuration via `idf.py
menuconfig` without forcing every caller to know per-sub-project target
names.
Add a small registration + dispatcher mechanism in cmakev2:
* `idf_register_menuconfig(NAME <label> TARGET <cmake-target>)` records
a menuconfig target with the build. It is intentionally decoupled
from `idf_create_menuconfig` so that any custom target may be
registered, including the parent-side proxy targets that sub-projects
add via `externalproject_add` (e.g. ULP's `menuconfig-<app_name>`).
* `__finalize_menuconfig()` at project finalization creates the
user-facing `menuconfig` target:
- With a single registration, `menuconfig` is a thin alias for that
target. Behaviour is identical to before for all existing single
-menuconfig projects (bootloader/TEE share the main configuration,
so they do not register).
- With two or more registrations, `menuconfig` runs a small Python
dispatcher (`tools/kconfig_new/menuconfig_dispatcher.py`) that
lists the registered configurations and re-invokes the selected
target. The prompt surfaces the direct target name (e.g.
`idf.py menuconfig-ulp_app`) so users learn the per-sub-project
target after first use.
`__project_default()` is updated to register the main application as
`menuconfig-app` and then call `__finalize_menuconfig`. The component
side change is one line in `components/ulp/project_include.cmake`,
registering the existing `menuconfig-<app_name>` proxy target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The GENERATE_SDKCONFIG build property was unconditionally set to 1 in
__init_kconfig(). Change it to use __get_default_value() so that a
value passed via -DGENERATE_SDKCONFIG=0 (e.g. from a parent build's
externalproject_add) is respected. This allows sub-projects like the
bootloader to skip writing back to the parent's sdkconfig file without
needing to explicitly set the property in their CMakeLists.txt.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Sub-projects like ULP are built as external CMake projects via
externalproject_add() and may use toolchains that are completely
different from the IDF target toolchain. For example, the ULP RISC-V
co-processor on an xtensa target (ESP32-S2/S3) uses
riscv32-esp-elf-gcc while the main project uses xtensa-esp-elf-gcc.
This causes problems in cmakev2:
1. __init_toolchain() in idf.cmake validates that CMAKE_TOOLCHAIN_FILE
matches IDF_TARGET and then overrides it. For sub-projects with
custom toolchains (passed via -DCMAKE_TOOLCHAIN_FILE from the
parent), this validation fails because the toolchain doesn't follow
the IDF naming convention (toolchain-<target>.cmake).
2. Several component project_include.cmake files use functions from
the IDF toolchain response file machinery (idf_toolchain_add_flags,
idf_toolchain_remove_flags, idf_toolchain_rerun_abi_detection) that
are not available in sub-projects with custom toolchains, or perform
validation that assumes the IDF target toolchain is in use.
Introduce IDF_CUSTOM_TOOLCHAIN, a CMake variable passed via -D from
the parent build to indicate that the toolchain was provided externally
and should not be resolved or validated by the IDF build system.
When IDF_CUSTOM_TOOLCHAIN is set:
- __init_toolchain() records the pre-set CMAKE_TOOLCHAIN_FILE and
skips IDF_TARGET-based resolution and validation.
- Component project_include.cmake files that manipulate IDF toolchain
flags (xtensa, esp_libc, esp_psram, soc) return early, as those
operations are not applicable to custom toolchains.
This is a generic mechanism usable by any sub-project that provides
its own toolchain (ULP, or any future sub-project with a non-IDF
toolchain).
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev2 path snapshots SDKCONFIG_DEFAULTS once during early init. Build
system v1 apps that mutate SDKCONFIG_DEFAULTS after `include(project.cmake)`
but before `project()` (e.g. `list(PREPEND SDKCONFIG_DEFAULTS ...)` to inject
tools/test_apps/configs/sdkconfig.debug_helpers) miss those mutations when
the shim drives the cmakev2 path, because the snapshot was taken before the
mutation.
Re-resolve SDKCONFIG_DEFAULTS at sdkconfig-generation time when
__V1_COMPAT_SHIM is active so the late-mutated value is honored. Native
cmakev2 apps still use the property-based contract via
idf_build_set_property(SDKCONFIG_DEFAULTS ... APPEND).
Promote the build system v2 functions and macros used by the
examples/build_system/cmakev2 examples to the generated API reference,
document the component-scope and version variables, the public build
properties, and the public component properties, and add cmakev2
build_property and component_property directives with dedicated Build
Properties and Component Properties sections to the esp-docs extension.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When an app sets __COMPONENT_REQUIRES_COMMON before project(), the
explicit value used to bypass the default branch that includes
${idf_target_arch}, dropping the arch component from the build.
IDF_TARGET_ARCH isn't known pre-project(), so apps shouldn't have to
hand-roll a target -> arch map. Append it after the explicit list during
common-component initialization; empty on linux means no append.
Read sdkconfig.cmake's CONFIG_* values and re-publish them on the
build-properties target so component CMakeLists.txt code that queries
Kconfig via idf_build_get_property(var CONFIG_FOO) returns the expected
value.
Add a COMMAND check on esptool_py_flash_target before invoking it in
__init_project_flash_targets. Apps that restrict the build set without
esptool_py would otherwise hit "Unknown CMake command".
Move managed-dependency injection and recursive inclusion to BEFORE the
component's add_subdirectory() call, so its CMakeLists.txt can resolve
managed-dep targets at register time. Inside idf_component_register,
union the manager-injected REQUIRES/PRIV_REQUIRES with what the
component author wrote instead of overwriting them.
Under __V1_COMPAT_SHIM, populate EXECUTABLE, EXECUTABLE_NAME, and
BUILD_COMPONENTS build properties in __project_default(), and propagate
project_elf to the caller scope. Relax the BUILD_COMPONENTS query gate
in build.cmake so component code that uses
idf_build_get_property(... BUILD_COMPONENTS) keeps working.
Co-authored-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Replicate Build system v1 behavior where the 'main' component implicitly
depends on the discovered components (or the app-restricted
__SHIM_COMPONENTS list) when no explicit REQUIRES/PRIV_REQUIRES is set.
Gated on __V1_COMPAT_SHIM so native Build system v2 projects retain
their explicit dependency contract.
Co-authored-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When IDF_BUILD_V2 evaluates to a CMake-truthy value, project.cmake
delegates to Build system v2 (cmakev2). The shim wraps project(),
forwards the app-declared COMPONENTS list via __SHIM_COMPONENTS, and
publishes __V1_COMPAT_SHIM so Build system v2 internals can gate Build
system v1 compatibility behavior. Existing app CMakeLists.txt files
build unchanged.
Co-authored-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
- Harden prefix_map.cmake for paths with spaces on Windows
- ldgen: fragments-list-file, list normalization, CMake integration (tools/cmake + cmakev2)
- CI exclude list for check_tools; test_spaces bundle tweak
Made-with: Cursor
cmakev2 does not use configdep yet, so the manually specified variable
CONFIGDEP_ENABLE is unused. CMake warns about it, which causes CI to
fail. Until configdep is adopted by cmakev2, suppress the warning.
Co-authored-by: Cursor <cursoragent@cursor.com>
When esp-idf-kconfig >= 3.9.0 is installed, we can use the optimized
menuconfig target without the need to preprocess the sdkconfig file by
kconfgen (removing deprecated options - menuconfig - readding deprecated
options).
Also removed unused {menuconfig_depends} from the new target.
Co-authored-by: Cursor <cursoragent@cursor.com>
Refactor the esp_err_to_name() system to decouple esp_common from
higher-level components. Instead of a monolithic generated table,
each component registers its error codes into a dedicated linker
section (.esp_err_msg_table) via idf_define_esp_err_codes() in its
CMakeLists.txt.
New files:
- tools/err_codes_extract.py: extract ESP_ERR_* defines from headers to CSV
- tools/err_codes_to_c.py: generate C source placing entries into linker section
- tools/err_codes_to_rst.py: generate RST documentation from error codes
- tools/cmake/err_codes.cmake: CMake module providing idf_define_esp_err_codes()
- components/esp_common/include/esp_err_codes.h: esp_err_msg_t typedef
- components/esp_common/src/esp_err_to_name_new.c: new lookup using link-time array
- tools/test_apps/build_system/err_codes_check/: CI test app
Changes:
- Remove all optional component dependencies from esp_common/CMakeLists.txt
- Add .esp_err_msg_table section to all 5 linker scripts
- Register error codes in 18 components via idf_define_esp_err_codes()
- Add new scripts to .gitlab/ci/rules.yml build_check patterns
- use new scripts to generate doc and add CI validation
- Update esp_err.rst to add description of composable code registration
In cmakev1, the executable target is named "${project}.elf". In cmakev2,
the executable is named "${project}" with .elf as the output suffix.
Strip the .elf suffix and look up the bare target name when the original
name doesn't exist.
Two compatibility fixes:
- idf_component_register: apply separate_arguments to REQUIRES and
PRIV_REQUIRES after cmake_parse_arguments. Some managed components
publish CMakeLists.txt with PRIV_REQUIRES "log esp_eth" as a single
quoted string (e.g. espressif__rtl8201).
- Add register_component() macro and idf_component_add_link_dependency()
shims for legacy ESP-IDF examples (ULP apps, BLE mesh) and managed
components (esp_flash_nor).
kconfgen runs while the component manager iterates to convergence.
Those passes operate on partial component sets and emit "unknown
kconfig symbol" warnings for symbols defined in not-yet-downloaded
components — idf-build-apps treats those as build failures.
Suppress kconfgen output on the intermediate passes; only the final
pass against the converged set emits warnings.
The cmakev2 compat layer's idf_component_register() was passing
${sources} quoted to add_library(), collapsing the semicolon-delimited
list into a single argument. Drop the quotes so each source becomes a
separate argument, matching cmakev1's idf_component_register().
When a component's manifest declared a managed dep `<ns>/<name>` and the
project had a local `components/<name>` shadowing it, configure failed
with `IDF: Failed to resolve component '<ns>__<name>'`.
cmakev1 does a project-wide injection so the manager's _choose_component
sees every component name in known_components and rewrites the
namespaced dep to its locally-shadowing short name. cmakev2 invokes the
manager per component with a single-component requirements file, so
known_components has only one entry and the rewrite cannot fire --
`<ns>__<name>` is written verbatim into MANAGED_REQUIRES and the
force-include at component.cmake:995 aborts.
Seed the per-component requirements file with one __COMPONENT_SOURCE
entry per discovered component so handle_project_requirements() builds
the same known_components list cmakev1 produces. The seeded entries are
inert on the cmake side -- the __component_set_property shim ignores
__COMPONENT_SOURCE -- they exist purely to feed the manager's rewrite.
Add a regression test in tools/test_build_system/buildv2/test_component.py.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add an OPTIONAL flag to idf_component_include so callers can ask
"include this component if it exists" without aborting the build
when the component is unknown.
Without OPTIONAL, the function behaves exactly as before -- it
calls __get_component_interface_or_die and the build aborts on a
miss. With OPTIONAL, the function performs a non-fatal interface
lookup and returns silently if the component is not known. When
combined with INTERFACE <variable>, the variable is set to the
empty string on miss and to the component's interface target on
hit, so callers can write:
idf_component_include(button OPTIONAL INTERFACE button_iface)
if(button_iface)
target_sources(${COMPONENT_TARGET} PRIVATE button_glue.c)
target_link_libraries(${COMPONENT_TARGET} PRIVATE ${button_iface})
endif()
This gives consumers a public, non-fatal way to wire up integrations
with components that may or may not be in the build (managed
dependencies pulled in only by some board configurations, optional
feature glue, etc.) without reaching into the private
__get_component_interface helper or doing dual-namespace
COMPONENTS_DISCOVERED checks against both <name> and <ns>__<name>
forms.
Backward compatible: existing call sites do not pass OPTIONAL and
continue to fail loudly on miss.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In the cmakev2 build framework, the selected toolchain was not
consistently propagated to sdkconfig generation. For clang builds,
this produced a configuration that was incompatible with the clang
toolchain and broke compilation.
Ensure the toolchain selection is observed consistently so sdkconfig
and component configuration reflect the intended toolchain for both
GCC and clang builds.
Made-with: Cursor
The __init_project_configuration() function in cmakev2's project.cmake
unconditionally applied app-level compiler optimization flags based on
CONFIG_COMPILER_OPTIMIZATION_* Kconfig options. When the bootloader
subproject was built with cmakev2, these app-level flags leaked into the
bootloader compile command alongside the correct bootloader-specific
flags from CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_*.
For example, with the default configuration (app: DEBUG, bootloader:
SIZE), the bootloader received both "-Og -fno-shrink-wrap" (from app
config) and "-Os -freorder-blocks" (from bootloader config). While GCC
uses the last -O flag (-Os wins), the stray -fno-shrink-wrap persisted.
Introduce a SET_COMPILER_OPTIMIZATION build property that defaults to
YES when unset. Subprojects that manage their own optimization flags
(like the bootloader) can set this to NO before calling
idf_project_init() to prevent the default optimization flags from being
applied. This keeps project.cmake generic without requiring it to know
about specific subproject types.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_path variable was used in -fmacro-prefix-map and
-fdebug-prefix-map flags but never read from the IDF_PATH build
property, resulting in an empty substitution. This caused full
filesystem paths to leak into .rodata instead of being mapped to /IDF.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Commit f62f45cf5c changed target_add_binary_data to resolve the
embedded file path relative to the component directory by using
idf_component_get_property to obtain the COMPONENT_DIR. This fixed the
path resolution when target_add_binary_data is called from
idf_component_include, which runs outside the component's directory
context.
However, target_add_binary_data can also be called directly from a
project's CMakeLists.txt with a non-component target, e.g.
target_add_binary_data(app.elf ...) as done in
examples/security/security_features_app. In this case,
idf_component_get_property fails because the target is not a component.
Fix this by moving the path resolution to idf_component_include, where
the embed file paths from EMBED_FILES and EMBED_TXTFILES component
properties are resolved to absolute paths relative to COMPONENT_DIR
before being passed to target_add_binary_data. This way,
target_add_binary_data receives already absolute paths from
idf_component_include and can use plain get_filename_component(ABSOLUTE)
for direct calls, which correctly resolves relative to
CMAKE_CURRENT_SOURCE_DIR.
Fixes: f62f45cf5c ("fix(cmakev2/utilities): add a dependency target for the embedded file")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
