Mirror the link-time optimization support into the cmakev2 build system so
that both build systems behave identically.
- project.cmake (__init_project_configuration): emit -flto=auto as a link
option when CONFIG_COMPILER_LTO_LINKTIME is set, except for bootloader and
ESP-TEE builds, otherwise keep -fno-lto.
- build.cmake (idf_build_library): when CONFIG_COMPILER_LTO_COMPILETIME is set,
compile each linked component with -flto=auto unless it has linker fragments,
is placed by another component's fragment (see tools/cmake/lto.cmake), has
opted out via NO_LTO, or is not a static library.
- project.cmake: when CONFIG_APP_REPRODUCIBLE_BUILD is also enabled, apply
the same three flags as the legacy build system to keep LTO output
reproducible: pass the prefix-map options to the linker (so link-time code
generation remaps DW_AT_comp_dir), add -save-temps (stable LTRANS object
names instead of random $TMPDIR paths in the .map), and pin -frandom-seed
(byte-identical LTO GIMPLE bytecode). See the commit message of
"feat(build): add options to enable link-time optimization (LTO)" for the
full analysis.
The gcc-ar / gcc-ranlib selection and the NO_LTO component property are shared
with the legacy build system through tools/cmake/toolchain.cmake and the common
component registration code, so no cmakev2-specific changes are needed there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sub-projects that grow their own configuration (e.g. the ULP cmakev2
flow, where the sub-project runs its own kconfgen and writes its own
sdkconfig) need a way to expose that configuration via `idf.py
menuconfig` without forcing every caller to know per-sub-project target
names.
Add a small registration + dispatcher mechanism in cmakev2:
* `idf_register_menuconfig(NAME <label> TARGET <cmake-target>)` records
a menuconfig target with the build. It is intentionally decoupled
from `idf_create_menuconfig` so that any custom target may be
registered, including the parent-side proxy targets that sub-projects
add via `externalproject_add` (e.g. ULP's `menuconfig-<app_name>`).
* `__finalize_menuconfig()` at project finalization creates the
user-facing `menuconfig` target:
- With a single registration, `menuconfig` is a thin alias for that
target. Behaviour is identical to before for all existing single
-menuconfig projects (bootloader/TEE share the main configuration,
so they do not register).
- With two or more registrations, `menuconfig` runs a small Python
dispatcher (`tools/kconfig_new/menuconfig_dispatcher.py`) that
lists the registered configurations and re-invokes the selected
target. The prompt surfaces the direct target name (e.g.
`idf.py menuconfig-ulp_app`) so users learn the per-sub-project
target after first use.
`__project_default()` is updated to register the main application as
`menuconfig-app` and then call `__finalize_menuconfig`. The component
side change is one line in `components/ulp/project_include.cmake`,
registering the existing `menuconfig-<app_name>` proxy target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Promote the build system v2 functions and macros used by the
examples/build_system/cmakev2 examples to the generated API reference,
document the component-scope and version variables, the public build
properties, and the public component properties, and add cmakev2
build_property and component_property directives with dedicated Build
Properties and Component Properties sections to the esp-docs extension.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a COMMAND check on esptool_py_flash_target before invoking it in
__init_project_flash_targets. Apps that restrict the build set without
esptool_py would otherwise hit "Unknown CMake command".
Under __V1_COMPAT_SHIM, populate EXECUTABLE, EXECUTABLE_NAME, and
BUILD_COMPONENTS build properties in __project_default(), and propagate
project_elf to the caller scope. Relax the BUILD_COMPONENTS query gate
in build.cmake so component code that uses
idf_build_get_property(... BUILD_COMPONENTS) keeps working.
Co-authored-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When IDF_BUILD_V2 evaluates to a CMake-truthy value, project.cmake
delegates to Build system v2 (cmakev2). The shim wraps project(),
forwards the app-declared COMPONENTS list via __SHIM_COMPONENTS, and
publishes __V1_COMPAT_SHIM so Build system v2 internals can gate Build
system v1 compatibility behavior. Existing app CMakeLists.txt files
build unchanged.
Co-authored-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
kconfgen runs while the component manager iterates to convergence.
Those passes operate on partial component sets and emit "unknown
kconfig symbol" warnings for symbols defined in not-yet-downloaded
components — idf-build-apps treats those as build failures.
Suppress kconfgen output on the intermediate passes; only the final
pass against the converged set emits warnings.
The __init_project_configuration() function in cmakev2's project.cmake
unconditionally applied app-level compiler optimization flags based on
CONFIG_COMPILER_OPTIMIZATION_* Kconfig options. When the bootloader
subproject was built with cmakev2, these app-level flags leaked into the
bootloader compile command alongside the correct bootloader-specific
flags from CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_*.
For example, with the default configuration (app: DEBUG, bootloader:
SIZE), the bootloader received both "-Og -fno-shrink-wrap" (from app
config) and "-Os -freorder-blocks" (from bootloader config). While GCC
uses the last -O flag (-Os wins), the stray -fno-shrink-wrap persisted.
Introduce a SET_COMPILER_OPTIMIZATION build property that defaults to
YES when unset. Subprojects that manage their own optimization flags
(like the bootloader) can set this to NO before calling
idf_project_init() to prevent the default optimization flags from being
applied. This keeps project.cmake generic without requiring it to know
about specific subproject types.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_path variable was used in -fmacro-prefix-map and
-fdebug-prefix-map flags but never read from the IDF_PATH build
property, resulting in an empty substitution. This caused full
filesystem paths to leak into .rodata instead of being mapped to /IDF.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When the component manager is disabled via IDF_COMPONENT_MANAGER=0, the
cmakev2 build system still entered the full component manager flow
(__fetch_components_from_registry), which called
__download_managed_component(). That function returned early with
result=0 without creating the expected output file, violating the
post-condition in __download_component_level_managed_components() that
checks result==0 => file exists, causing a fatal error.
Instead of patching the low-level function to write an empty stub file,
this commit properly skips the entire component manager flow when the
manager is disabled:
- Move __init_component_manager() to idf.cmake global initialization
sequence alongside other __init_*() calls, so IDF_COMPONENT_MANAGER
build property is available early.
- Set __SDKCONFIG_ORIG to the real sdkconfig path in __init_kconfig()
as its default value. Previously it was only set inside
__create_sdkconfig_orig_copy() and had a defensive fallback in
__create_base_kconfgen_command(). The default ensures __SDKCONFIG_ORIG
is always valid regardless of whether the component manager runs.
- Guard __create_sdkconfig_orig_copy() with an IDF_COMPONENT_MANAGER
check. The sdkconfig backup exists solely to preserve unknown Kconfig
options from managed components during intermediate kconfgen rounds.
When the manager is disabled, no managed components exist, so the
backup is unnecessary.
- Guard __fetch_components_from_registry() call in project.cmake behind
IDF_COMPONENT_MANAGER == 1. When disabled, only the manifest warning
is issued. No download loop runs, no temp files are created, and no
"Component manager round N..." messages are printed.
- Remove the now-redundant IDF_COMPONENT_MANAGER guard from
__download_managed_component(), since it is only reachable when the
manager is enabled.
Closes https://github.com/espressif/esp-idf/issues/18372
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmake v1, __build_process_project_includes() exports all build
properties as CMake variables before including project_include.cmake
files. cmakev2 was missing this step, causing components like ULP that
reference build properties as CMake variables (e.g. ${SDKCONFIG_HEADER})
to receive empty values.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit introduces a new build property, __OPTIONAL_REQUIRES_MODE,
and uses it to either defer or link immediately, optional requirements
to components that request such linkage via the
idf_component_optional_requires() function in build system v2. The
DEFERRED mode is intended for single-binary projects where in the linking
of optional components happens after the library target is created the
dependency graph is available to the build system, thereby allowing it to
behave like the v1 version of the function.
Made-with: Cursor
When MAPFILE_TARGET is used in idf_build_executable, add the linker
--cref option so the cross-reference table is written to the map file
instead of stdout. Remove the global -Wl,--cref from default link
options in project.cmake so cref is only applied where a map file
is requested.
The sdkconfig file may contain configuration options defined in Kconfig
files of managed components. Since kconfgen runs before the component
manager fetches these components, the Kconfig definitions for managed
component options are not yet available. The kconfgen --output config
flag regenerates sdkconfig from kconfiglib's internal state, which only
knows about options with loaded Kconfig definitions. This causes unknown
options (i.e., those from managed components) to be silently dropped
from sdkconfig during intermediate regeneration rounds.
Note that kconfgen's --config flag (used for reading sdkconfig) only
performs deprecated option name replacement and does NOT drop unknown
options. The problem is exclusively in --output config, which writes a
fresh sdkconfig from the parsed Kconfig tree state.
Fix this by introducing a __SDKCONFIG_ORIG build property that provides
an indirection layer for the --config input path:
- Before the component manager runs: __SDKCONFIG_ORIG points to a copy
of the original sdkconfig (build/sdkconfig.orig), created by the new
__create_sdkconfig_orig_copy() function. This copy preserves all
original options, including those from managed components.
- During intermediate kconfgen runs: --config reads from the preserved
copy (so unknown options survive as input), while --output config
writes to the real sdkconfig (unknown options may be dropped there,
but this is harmless since kconfgen always reads from the copy).
- After the component manager completes: __SDKCONFIG_ORIG is reset to
point to the real sdkconfig and __BASE_KCONFGEN_CMD is rebuilt, so
that subsequent operations (menuconfig, save-defconfig, confserver)
read and write the actual sdkconfig file directly.
The flow is:
__create_sdkconfig_orig_copy()
-> __SDKCONFIG_ORIG = build/sdkconfig.orig
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
__fetch_components_from_registry():
loop:
download_components()
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
if success: break
endloop
-> __SDKCONFIG_ORIG = project/sdkconfig
-> rebuild __BASE_KCONFGEN_CMD
idf_create_menuconfig() / save-defconfig / confserver
-> uses --config project/sdkconfig (the real file)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, idf_build_generate_metadata only accepts binary targets for
which it generates metadata (project_description.json). On Linux
targets, binary images are not generated, but we still need to generate
project_description.json. Extend the current function to accept both
executable and binary targets and ensure project_description.json is
generated when a Linux target is used.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the CONFIG_APP_BUILD_GENERATE_BINARIES option is used to
determine whether binary images will be generated in
idf_project_default. At present, this option is enabled even for the
Linux target, where the esptool_py component is not included, preventing
the generation of binary images. To address this issue, verify if the
target for esptool_py is present.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The IDF_TOOLCHAIN build property is currently incorrectly set to the
default `gcc` value for the linux target, whereas it should be empty.
This misconfiguration causes confusion for components like `soc`, which
adjust toolchain options based on the
IDF_TOOLCHAIN(CONFIG_IDF_TOOLCHAIN_GCC) build property's setting.
When sdkconfig is generated, the IDF_TOOLCHAIN build property is passed
as an environmental variable to kconfgen, and the CONFIG_IDF_TOOLCHAIN
configuration option is set based on this variable. Additionally, the
CONFIG_IDF_TOOLCHAIN_GCC and CONFIG_IDF_TOOLCHAIN_CLANG configuration
options are set accordingly. Subsequently, CONFIG_IDF_TOOLCHAIN_GCC is
used in several places, such as `components/soc/project_include.cmake`,
to configure the toolchain (compiler flags) by invoking functions from
`tools/cmake/toolchain_flags.cmake`, which is included only for
non-linux targets. As a result the configuration fails, because
functions from `tools/cmake/toolchain_flags.cmake` are not available on
linux target.
Since the IDF_TOOLCHAIN cmake cache variable is actually set in the
`tools/cmake/toolchain.cmake` file, the IDF_TOOLCHAIN build property
should be set after the toolchain is initialized in cmakev2's project
initialization. Note that each toolchain file, except for linux,
includes `toolchain.cmake`, which in turn includes
`toolchain_flags.cmake`. This means the IDF_TOOLCHAIN cmake cache
variable is set for every target except linux, because the toolchain
file for linux is empty. As a result CONFIG_IDF_TOOLCHAIN is empty and
CONFIG_IDF_TOOLCHAIN_GCC not set as for cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In some instances, the COMPONENTS_DISCOVERED build property is used to
walk through the component list and obtain component properties, such as
during configuration preparation or the inclusion of project_include
files. Since we know the component interfaces from the
COMPONENT_INTERFACES build property, we can switch to the faster
`__idf_component_get_property_unchecked` method to obtain component properties
in these cases. This change reduces the reconfiguration time by 0.4
seconds on my computer.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit removes picolibc flags from project.cmake file since they
are added by the esp_libc component when project_include.cmake files are
evaluated.
This commit moves the project initialization by moving the
__init_project_configuration() after the sdkconfig is generated and
included. This is because __init_project_configuration() depends on
config options.
The idf_build_generate_depgraph function creates a component dependency
graph in dot (graphviz) format for a specified executable. It uses
existing helper functions from cmakev1, ensuring that the generated dot
files are produced in the same manner as in cmakev1. While adjustments
might be needed in the future if necessary, the current implementation
is intended to offer the same functionality as cmakev1. Similar to
cmakev1, the dot files are only generated only when the
__BUILD_COMPONENT_DEPGRAPH_ENABLED build property is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_create_size_report function allows for the creation of size
report targets based on the generated link map file. The size report
targets are created using the TARGET option name: "<target>",
"<target>-files", and "<target>-components". These size report targets
are added to the idf_default_project with the TARGET set to "size",
resulting in the creation of "size", "size-files", and "size-components"
targets for the default project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_project_init macro is evaluated within the global variable
scope, and currently, there are a few variables still set. Let's ensure
all variables are properly unset to avoid polluting the global variable
namespace.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The link flag for generating the map file is currently added globally to
the link_options. If multiple executables are generated, the link map
file is overwritten by the last created executable. Since cmakev2
supports building multiple executables, the link map file options need
to be set for each executable individually. To address this, add a new
MAPFILE_TARGET option to the idf_build_executable function. When set,
the link map file will be generated by the linker, and a target
specified in the MAPFILE_TARGET option will be created for the map file.
This also splits the idf_project_default macro. Only the
idf_project_init macro needs be called within the global scope, as it
includes the project_include.cmake files and the cmake version of the
configuration. The remaining functionality of the idf_project_default
macro is implemented in a __project_default helper function to avoid
polluting the global variable space.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the `idf_build_generate_metadata` function takes an
executable target as an argument, and the binary name is stored in the
`EXECUTABLE_BINARY` property. This approach is inconvenient because a
single executable might generate multiple binary images, making it
unreliable to store the binary image name in the executable property due
to the N:1 relationship. To address this, pass the binary image target
instead of the executable target to the `idf_build_generate_metadata`
function. This change is facilitated by linking the binary target,
executable target, and library interface targets with the following
properties:
binary:EXECUTABLE_TARGET -> executable:LIBRARY_INTERFACE -> library
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With commit b719292b75 ("refactor(build-system): Simplify flash target creation"),
partition table and bootloader components are correctly adding their
dependency on the flash target, so there is no need for this to be done
in the build system. This removes the temporary workaround.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit adds support for uf2 targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/uf2.cmake.
- Adds the idf_create_uf2() function to create the uf2 targets. This
function now takes the executable as an argument thus allowing the uf2
target to be created per-executable.
- idf_project_default() is updated to create the uf2 targets.
This commit adds support for dfu targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/dfu.cmake file on similar lines as the
tools/cmake/dfu.cmake file.
- Adds the idf_create_dfu() function to create the dfu targets. This
function now takes the executable as an argument thus allowing the dfu
target to be created per-executable.
- idf_project_default() is updated to create the dfu target.
The COMPONENTS and EXCLUDE_COMPONENTS variables are unused in cmakev2. A
deprecation warning will be printed if they are set. This change also
includes a helper functions for printing deprecation warnings.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 dropped generation of prefix_map_gdbinit, reflect this
change also in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, we include numerous functions in the automatically generated
documentation for the build system API. Let's begin with only the
essential functions and gradually add more to the API based on requests
and actual needs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a revised version of the existing __create_confserver_target
function. It creates a specified confserver target for a given
executable. The kconfig_menus.json file, used by the IDEs, is generated
when the confserver starts. This differs from the previous behavior,
where kconfig_menus.json was created globally along with other sdkconfig
formats. The reason for this change is that kconfig_menus.json contains
the Kconfig menu hierarchy and it is not just a flat option-value
format. It needs to accurately reflect which configurations for which
components are included or excluded. The kconfig_menus.json is generated
at `build/kconfig_menus.json`, where IDEs expect it. This means the file
is overwritten every time the server starts by kconfig_menus.json
version for given executable, so only one confserver can run at a time.
This is likely acceptable, as I don't believe it's possible to safely
run multiple instances of confserver due to the potential race
conditions when the sdkconfig file are generated.
In the future, we may include the location of kconfig_menus.json in
project_description.json so it can be easily identified by IDEs for
each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is basically a renamed version of __create_save_defconfig_target.
The creation of the save-defconfig target is done in
`idf_project_default()`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For a specified executable, create a menuconfig target using the name
provided in the TARGET option. The function generates a config.env file
specific to the executable, where Kconfig files for components linked to
the executable and Kconfig files for components not linked to it are
separated. This separation allows for a clear visual distinction between
the configuration of components that are linked and those that are not
linked to the executable within the menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit updates the order of initialization for build system v2. All
non-project initializations are handled in idf.cmake and all
project-specific initializations are now moved to idf_project_init().
Generate the flasher_args.json file for the global flash target for
tools that require it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Renz Christian Bagaporo
Ivan Grokhotkov (bot)andClaude Opus 4.8
Renz Bagaporo
Frantisek Hrbata
Alexey Lapshin
Sudeep Mohanty
Roland Dobai
Roland Dobai
Jan Beran
Peter Dragun