The ULP sources include sdkconfig.h for compile-time configuration
values. The sdkconfig directory was previously available to the ULP
target indirectly through the COMPONENT_INCLUDES variable, which is
populated from the parent component's INTERFACE_INCLUDE_DIRECTORIES.
In cmake v1, idf_component_register() adds the config_dir as PUBLIC
include directory to every component (component.cmake:498), so it
ends up in INTERFACE_INCLUDE_DIRECTORIES and gets passed to the ULP
subproject via COMPONENT_INCLUDES.
In cmakev2, the config_dir is added as a build-level property and
applied to components as PRIVATE (component.cmake:1053), so it no
longer appears in INTERFACE_INCLUDE_DIRECTORIES. As a result, the
sdkconfig directory is missing from the ULP target's include paths.
Add the sdkconfig directory explicitly to the ULP target's include
directories. Note that SDKCONFIG_HEADER is already passed to the ULP
subproject and its directory is already extracted into sdkconfig_dir
for linker script preprocessing (IDFULPProject.cmake:40).
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmake v1, __build_process_project_includes() exports all build
properties as CMake variables before including project_include.cmake
files. cmakev2 was missing this step, causing components like ULP that
reference build properties as CMake variables (e.g. ${SDKCONFIG_HEADER})
to receive empty values.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev2 kconfig module sets sdkconfig output paths using internal
property names (__SDKCONFIG_HEADER, __SDKCONFIG_CMAKE, etc.), but
components like ULP read the public names (SDKCONFIG_HEADER,
SDKCONFIG_CMAKE). This results in empty values being passed to the ULP
sub-project, causing its CMake configure step to fail.
Add public aliases matching the cmake v1 property names for backward
compatibility.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
CMakev2's DEFERRED mode for idf_component_optional_requires only links
optional components already present in the dependency graph. This caused
esp_coex headers to be unavailable when building minimal examples on
targets with both WiFi and IEEE802154 (e.g. esp32c5, esp32c6).
Moving init_coexist to esp_coex removes the cross-component dependency
and ensures the init function is only compiled when esp_coex itself is
part of the build. A linker hook is added to guarantee the object file
is not silently discarded by the linker.
Made-with: Cursor
This commit introduces a new build property, __OPTIONAL_REQUIRES_MODE,
and uses it to either defer or link immediately, optional requirements
to components that request such linkage via the
idf_component_optional_requires() function in build system v2. The
DEFERRED mode is intended for single-binary projects where in the linking
of optional components happens after the library target is created the
dependency graph is available to the build system, thereby allowing it to
behave like the v1 version of the function.
Introduce a callback mechanism that lets components register CMake
functions to be called at specific points in the build lifecycle.
Currently, this framework only supports registering callbacks to be
called after the executable target is created, i.e, the POST_ELF phase
of the build but before the binary target is created.
This commit updates the dependencies list for the esp_wifi component
when compiled for non-WiFi targets. The reason for this change is
because cmakev2 explicitly includes the esp_wifi component even on
non-WiFi targets leading to compilation errors due to missing
dependencies.
This commit adds a new example at
examples/build_system/cmakev2/features/import_lib_direct to
demonstrate the cmakev2 ability to integrate with external CMake
projects easily.
This commit adds a new example at
examples/build_system/cmakev2/features/multi_binary to
demonstrate the cmakev2 ability to build multiple binaries in a single
CMake project.
This commit adds a new example at
examples/build_system/cmakev2/features/conditional_components to
demonstrate the cmakev2 ability to conditionally include components in
the build based on Kconfig options.
This commit updates the examples/build_system/cmake/plugins
example for the new build system and adds the newly created example at
examples/build_system/cmakev2/features/plugins.
This commit updates the examples/build_system/cmake/multi_config
example for the new build system and adds the newly created example at
examples/build_system/cmakev2/features/multi_config.
This commit updates the examples/build_system/cmake/import_prebuilt
example for the new build system and adds the newly created example at
examples/build_system/cmakev2/features/import_prebuilt.
This commit updates the examples/build_system/cmake/import_lib
example for the new build system and adds the newly created example at
examples/build_system/cmakev2/features/import_lib.
This commit updates the examples/build_system/cmake/component_manager
example for the new build system and adds the newly created example at
examples/build_system/cmakev2/features/component_manager.
This commit updates the examples/build_system/cmake/idf_as_lib
example for the new build system and adds the newly created example at
examples/build_system/cmakev2/features/idf_as_lib.
Add idf_build_get_compile_options() to aggregate COMPILE_OPTIONS,
C_COMPILE_OPTIONS, CXX_COMPILE_OPTIONS, and ASM_COMPILE_OPTIONS build
properties with generator expressions. Replace internal
__get_compile_options(OUTPUT ...) usage in idf_component_register and
idf_component_include with the new public function.
When MAPFILE_TARGET is used in idf_build_executable, add the linker
--cref option so the cross-reference table is written to the map file
instead of stdout. Remove the global -Wl,--cref from default link
options in project.cmake so cref is only applied where a map file
is requested.
This commit updates the examples/get-started/hello_world example for the
new build system and adds the newly created example at
examples/build_system/cmakev2/get_started/hello_world.
Currently, embedded files can be added in two ways. First, by calling
the `target_add_binary_data` function directly within a component.
Second, by passing `EMBED_FILES` or `EMBED_TXTFILES` to
`idf_component_register`, or in cmakev2 by setting the `EMBED_FILES` or
`EMBED_TXTFILES` component properties.
The source file for an embedded file is generated using
`add_custom_command`. When the embedded file is added directly in the
component's CMakeLists.txt file by using the `target_add_binary_data`
function, the `add_custom_command` command is evaluated in the same
directory context where the component target is created. As a result,
the generated embedded file can be used automatically as a file
dependency in component's sources.
However, when an embedded file is added via `idf_component_register` or
by setting the component property, the call to `add_custom_command`
inside `target_add_binary_data` occurs after the component has been
evaluated, specifically after the `add_subdirectory` call, within
`idf_component_include`. This means it is not created in the same
directory context as the component's target. In this case, the embedded
file dependency is not added to the component's target, and the embedded
file is not generated. This behavior is described in the
`add_custom_command` documentation [1].
> A target created in the same directory (CMakeLists.txt file)
> that specifies any output of the custom command as a source
> file is given a rule to generate the file using the command at
> build time.
To fix this issue, an explicit custom target for the generated embedded
file is created and added as a dependency of the component's target,
ensuring that the file is generated correctly.
[1] - https://cmake.org/cmake/help/latest/command/add_custom_command.html
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev2, a component can be evaluated within the context of another
component, so it's important to properly initialize each variable used
by the component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Pytest's default import mode (prepend/rootdir) imports test modules as
top-level modules keyed by filename. When two directories contain test
files with the same basename (e.g. test_sdkconfig.py in both
test_build_system/ and test_build_system/buildv2/), pytest tries to
register both as the module name "test_sdkconfig". The second collection
fails with "import file mismatch" because the module object already
cached in sys.modules points to the first file.
Adding __init__.py to the buildv2/ directory makes it a proper Python
package. Pytest then imports its test modules under the package
namespace (buildv2.test_sdkconfig), which is distinct from the
top-level test_sdkconfig, resolving the collision.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The sdkconfig file may contain configuration options defined in Kconfig
files of managed components. Since kconfgen runs before the component
manager fetches these components, the Kconfig definitions for managed
component options are not yet available. The kconfgen --output config
flag regenerates sdkconfig from kconfiglib's internal state, which only
knows about options with loaded Kconfig definitions. This causes unknown
options (i.e., those from managed components) to be silently dropped
from sdkconfig during intermediate regeneration rounds.
Note that kconfgen's --config flag (used for reading sdkconfig) only
performs deprecated option name replacement and does NOT drop unknown
options. The problem is exclusively in --output config, which writes a
fresh sdkconfig from the parsed Kconfig tree state.
Fix this by introducing a __SDKCONFIG_ORIG build property that provides
an indirection layer for the --config input path:
- Before the component manager runs: __SDKCONFIG_ORIG points to a copy
of the original sdkconfig (build/sdkconfig.orig), created by the new
__create_sdkconfig_orig_copy() function. This copy preserves all
original options, including those from managed components.
- During intermediate kconfgen runs: --config reads from the preserved
copy (so unknown options survive as input), while --output config
writes to the real sdkconfig (unknown options may be dropped there,
but this is harmless since kconfgen always reads from the copy).
- After the component manager completes: __SDKCONFIG_ORIG is reset to
point to the real sdkconfig and __BASE_KCONFGEN_CMD is rebuilt, so
that subsequent operations (menuconfig, save-defconfig, confserver)
read and write the actual sdkconfig file directly.
The flow is:
__create_sdkconfig_orig_copy()
-> __SDKCONFIG_ORIG = build/sdkconfig.orig
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
__fetch_components_from_registry():
loop:
download_components()
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
if success: break
endloop
-> __SDKCONFIG_ORIG = project/sdkconfig
-> rebuild __BASE_KCONFGEN_CMD
idf_create_menuconfig() / save-defconfig / confserver
-> uses --config project/sdkconfig (the real file)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For Build System v2 on linux target, esp_timer dependency must be
conditional on CONFIG_LWIP_IPV4 rather than checking BUILD_COMPONENTS.
v2 uses configuration-driven dependencies.
For Build System v2 on linux target, lwip dependency and ESP_TLS_WITH_LWIP
definition must be conditional on CONFIG_LWIP_ENABLE rather than checking
BUILD_COMPONENTS. v2 uses configuration-driven dependencies.
When CONFIG_APP_BUILD_TYPE_RAM is set (loadable ELF app build),
bootloader_support was adding micro-ecc to priv_requires. The micro-ecc
component lives in the bootloader subproject and is not available in
normal app builds with cmakev2, causing component resolution to fail
when built for a build system v2 project.
micro-ecc is only needed for secure boot sources built under
BOOTLOADER_BUILD (secure_boot_signatures_bootloader.c). For app builds
with CONFIG_APP_BUILD_TYPE_RAM we do not build those sources, so
only add micro-ecc to priv_requires when BOOTLOADER_BUILD is set.
The idf_component_optional_requires compatibility function in cmakev2
currently attempts to mimic the behavior of cmakev1 by adding a
dependency only if a component is already included in the build. In
cmakev1, this is handled by checking the BUILD_COMPONENTS list, which is
created during an early evaluation phase. Because cmakev2 removed this
early phase to allow for configuration-based component dependencies, the
build system does not inherently know which components will be part of
the build beforehand. To compensate, the current compatibility function
relies on the TARGET_EXISTS generator expression to determine if a
component should be linked.
This approach poses a problem because generator expressions are not
evaluated until the generation phase, but cmakev2 processes linker
scripts during the configuration phase by recursively scanning targets
linked to a library. Because the scanner does not recognize and cannot
evaluate generator expressions, any component linked optionally through
generator expression is skipped. If that component carries a linker
script, the script is never added to the library interface, resulting in
build failures. Since cmakev2 aims to support multiple libraries, a
component might also exist globally, causing TARGET_EXISTS to evaluate
to true, yet still be missed during a specific library's recursive scan,
leading to the same omission of necessary linker scripts.
To resolve this, the implementation of idf_component_optional_requires
is changed to check if a component is known to the build system in
COMPONENTS_DISCOVERED build property. If so, it is explicitly included
and linked directly via its interface target. While this may pull more
components into a build than the previous generator expression method,
potentially increasing build times, it ensures that dependency trees are
fully visible to the library target scanner. This serves as a practical
middle ground that maintains compatibility with existing components and
ensures build stability. This approach allows for the gradual fixing of
idf_component_optional_requires usage in components for cmakev2, with
the eventual goal of removing its usage in cmakev2 entirely.
This change also fixes cases where idf_component_optional_requires is
used to conditionally add requirements based on configuration options.
In cmakev2, the presence of a configuration option does not guarantee[1]
that a component has been included via add_subdirectory, unlike the
behavior in cmakev1. With this change even constructions like
```cmake
if(CONFIG_VFS_SUPPORT_IO)
idf_component_optional_requires(PRIVATE vfs)
endif()
```
will now work in cmakev2, as the updated idf_component_optional_requires
explicitly includes the required component if it is available to the
build system.
Closes https://github.com/espressif/esp-idf/issues/18133
[1] https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/build-system-v2.html#id9
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a simple test to verify that the buildv2_test_app can be built for
the Linux target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, cmakev2 is being tested only in backward-compatible mode by
using the existing cmakev1 tests with the cmakev2 test application. We
also need to add tests specific to cmakev2, and it is convenient to
reuse the existing build system testing framework. Let's add a `buildv2`
subdirectory to the existing `tools/test_build_system` directory and use
the `pytest_collection_modifyitems` hook to ignore tests in this
directory unless the `--buildv2` option is used.
Without the `--buildv2` option, only the existing cmakev1 tests are
executed and tests in `buildv2` directory are skipped. With the
`--buildv2` option, the existing cmakev1 tests run with the cmakev2
testing application for backward compatibility testing, and all cmakev2
tests within the `buildv2` subdirectory are also executed.
Note: we cannot use the `pytest_ignore_collect` hook, because the
`--buildv2` option is not known to the pytest, so the
`config.getoption('--buildv2', False)` returns always False. We would
likely need to add the `--buildv2` option in the conftest.py in the
esp-idf root directory.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
fix: lsadjf las jflasjfl aslfsald asl fsadlf sladsal jfsadfas
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This adds a clear header to the pytest output, indicating which build
system version is currently being tested.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev1, the initial order of components linked to the executable is
determined by the `__build_expand_requirements` function and stored as a
list in the `BUILD_COMPONENT_ALIASES` build property. Later, the
components from `BUILD_COMPONENT_ALIASES` are linked to the executable
using `target_link_libraries`. The issue is that
`BUILD_COMPONENT_ALIASES` contains components in a "reversed" order
compared to how they would typically appear on the link command line. In
other words, components (archives) deeper in the dependency chain are
placed first. As a result, cmake has to repeat the archives in the link
command more times than is actually necessary. This has the beneficial
side effect of placing libfreertos.a before libmain.a, allowing the
`app_main` symbol to be resolved.
In cmakev2, there is no predefined order because it does not perform
early evaluation. Instead, the entire link command is created based on
actual component dependencies using `target_link_libraries`.
Consequently, freertos must set the `app_main` symbols as undefined.
This is already done for non-Linux targets, but it must also be done for
the Linux target.
Another issue arises with macOS, which mangles C symbols by prepending
an underscore. We cannot use the same method as in
`components/linux/assert_func.c` for `__assert_func`, so for macOS,
`_app_main` is used. In the future, if more undefined symbols need to be
added for the Linux target, we should consider introducing a helper.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, idf_build_generate_metadata only accepts binary targets for
which it generates metadata (project_description.json). On Linux
targets, binary images are not generated, but we still need to generate
project_description.json. Extend the current function to accept both
executable and binary targets and ensure project_description.json is
generated when a Linux target is used.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some components simply return when included in a build for the Linux
target. This is silently ignored in the `idf_component_include`, and the
component interface target is linked as requirement in
`idf_component_register`. This has the side
effect of such components being reported as included, with their
configuration displayed in the menuconfig for included components,
rather than in the submenu for excluded components. For example, the
`bt` component, if added as a dependency in `idf_component_register`,
will be displayed in menuconfig as an included component for Linux
target. The `idf_component_include` function sets the component's
`COMPONENT_REAL_TARGET` property to `NOTFOUND` in such situations. Use
this information when config.env is generated put such components into
excluded submenu. Note that we cannot avoid linking empty component interface
into library, because there might be recursive dependencies and at the
time when the component is included we might not now if it has a real
target or not.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the CONFIG_APP_BUILD_GENERATE_BINARIES option is used to
determine whether binary images will be generated in
idf_project_default. At present, this option is enabled even for the
Linux target, where the esptool_py component is not included, preventing
the generation of binary images. To address this issue, verify if the
target for esptool_py is present.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev2, the configuration for all components is included because the
complete configuration is generated very early, before any component is
actually evaluated. This poses a problem for the Linux target, which
does not have a controller Kconfig.in file, causing kconfgen to fail due
to the missing include. To address this, let's use the osource command,
which does not include the referenced Kconfig file if it does not exist.
Closes https://github.com/espressif/esp-idf/issues/18142
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Sudeep Mohanty
Frantisek Hrbata
Marius Vikhammer