In cmakev1, certain arguments provided to the idf_component_register
function are stored as component properties. These properties are used
to generate the project_description.json file, which other tools rely
on. Since the idf_component_register function is obsolete in cmakev2, we
need to recreate component properties such as INCLUDE_DIRS,
PRIV_INCLUDE_DIRS, REQUIRES, and PRIV_REQUIRES, which were previously
provided by the idf_component_register function in cmakev1. To achieve
this, let's examine the component's real target cmake properties and
reconstruct them to maintain compatibility with cmakev1.
The real target cmake properties may include generator expressions,
which are ignored. This should be acceptable, as they were not addressed
in cmakev1 either, and handling them is likely not feasible.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Return the difference between two lists, meaning the elements that are
present in the first list but not in the second.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function removes entries containing generator expressions
from the given list. The list is modified in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a helper function that returns a list of paths relative to a
given base directory for a list of input paths.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When a linker script file with a .in extension is added using the
target_linker_script function, it is processed with the C preprocessor.
The linker scripts are preprocessed only once, even if they are used in
multiple libraries, because they are the same.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, add the COMPONENT_BUILD_DIR property. This can be used to
store component-specific generated files, such as preprocessed linker
scripts.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `includes` list variable is used without initialization. Initialize
it explicitly.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component has been renamed, update the name in the common
requirements for the cmakev1 components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component can be referenced by multiple identifiers such as the
component name, which is derived from the component directory name,
component target, interface, or aliases. All component properties are
attached to the component interface target, which is also used for
declaring component dependencies. The cmakev2 build system has a
function called __get_component_interface, which is responsible for
identifying the component interface based on the given component
identifier. Since this function is called frequently, it needs to be
reasonably fast.
This introduces a new __idf_component_interface_cache INTERFACE target,
which serves as a mapping cache between component identifiers and the
component interface. The cache for each component is initialized in the
__init_component function, which introduces a component to the build
system. Currently, the component interface search is conducted by
examining the COMPONENTS_DISCOVERED and COMPONENT_INTERFACES lists
stored as build properties. Since the build system is aware of most
component identifiers, such as component name, target, and alias, during
component initialization, it can add mappings between component
identifiers and the component interface to a cache, which is built as
the components are initialized. This cache is used in the
__get_component_interface function instead of looking into the
COMPONENTS_DISCOVERED or COMPONENT_INTERFACES lists. This significantly
speeds up the component interface search and also makes the code much
simpler and more readable.
The component interface cache also completely replaces the existing
component name resolution, which was introduced because of the component
manager, and the cache used for resolved component names. This is
possible because all the necessary information is available during
component initialization when the component interface cache is
populated. The ambiguity of components is resolved based on component
source/priority.
Here is an example of the component interface mapping for the
espressif__led_strip component to the idf_espressif__led_strip interface
target. The component name, without the namespace, is referred to as the
short name. In this example, it is led_strip.
- espressif__led_strip -> idf_espressif__led_strip # name(directory name)
- idf_espressif__led_strip -> idf_espressif__led_strip # interface
- idf::espressif__led_strip -> idf_espressif__led_strip # alias
- _idf_espressif__led_strip -> idf_espressif__led_strip # real target
- led_strip -> idf_espressif__led_strip # short name
When another component with the same espressif__led_strip name is
initialized with a different priority, there is no need to change
anything in the cache, because the mapping stays the same.
The cache must be updated when two components share the same short name
but belong to different namespaces. This situation is likely uncommon.
For instance, consider espressif__led_strip and my__led_strip. If
my__led_strip has a higher priority, the cache is updated to reflect the
short name as follows:
- led_strip -> idf_my__led_strip
If both components have the same priority, the short name mapping for
led_strip is entirely removed. Conversely, if my__led_strip has a lower
priority, no short name mapping is added for it.
The short name is also added to the COMPONENT_SHORT_NAME property of the
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a revised version of the existing __create_confserver_target
function. It creates a specified confserver target for a given
executable. The kconfig_menus.json file, used by the IDEs, is generated
when the confserver starts. This differs from the previous behavior,
where kconfig_menus.json was created globally along with other sdkconfig
formats. The reason for this change is that kconfig_menus.json contains
the Kconfig menu hierarchy and it is not just a flat option-value
format. It needs to accurately reflect which configurations for which
components are included or excluded. The kconfig_menus.json is generated
at `build/kconfig_menus.json`, where IDEs expect it. This means the file
is overwritten every time the server starts by kconfig_menus.json
version for given executable, so only one confserver can run at a time.
This is likely acceptable, as I don't believe it's possible to safely
run multiple instances of confserver due to the potential race
conditions when the sdkconfig file are generated.
In the future, we may include the location of kconfig_menus.json in
project_description.json so it can be easily identified by IDEs for
each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the initial sdkconfig is generated, the
kconfig_menus.json format is also created alongside other formats like
CMake, JSON, or C header. The kconfig_menus.json depends on the Kconfig
hierarchy and cannot be generated globally. It must be generated for
each executable to ensure that included and excluded components are
correctly positioned within the Kconfig menu hierarchy. Remove the
automatic generation of kconfig_menus.json.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current base kconfgen command is stored in the __BASE_KCONFGEN_CMD
build property, and it includes the --env-file option pointing to the
`build/config.env` file. The `build/config.env` file does not separate
component configuration files into included and excluded components,
because it is created before any executable is added and we need it for
the initial sdkconfig generation. The global `build/config.env` can be
used for every sdkconfig format (cmake, json, txt), but not for the
kconfig_menus.json format, because kconfig_menus.json relies on the
Kconfig layout and hierarchy. Let's allow specifying the --env-file for
each kconfgen invocation and remove it from the base kconfgen command.
This allows you to specify an --env-file for each executable, each with
its own config.env file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __run_kconfgen function does not take any arguments. The output
formats and related files are stored in the __KCONFGEN_OUTPUTS_CMD build
property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is basically a renamed version of __create_save_defconfig_target.
The creation of the save-defconfig target is done in
`idf_project_default()`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For a specified executable, create a menuconfig target using the name
provided in the TARGET option. The function generates a config.env file
specific to the executable, where Kconfig files for components linked to
the executable and Kconfig files for components not linked to it are
separated. This separation allows for a clear visual distinction between
the configuration of components that are linked and those that are not
linked to the executable within the menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate the config.env file for the specified executable. The
configuration file will be stored in the build directory, within a
directory named after the executable target name. The kconfigs*.in
files, which are generated by prepare_kconfig_files.py, will also be
stored in this directory.
This function primarily prepares the arguments for the
__create_config_env_file function based on the components linked to the
executable, ensuring that component Kconfig files are stored in the
appropriate kconfigs*.in files, depending on whether the component is
linked to the executable.
The directory where the generated files are stored is added to the
executable CONFIG_ENV_DIR property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __create_config_env_file function generates a configuration file for
the prepare_kconfig_files.py script. Currently, it specifies the paths
for the kconfigs*.in files, which are generated by
prepare_kconfig_files.py, to be located in the build directory. The
kconfigs*.in files, along with the configuration file, must also be
generated for each executable, as each executable may have a
different set of linked components. Set the paths for the
kconfigs*.in files to be generated by the prepare_kconfig_files.py
script in the same directory as the configuration file.
Also allow explicit specification of which component configuration files
should be placed in different kconfigs*.in files. For sdkconfig
generation, it is not relevant whether a component's Kconfig file is
placed in the kconfigs.in or kconfigs_excluded.in file. This distinction
is only important for visually separating the configuration of included
and excluded components in the menuconfig. For the main sdkconfig, all
component Kconfig files are placed in the kconfigs.in file, and the
generated kconfigs*.in files are stored in the build directory. For
executables, the generated kconfigs*.in files will be stored in separate
directories for each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The excluded Kconfig files cannot be collected globally, they must be
collected per executable. The distinction between included and excluded
Kconfig files is relevant only for menuconfig, which is specific to each
executable, as each executable may have a different set of linked
components. For the global sdkconfig file and its CMake and JSON
variants, whether the Kconfig files are excluded or not is irrelevant.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Commit 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
introduced a temporary workaround for the cmakev2 testing
application by including the cmakev1 project.cmake file after the
return() statement. The issue is that some tests, such as
test_build_fail_on_build_time, append commands to the project's
CMakeLists.txt. These commands are placed after the newly added return()
statement, causing them to be ignored and resulting in test failures. To
resolve this, include the cmakev1 project.cmake within the cmake block
comment.
Fixes: 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Three tests are disabled for cmakev2:
* test_build_dfu
* test_build_uf2
* test_build_loadable_elf
and one is modified:
* test_build_skdconfig_phy_init_data
the other 14 tests should pass.
The tests, test_build_dfu and test_build_uf2, are temporarily disabled
because cmakev2 currently does not include the dfu and uf2 targets.
These tests should be re-enabled once support is added to cmakev2.
The test_build_loadable_elf is disabled because the bootloader_support
component determines its requirements based on the sdkconfig values,
specifically the CONFIG_APP_BUILD_TYPE_RAM used in this test. When
CONFIG_APP_BUILD_TYPE_RAM is set, bootloader_support declares a
dependency on micro-ecc. This is wrong even in cmakev1 and should be
fixed.
The test_build_skdconfig_phy_init_data test requires the esp_phy
component but does not specify it in its requirements. It relies on the
cmakev1 behavior, where all components are included in the build by
default. This approach does not work for cmakev2. To resolve this, make
the dependency on esp_phy explicit, ensuring compatibility with both
versions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the cmakev2 implementation of Kconfig name consistency has a
slightly different warning message compared to cmakev2, causing the test
`test_build.py::test_build_with_misspelled_kconfig` to fail. Make the
warning messages consistent with cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The test_build_with_misspelled_kconfig test expects a KConfig.projbuild
file in the main component. However, the file is correctly named
Kconfig.projbuild, which causes the test to fail for cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, cmakev2 evaluates each component using add_subdirectory with
the EXCLUDE_FROM_ALL option. The intention was to exclude all components
from being built by default unless they are explicitly linked to an
executable based on dependencies. This approach aims to avoid building
components that are included but not actually linked to the executable.
However, this has the side effect of preventing components from adding
their custom targets to the "all" target, which is used, for example, by
esp_phy. Generally, we should not restrict components from adding
targets to "all". Since components are only included if explicitly
requested by default, removing the EXCLUDE_FROM_ALL option should be
acceptable.
The downside is that if IDF_INCLUDE_ALL_COMPONENTS is set and the user
runs `idf.py build`, all components will be built, even if they are not
linked to the executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the check is based on the existence of sdkconfig, but there
may be situations where sdkconfig exists without containing
CONFIG_IDF_TARGET. Perform a target consistency check only if the target
is identified in the main sdkconfig file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add stump files for the build system v2 documentation with automatic API
generation from CMake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Load the esp_docs_cmakev2_extension to process documentation comments
from CMake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a Sphinx extension that introduces a new `cmakev2` domain with
multiple directives, allowing for the automatic extraction of
documentation comments from CMake files and their inclusion in the
Sphinx-generated documentation.
Directives:
- `cmakev2:include`: The included CMake file is processed for
documentation comments within the `#[[api` and `#]]` marks, which
should contain valid reStructuredText markup.
- `cmakev2:function`: Creates a CMake function node. All function nodes
are sorted by name and placed into the `_cmakev2_functions` section.
- `cmakev2:macro`: Creates a CMake macro node. All macro nodes are
sorted by name and placed into the `_cmakev2_macros` section.
- `cmakev2:variable`: Describes a CMake variable node. All variable
nodes are sorted by name and placed into the `_cmakev2_variables`
section.
Each node can be referenced with `` :cmakev2:ref:`<node name>` ``, where
the node name is the function, macro, or variable name as used in the
related directive.
Example:
CMake file:
```
#[[api
.. cmakev2:function:: idf_flash_binary
#]]
```
This function can be referenced with `` :cmakev2:ref:`idf_flash_binary` ``
and will be placed in the `.. _cmakev2_functions:` section.
The extension is currently located in esp-idf, but in the future, we
should consider moving it to esp-docs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The build system should include the file generated by the component
manager after downloading components even if the component manager
requested a re-run due to a missing Kconfig.
This commit updates the order of initialization for build system v2. All
non-project initializations are handled in idf.cmake and all
project-specific initializations are now moved to idf_project_init().
Generate the flasher_args.json file for the global flash target for
tools that require it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate binary images and add flash targets for hello_world_example and
fatfs_example.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
All other functions, such as idf_build_binary, use OUTPUT_FILE. This is
more descriptive and provides more flexibility to add additional
file-related options in the future if necessary. Make
idf_build_generate_metadata consistent with the other function
interfaces by using OUTPUT_FILE.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Use idf_build_binary, idf_sign_binary, and idf_flash_binary to generate
and flash the default project application.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Given the binary target generated by the idf_build_binary or
idf_sign_binary function, add a new POST_BUILD dependency that displays
a message indicating the binary image must be manually signed before
flashing. This situation arises when CONFIG_SECURE_SIGNED_APPS is
enabled, but CONFIG_SECURE_BOOT_BUILD_SIGNED_BINARIES is not set. As a
result, the binary image is not signed during the build process and
requires manual signing.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Given the binary target created by the idf_build_binary or
idf_sign_binary function, add a new dependency that verifies whether the
binary image fits within the smallest application partition.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a new flash target for a previously created binary target using
the idf_build_binary or idf_sign_binary function. Specify the new target
name with the TARGET option. The binary image path is stored in the
BINARY_PATH property of the binary target and is set when the binary
target is created using the idf_build_binary or idf_sign_binary
function. Use the NAME option as a prefix for the generated esptool
argument file, and the FLASH option can be used to include the binary
image flashing in the global flash target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a signed binary image from a previously generated binary using
the `idf_build_binary` function. This image can be signed with an
optional key from a KEYFILE or, by default, with the key specified in
the sdkconfig. A new custom command is added to generate the signed
binary, which is stored at the location specified by the OUTPUT_FILE
option and a new custom target is created for it, with the name provided
in the TARGET option. The BINARY_PATH property is added to this new
TARGET, containing the path to the signed binary image.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_binary` function generates a binary image for a specified
executable target, which may or may not have been created using the
`idf_build_executable` function. It adds a custom command to generate
the binary image file at the location specified by the `OUTPUT_FILE`
option and creates a new target for it, with the name specified by the
`TARGET` option. The path of the generated binary file is also stored in
the `BINARY_PATH` property of the newly created binary target.
Additionally, this binary path is stored in the `EXECUTABLE_BINARY`
property of the given executable. This information is used by the
`idf_build_generate_metadata` function, which takes the executable
target as an argument and includes the binary name in the
`project_description.json`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The global flash targets are created using
__create_project_flash_targets, but these are merely placeholders that
allow components to declare their dependencies on the global flash or
encrypted-flash target in their project_include.cmake. The actual logic
for flash targets is located in the project_include.cmake of the
esptool_py component. Therefore, the targets must be initialized
after this file has been included. To address this, a new function,
__init_project_flash_targets, has been added for this purpose.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a phony target that intentionally fails the build, displaying the
error messages specified in the message arguments. An empty CMake file is
created and included, which is then deleted when the target is executed.
This forces CMake to rerun, as the file is added to the CMake rerun
dependencies.
This has been carried over from cmakev1 as it is used by several
components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the lwip is linked to mbedtls if CONFIG_LWIP_ENABLE is set.
This approach works in cmakev1 because only the configuration for
BUILD_COMPONENTS is available during component evaluation. However, this
is not the case in cmakev2, where the configuration for all components
is available. This means that even if CONFIG_LWIP_ENABLE is set, it does
not guarantee that the lwip component is included in the build. For
cmakev2, modify the check to use a generator expression to ensure that
lwip is linked to mbedtls only if lwip is actually included in the
build.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
