- Added more log for BLE
- Support BLE log filter and simeple output
- Support BLE log compression
(cherry picked from commit 03758ba550)
Co-authored-by: chenjianhua <chenjianhua@espressif.com>
Currently, the toolchain CMake files use the remove_duplicated_flags
function from utilities.cmake. The cmakev2 implementation also includes
this function for backward compatibility. Move the
remove_duplicated_flags function to a separate file,
deduplicate_flags.cmake, so it can be shared between cmakev1 and
cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Enable check-kconfig-files. It was temporarily disabled because it was
not possible to add a new sourced file through an environment variable,
as kconfcheck had a fixed list of allowed environment variable names.
This issue should now be resolved.
Currently, the buildv2 tests are initiated using the same patterns as
the tests for the current build system. This means that any change in
the current build system will also trigger the buildv2 tests. Initially,
it might be wise not to block the CI for changes in the current build
system in case there is an issue with buildv2. Therefore, let's
explicitly start the buildv2 tests only when the buildv2 label is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The esp_libc relies on the stdio implementation of the vfs component if
CONFIG_VFS_SUPPORT_IO is enabled. This is not an issue in cmakev1
because if the vfs component is not included in the project build, its
configuration is not available, and CONFIG_VFS_SUPPORT_IO is not set.
However, in cmakev2, the configuration for all components is available,
and the presence of some component configuration options does not
necessarily mean that the component is included in the project build.
When esp_libc is compiled without the vfs dependency but with
CONFIG_VFS_SUPPORT_IO enabled, the libc initialization will encounter a
panic due to a NULL pointer dereference (fp->_flags) in __swsetup_r, as
fopen in esp_libc_init_global_stdio will return NULL.
Although the current documentation exposes only limited functionality,
which is unlikely to change, add an explicit note that the build system
v2 is currently a technical preview intended for testing and early
evaluation.
Ensure consistency in referring to v2 throughout the documentation. In
some places, it is referred to as V2 or Version 2; unify this to v2.
Additionally, update the hello_world example for v2 to use the
idf::spi_flash alias instead of the interface variable to maintain
consistency with the rest of the documentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Extend the current documentation to recommend using the
idf_component_register function as the preferred method for creating a
new component for v2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Using idf_component_register is the preferred method for creating new
components for cmakev2. This approach ensures compatibility with both
versions of the build system. The KCONFIG and KCONFIG_PROJBUILD options
have been removed from the API documentation, but are retained in code
in case a cmakev1 component uses these options to warn about
incompatibility. Also remove a note about `project_include.cmake`,
because cmakev2 includes all project_include files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Enhance the `esp_target_info` component example by adding some of
the missing features that the component might require. This includes
adding a linker script, a linker fragment, linking the component archive
as a whole library, and providing an example of how the linker options
can be set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Components designed exclusively for cmakev2 are not compatible with
cmakev1 due to the lack of forward compatibility. It is important to
clarify this at the very beginning of the `Creating a New Component`
section. Additionally recommended a guidance on how to create a
component that will be compatible with both versions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the archive file name is set only in the cmakev1
idf_component_register shim. The predictable component archive file name
is important, for example, for usage in linker fragments. Ensure that
the cmakev2 component also has the archive file name set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev2 component sources are set, the SRCS target
property is used. This is obviously wrong because the correct CMake
property is SOURCES.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Rename the LINKER_SCRIPTS_STATIC component property to LINKER_SCRIPTS.
This property stores linker scripts that are not processed by ldgen,
which essentially includes all of them. The only linker script processed
by ldgen is sections.ld, which is handled by the esp_system component.
This implies that there is likely no practical use case for other
components to utilize ldgen processed linker scripts. This change is
purely cosmetic to allow components to add linker scripts with:
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS linker_script.ld APPEND)
instead of
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS_STATIC linker_script.ld APPEND)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add very basic documentation. It is currently written with the
expectation that the reader is already familiar with the existing build
system and focuses more on the changes that might be necessary to adapt
existing components to work with cmakev2. Additionally, it provides
basic information on how to create a new component and project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, we include numerous functions in the automatically generated
documentation for the build system API. Let's begin with only the
essential functions and gradually add more to the API based on requests
and actual needs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit adds a workaround in idf_build_component shim to avoid
initialization of a duplicate managed component. The component manager
when running on a Windows system returns duplicate managed components
upon successive invocations.
The current approach involves wrapping the library target within the
library interface target with the whole-archive flags, such as for the
GCC linker. However, this does not work as expected because the library
target is expanded, and the whole-archive flags are also applied to
other library targets that the wrapped library target depends on. IOW
the whole-archive flags surround multiple archives, not just the one
requrested. Generally, using linker flags like whole-archive in the
library INTERFACE(INTERFACE_LINK_LIBRARIES) does not seem to work as
CMake may perform deduplication and rearrange the flags [1].
The proof of concept used `CMAKE_LINK_LIBRARY_USING_<FEATURE>` with the
WHOLE_ARCHIVE feature, which was introduced in CMake 3.24 to specify how
the library target should be linked. There are two issues with this.
First, this feature is only available from CMake 3.24 onwards, while our
minimum CMake version is set to 3.22. More importantly, all occurrences
of a library on the link line are wrapped. For example, if a library
like `vfs` appears multiple times on the link line due to dependencies,
each occurrence will be wrapped with whole-archive, causing the linker
to complain about multiple symbol definitions. Therefore, even though
WHOLE_ARCHIVE is recommended for handling whole-archive linkage, it does
not seem suitable for our purposes. This was overlooked in the PoC
because only a simple testing component was used to check the
WHOLE_ARCHIVE behavior.
One way to address this issue is to adopt the same approach used in
cmakev1, which involves specifying the whole-archive flags when linking
component library targets to the final executable. In this case, CMake
retains the flags without any alteration, unlike when the flags are
specified in INTERFACE_LINK_LIBRARIES for the component interface
targets. While this approach is feasible, it would alter the current
logic, where we have a single library interface for the entire idf
library that can be linked to the executable. This change would also
complicate the direct use of the idf library, as it would no longer be
possible to simply link it to the executable. Instead, the executable
would need to correctly link the component libraries with the
whole-archive flags. We could encapsulate this process within the
idf_build_executable function, but projects that only use
idf_build_library would need to implement the same solution.
It appears possible to address this issue by using target_link_options
instead. This allows to specify exactly what should appear on the link
command line. One side effect of this approach is that the library
appears on the link command line multiple times: first when used with
target_link_options and second when used in target_link_libraries.
However, this does not seem to pose a problem. The flags specified with
target_link_options appear on the command line first, ensuring that the
whole-archive is prioritized, and the component archive libraries may be
repeated on the link line anyway due to dependencies. Essentially, the
final link command line is very similar to the one from cmakev1. We may
revisit this approach in the future, but for now, it seems to work as
expected.
[1] https://gitlab.kitware.com/cmake/cmake/-/issues/20078
[2] https://cmake.org/cmake/help/latest/variable/CMAKE_LINK_LIBRARY_USING_FEATURE.html
[3] https://discourse.cmake.org/t/automatically-wrapping-a-static-library-
in-whole-archive-no-whole-archive-when-used-during-linking/5883
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev1 properties for INCLUDE_DIRS and
PRIV_INCLUDE_DIRS are constructed in the
__set_component_cmakev1_properties function, the return values from the
get_target_property function for INCLUDE_DIRECTORIES and
INTERFACE_INCLUDE_DIRECTORIES properties are not checked. If a component
target does not set e.g. INCLUDE_DIRECTORIES property,
get_target_property will return a value such as `include_dirs-NOTFOUND`.
This value is subsequently passed to __get_relative_paths in the PATHS
argument, causing the file(RELATIVE_PATH) call in __get_relative_paths
to fail with an error.
```
CMake Error at /home/fhrbata/work/esp-idf/tools/cmakev2/utilities.cmake:235 (file):
file RELATIVE_PATH must be passed a full path to the file:
include_dirs-NOTFOUND
```
Fix this by explicitly set the PATHS to an empty list before passing it
to the __get_relative_paths function if the property is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
idf_build_executable() creates a dummy source file to create the
executable target. Since this file is created unconditionally everytime
CMake configuration happens, a source file change is registered which
causes CMake to execute other targets even though no other source file
changed.
The commit adb2d5deee ("feat(cmake): Produce warnings when component dependen..")
introduced additional checks for source files and include directories
used by a component that are located outside the component's directory.
If these files and directories belong to another component, a warning is
issued. This feature has not yet been implemented in cmakev2, so related
tests are temporary disabled.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit renames the buildv2_test_app.c file to build_test_app.c in
the buildv2_test_app template to enable tests to refer to the same
source file name when writing tests for both cmakev1 and cmakev2.
This commit enables the following tests for cmakev2 in CI which do not
need any updates:
- test_bootloader.py
- test_git.py
- test_kconfig.py
- test_partition.py
- test_reproducible_build.py
- test_sdkconfig.py
In cmakev1, certain arguments provided to the idf_component_register
function are stored as component properties. These properties are used
to generate the project_description.json file, which other tools rely
on. Since the idf_component_register function is obsolete in cmakev2, we
need to recreate component properties such as INCLUDE_DIRS,
PRIV_INCLUDE_DIRS, REQUIRES, and PRIV_REQUIRES, which were previously
provided by the idf_component_register function in cmakev1. To achieve
this, let's examine the component's real target cmake properties and
reconstruct them to maintain compatibility with cmakev1.
The real target cmake properties may include generator expressions,
which are ignored. This should be acceptable, as they were not addressed
in cmakev1 either, and handling them is likely not feasible.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Return the difference between two lists, meaning the elements that are
present in the first list but not in the second.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function removes entries containing generator expressions
from the given list. The list is modified in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a helper function that returns a list of paths relative to a
given base directory for a list of input paths.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When a linker script file with a .in extension is added using the
target_linker_script function, it is processed with the C preprocessor.
The linker scripts are preprocessed only once, even if they are used in
multiple libraries, because they are the same.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, add the COMPONENT_BUILD_DIR property. This can be used to
store component-specific generated files, such as preprocessed linker
scripts.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `includes` list variable is used without initialization. Initialize
it explicitly.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component has been renamed, update the name in the common
requirements for the cmakev1 components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component can be referenced by multiple identifiers such as the
component name, which is derived from the component directory name,
component target, interface, or aliases. All component properties are
attached to the component interface target, which is also used for
declaring component dependencies. The cmakev2 build system has a
function called __get_component_interface, which is responsible for
identifying the component interface based on the given component
identifier. Since this function is called frequently, it needs to be
reasonably fast.
This introduces a new __idf_component_interface_cache INTERFACE target,
which serves as a mapping cache between component identifiers and the
component interface. The cache for each component is initialized in the
__init_component function, which introduces a component to the build
system. Currently, the component interface search is conducted by
examining the COMPONENTS_DISCOVERED and COMPONENT_INTERFACES lists
stored as build properties. Since the build system is aware of most
component identifiers, such as component name, target, and alias, during
component initialization, it can add mappings between component
identifiers and the component interface to a cache, which is built as
the components are initialized. This cache is used in the
__get_component_interface function instead of looking into the
COMPONENTS_DISCOVERED or COMPONENT_INTERFACES lists. This significantly
speeds up the component interface search and also makes the code much
simpler and more readable.
The component interface cache also completely replaces the existing
component name resolution, which was introduced because of the component
manager, and the cache used for resolved component names. This is
possible because all the necessary information is available during
component initialization when the component interface cache is
populated. The ambiguity of components is resolved based on component
source/priority.
Here is an example of the component interface mapping for the
espressif__led_strip component to the idf_espressif__led_strip interface
target. The component name, without the namespace, is referred to as the
short name. In this example, it is led_strip.
- espressif__led_strip -> idf_espressif__led_strip # name(directory name)
- idf_espressif__led_strip -> idf_espressif__led_strip # interface
- idf::espressif__led_strip -> idf_espressif__led_strip # alias
- _idf_espressif__led_strip -> idf_espressif__led_strip # real target
- led_strip -> idf_espressif__led_strip # short name
When another component with the same espressif__led_strip name is
initialized with a different priority, there is no need to change
anything in the cache, because the mapping stays the same.
The cache must be updated when two components share the same short name
but belong to different namespaces. This situation is likely uncommon.
For instance, consider espressif__led_strip and my__led_strip. If
my__led_strip has a higher priority, the cache is updated to reflect the
short name as follows:
- led_strip -> idf_my__led_strip
If both components have the same priority, the short name mapping for
led_strip is entirely removed. Conversely, if my__led_strip has a lower
priority, no short name mapping is added for it.
The short name is also added to the COMPONENT_SHORT_NAME property of the
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a revised version of the existing __create_confserver_target
function. It creates a specified confserver target for a given
executable. The kconfig_menus.json file, used by the IDEs, is generated
when the confserver starts. This differs from the previous behavior,
where kconfig_menus.json was created globally along with other sdkconfig
formats. The reason for this change is that kconfig_menus.json contains
the Kconfig menu hierarchy and it is not just a flat option-value
format. It needs to accurately reflect which configurations for which
components are included or excluded. The kconfig_menus.json is generated
at `build/kconfig_menus.json`, where IDEs expect it. This means the file
is overwritten every time the server starts by kconfig_menus.json
version for given executable, so only one confserver can run at a time.
This is likely acceptable, as I don't believe it's possible to safely
run multiple instances of confserver due to the potential race
conditions when the sdkconfig file are generated.
In the future, we may include the location of kconfig_menus.json in
project_description.json so it can be easily identified by IDEs for
each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
