On some Windows systems, antivirus, endpoint-security or DLP/encryption
software intercepts short-lived toolchain processes and strips their
stdout when the build captures it through a pipe, while the same command
prints its output normally when run by hand. The tool exits successfully
but returns nothing, and each build step that reads toolchain output then
failed with a different, cryptic error far from the real cause:
- CMake configuration aborted with "Unknown arguments specified" in
components/xtensa/project_include.cmake, or "check_expected_tool_version
invoked with incorrect arguments" in components/esp_common.
- ldgen turned the empty objdump output into an opaque pyparsing
"Expected 'In archive'" traceback.
- idf_tools.py silently reported the compiler/debugger version as
"unknown", sending users into a fruitless reinstall loop.
Detect the empty result at each consumer and fail (or warn) with an
actionable message that names the likely cause and the remedy:
- tools/cmake/compiler_query.cmake: new __compiler_query() helper runs a
compiler query and fails with a clear error on empty or failed output.
It is a standalone module included by both the cmakev1 and cmakev2
utilities, since the esp_common and xtensa project_include.cmake that
call it are shared by both build systems. The xtensa if() arguments are
now quoted so an empty result no longer collapses into a parse error.
- tools/ldgen/ldgen.py: _run_objdump() rejects empty objdump output, and
non-empty-but-unparsable section info is caught and re-raised as a clear
LdGenFailure instead of a raw pyparsing traceback.
- tools/idf_tools.py: empty version output now warns with the cause and
returns UNKNOWN_VERSION instead of silently reporting "unknown".
Closes https://github.com/espressif/esp-idf/issues/18727
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Resolve #include lines in linker script templates against the linked
component graph: idf_build_library appends every linked component's
INCLUDE_DIRS to the C preprocessor invocation for each .in linker
script, so a template can include any component header (e.g. a ULP
memory-layout template including soc/soc.h) without the owning
component having to know about it.
Also:
- Always pass -I<config_dir> so templates can include sdkconfig.h.
- Add a FLAGS option to target_linker_script for explicit preprocessor
flags (e.g. -D__ASSEMBLER__ or ld-snippet include dirs); when given,
FLAGS replaces the parent-dir==target include heuristic for that
script.
- Add a MEMORY option to target_linker_script that emits the marked
linker script as -T before all others, so section-placement scripts
in one component can reference MEMORY regions and REGION_ALIASes
declared by a memory-layout script in another component.
- Make C-preprocessor comment keeping (-C) part of the default flag set
rather than hardcoded, so FLAGS can drop it. linker_script_preprocessor
no longer forces -C; both the cmakev1 and cmakev2 preprocessors add it
to their default CFLAGS (output unchanged for existing scripts). A ULP
template that includes soc/soc.h omits -C so ld does not choke on the
header's // comments.
- Store per-script metadata (generated output, flags) in MD5-keyed
component properties instead of parallel lists.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Promote the build system v2 functions and macros used by the
examples/build_system/cmakev2 examples to the generated API reference,
document the component-scope and version variables, the public build
properties, and the public component properties, and add cmakev2
build_property and component_property directives with dedicated Build
Properties and Component Properties sections to the esp-docs extension.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When esp-idf-kconfig >= 3.9.0 is installed, we can use the optimized
menuconfig target without the need to preprocess the sdkconfig file by
kconfgen (removing deprecated options - menuconfig - readding deprecated
options).
Also removed unused {menuconfig_depends} from the new target.
Co-authored-by: Cursor <cursoragent@cursor.com>
In cmakev1, the executable target is named "${project}.elf". In cmakev2,
the executable is named "${project}" with .elf as the output suffix.
Strip the .elf suffix and look up the bare target name when the original
name doesn't exist.
Commit f62f45cf5c changed target_add_binary_data to resolve the
embedded file path relative to the component directory by using
idf_component_get_property to obtain the COMPONENT_DIR. This fixed the
path resolution when target_add_binary_data is called from
idf_component_include, which runs outside the component's directory
context.
However, target_add_binary_data can also be called directly from a
project's CMakeLists.txt with a non-component target, e.g.
target_add_binary_data(app.elf ...) as done in
examples/security/security_features_app. In this case,
idf_component_get_property fails because the target is not a component.
Fix this by moving the path resolution to idf_component_include, where
the embed file paths from EMBED_FILES and EMBED_TXTFILES component
properties are resolved to absolute paths relative to COMPONENT_DIR
before being passed to target_add_binary_data. This way,
target_add_binary_data receives already absolute paths from
idf_component_include and can use plain get_filename_component(ABSOLUTE)
for direct calls, which correctly resolves relative to
CMAKE_CURRENT_SOURCE_DIR.
Fixes: f62f45cf5c ("fix(cmakev2/utilities): add a dependency target for the embedded file")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a helper function for printing multi-line post-build messages on a
target. The message lines are joined with newlines, written to a file at
configure time, and printed at build time using a single cmake -E cat
command, replacing the need for repeated cmake -E echo invocations.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add idf_build_get_compile_options() to aggregate COMPILE_OPTIONS,
C_COMPILE_OPTIONS, CXX_COMPILE_OPTIONS, and ASM_COMPILE_OPTIONS build
properties with generator expressions. Replace internal
__get_compile_options(OUTPUT ...) usage in idf_component_register and
idf_component_include with the new public function.
Currently, embedded files can be added in two ways. First, by calling
the `target_add_binary_data` function directly within a component.
Second, by passing `EMBED_FILES` or `EMBED_TXTFILES` to
`idf_component_register`, or in cmakev2 by setting the `EMBED_FILES` or
`EMBED_TXTFILES` component properties.
The source file for an embedded file is generated using
`add_custom_command`. When the embedded file is added directly in the
component's CMakeLists.txt file by using the `target_add_binary_data`
function, the `add_custom_command` command is evaluated in the same
directory context where the component target is created. As a result,
the generated embedded file can be used automatically as a file
dependency in component's sources.
However, when an embedded file is added via `idf_component_register` or
by setting the component property, the call to `add_custom_command`
inside `target_add_binary_data` occurs after the component has been
evaluated, specifically after the `add_subdirectory` call, within
`idf_component_include`. This means it is not created in the same
directory context as the component's target. In this case, the embedded
file dependency is not added to the component's target, and the embedded
file is not generated. This behavior is described in the
`add_custom_command` documentation [1].
> A target created in the same directory (CMakeLists.txt file)
> that specifies any output of the custom command as a source
> file is given a rule to generate the file using the command at
> build time.
To fix this issue, an explicit custom target for the generated embedded
file is created and added as a dependency of the component's target,
ensuring that the file is generated correctly.
[1] - https://cmake.org/cmake/help/latest/command/add_custom_command.html
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the `idf_build_generate_metadata` function takes an
executable target as an argument, and the binary name is stored in the
`EXECUTABLE_BINARY` property. This approach is inconvenient because a
single executable might generate multiple binary images, making it
unreliable to store the binary image name in the executable property due
to the N:1 relationship. To address this, pass the binary image target
instead of the executable target to the `idf_build_generate_metadata`
function. This change is facilitated by linking the binary target,
executable target, and library interface targets with the following
properties:
binary:EXECUTABLE_TARGET -> executable:LIBRARY_INTERFACE -> library
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The COMPONENTS and EXCLUDE_COMPONENTS variables are unused in cmakev2. A
deprecation warning will be printed if they are set. This change also
includes a helper functions for printing deprecation warnings.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the toolchain CMake files use the remove_duplicated_flags
function from utilities.cmake. The cmakev2 implementation also includes
this function for backward compatibility. Move the
remove_duplicated_flags function to a separate file,
deduplicate_flags.cmake, so it can be shared between cmakev1 and
cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, we include numerous functions in the automatically generated
documentation for the build system API. Let's begin with only the
essential functions and gradually add more to the API based on requests
and actual needs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Return the difference between two lists, meaning the elements that are
present in the first list but not in the second.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function removes entries containing generator expressions
from the given list. The list is modified in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a helper function that returns a list of paths relative to a
given base directory for a list of input paths.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a phony target that intentionally fails the build, displaying the
error messages specified in the message arguments. An empty CMake file is
created and included, which is then deleted when the target is executed.
This forces CMake to rerun, as the file is added to the CMake rerun
dependencies.
This has been carried over from cmakev1 as it is used by several
components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Search for the library interface target created with the
idf_build_library() function and linked to the executable, examine the
LINK_LIBRARIES for the executable and the LIBRARY_INTERFACES build
property, which stores all library interface targets created by the
idf_build_library() function.
This helper is intended for use by functions, such as the
idf_build_generate_metadata() function, which accepts the executable
target as an argument to find the library linked to it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the INTERFACE_LINK_LIBRARIES and LINK_LIBRARIES properties,
recursively retrieve the list of dependencies for the specified target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the generic compile_options are ignored because they are
unset after retrieving the build property. To fix this, avoid explicitly
setting compile_options, as this variable is initialized with the
idf_build_get_property(compile_options ...) call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add binary data into the build target by converting it into a generated
source file, which is then compiled into a binary object as part of the
build process.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The add_deprecated_target_alias function is used by some existing
components, so it should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The file_generate function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The fail_target function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The add_prefix function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The spaces2list function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add an optional BASE_DIR option to specify a directory that will serve
as the base directory for evaluating input paths. This can be useful,
for example, when collecting component sources.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Gather the compilation options from COMPILE_OPTIONS, C_COMPILE_OPTIONS,
CXX_COMPILE_OPTIONS, and ASM_COMPILE_OPTIONS build properties into a
single list using generator expressions. This list can then be used with
the target_compile_options call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, only the value stored in CMake's cache is considered when
setting the default value. Allow regular variables as well, which may be
set in the project's CMakeLists.txt file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When working with command output, it is sometimes necessary to handle it
line by line. Add a `__split` helper function that can split an input
string based on a specified separator, which defaults to a newline, and
store the results in a list. This function also offers modifiers to
strip whitespace and remove empty entries.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` functions share a lot of similar
code. Move these common parts into new `__(set|get)_property` helper
functions. With the upcoming `idf_build_library` API function, we might
need to add properties for the interface target created for the library,
which would otherwise lead to yet another code duplication for setting
and getting library interface properties.
Update the current implementations of `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` to utilize these new helper
functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Helper for obtaining the default value of a variable. It returns the
value of the specified variable based on the following order of
precedence, with the highest precedence first:
1. Environmental variable
2. CMake cache variable
3. Provided default value
This can be used to retrieve the value of variables that might also be
set in the environment or cache, such as PYTHON or SDKCONFIG.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With the default signature of `cmake_parse_arguments`, without using
`PARSE_ARGV`, it's not possible to determine if options were not
specified or set as an empty string (or empty list)[1]. If an empty
string is passed to the `PATHS` option, the variable parsed by
`cmake_parse_arguments` is not defined. This issue can be addressed by
using `PARSE_ARGV`, but this approach only works for functions and
requires CMake version 3.31 or newer. Additionally, when `PARSE_ARGV` is
used for multiple value option, the values are not concatenated into a
single list, which is inconvenient, as the lists are instead escaped. If
the `PATHS` option is not defined, set it to an empty string. This
allows passing an empty string, as well as a mix of lists and
individual strings, through PATHS.
The behaviour can be seen with a simple example:
$ cmake -P test.cmake
```test.cmake
cmake_minimum_required(VERSION 3.22)
function(test)
set(options)
set(one_value)
set(multi_value PATHS)
cmake_parse_arguments(ARG "${options}" "${one_value}" "${multi_value}" ${ARGN})
#cmake_parse_arguments(PARSE_ARGV 0 ARG "${options}" "${one_value}" "${multi_value}")
message("PATHS: ${ARG_PATHS}")
endfunction()
test(PATHS "one;two;three" "four" "five;six" "seven")
```
```
PATHS: one;two;three;four;five;six;seven
vs
PATHS: one\;two\;three;four;five\;six;seven
```
Also update the current usage of __get_absolute_paths, as the check for
empty PATHS is no longer necessary.
[1] https://cmake.org/cmake/help/latest/policy/CMP0174.html#policy:CMP0174
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Discover component directories and initialize components within them.
This process does not include managed components, which should be added
separately at a later stage. To facilitate this, some minimal
functionalities are introduced, such as build properties, component
properties, and other helper functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Stefan Stipanovic
Frantisek Hrbata
Sudeep Mohanty
morris
Jan BeranandCursor