When a component's manifest declared a managed dep `<ns>/<name>` and the
project had a local `components/<name>` shadowing it, configure failed
with `IDF: Failed to resolve component '<ns>__<name>'`.
cmakev1 does a project-wide injection so the manager's _choose_component
sees every component name in known_components and rewrites the
namespaced dep to its locally-shadowing short name. cmakev2 invokes the
manager per component with a single-component requirements file, so
known_components has only one entry and the rewrite cannot fire --
`<ns>__<name>` is written verbatim into MANAGED_REQUIRES and the
force-include at component.cmake:995 aborts.
Seed the per-component requirements file with one __COMPONENT_SOURCE
entry per discovered component so handle_project_requirements() builds
the same known_components list cmakev1 produces. The seeded entries are
inert on the cmake side -- the __component_set_property shim ignores
__COMPONENT_SOURCE -- they exist purely to feed the manager's rewrite.
Add a regression test in tools/test_build_system/buildv2/test_component.py.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Any idf.py invocation could hang indefinitely with no output while
spawning an unbounded chain of "idf.py --version" subprocesses,
eventually exhausting system memory.
During init_cli(), idf.py parses the project's dependencies.lock to
vet trusted component-provided idf_ext.py extensions. If the lock
contains a component whose manifest has an "if: idf_version" clause,
evaluating it calls idf-component-manager's _get_idf_version(). Outside
a CMake build the IDF_VERSION environment variable is not set, so that
function falls back to running "idf.py --version" as a subprocess,
which re-enters init_cli() and recurses without bound.
During a normal CMake build the component manager runs as a subprocess
that already has IDF_VERSION in its environment (see build/config.env),
so the fallback is never reached. The recursion happens only because
idf.py runs component-manager code in-process during its own CLI
startup, outside that context.
Seed IDF_VERSION into os.environ early in init_cli(), before any
dependencies.lock parsing, using the subprocess-free
idf_version_from_cmake() helper. This gives in-process component-manager
code the same IDF_VERSION a CMake build would provide.
Add a buildv2 test that reconfigures the test app with both GCC and
clang toolchains and asserts that the selected IDF_TOOLCHAIN value
is propagated to sdkconfig. The clang case is skipped when a clang
compiler is not available in PATH.
Verify that setting IDF_COMPONENT_MANAGER=0 skips the component manager
flow entirely and produces a successful build. The test also asserts that
no "Component manager round" messages appear in the output, confirming
the manager loop is not entered.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Pytest's default import mode (prepend/rootdir) imports test modules as
top-level modules keyed by filename. When two directories contain test
files with the same basename (e.g. test_sdkconfig.py in both
test_build_system/ and test_build_system/buildv2/), pytest tries to
register both as the module name "test_sdkconfig". The second collection
fails with "import file mismatch" because the module object already
cached in sys.modules points to the first file.
Adding __init__.py to the buildv2/ directory makes it a proper Python
package. Pytest then imports its test modules under the package
namespace (buildv2.test_sdkconfig), which is distinct from the
top-level test_sdkconfig, resolving the collision.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The sdkconfig file may contain configuration options defined in Kconfig
files of managed components. Since kconfgen runs before the component
manager fetches these components, the Kconfig definitions for managed
component options are not yet available. The kconfgen --output config
flag regenerates sdkconfig from kconfiglib's internal state, which only
knows about options with loaded Kconfig definitions. This causes unknown
options (i.e., those from managed components) to be silently dropped
from sdkconfig during intermediate regeneration rounds.
Note that kconfgen's --config flag (used for reading sdkconfig) only
performs deprecated option name replacement and does NOT drop unknown
options. The problem is exclusively in --output config, which writes a
fresh sdkconfig from the parsed Kconfig tree state.
Fix this by introducing a __SDKCONFIG_ORIG build property that provides
an indirection layer for the --config input path:
- Before the component manager runs: __SDKCONFIG_ORIG points to a copy
of the original sdkconfig (build/sdkconfig.orig), created by the new
__create_sdkconfig_orig_copy() function. This copy preserves all
original options, including those from managed components.
- During intermediate kconfgen runs: --config reads from the preserved
copy (so unknown options survive as input), while --output config
writes to the real sdkconfig (unknown options may be dropped there,
but this is harmless since kconfgen always reads from the copy).
- After the component manager completes: __SDKCONFIG_ORIG is reset to
point to the real sdkconfig and __BASE_KCONFGEN_CMD is rebuilt, so
that subsequent operations (menuconfig, save-defconfig, confserver)
read and write the actual sdkconfig file directly.
The flow is:
__create_sdkconfig_orig_copy()
-> __SDKCONFIG_ORIG = build/sdkconfig.orig
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
__fetch_components_from_registry():
loop:
download_components()
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
if success: break
endloop
-> __SDKCONFIG_ORIG = project/sdkconfig
-> rebuild __BASE_KCONFGEN_CMD
idf_create_menuconfig() / save-defconfig / confserver
-> uses --config project/sdkconfig (the real file)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a simple test to verify that the buildv2_test_app can be built for
the Linux target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, cmakev2 is being tested only in backward-compatible mode by
using the existing cmakev1 tests with the cmakev2 test application. We
also need to add tests specific to cmakev2, and it is convenient to
reuse the existing build system testing framework. Let's add a `buildv2`
subdirectory to the existing `tools/test_build_system` directory and use
the `pytest_collection_modifyitems` hook to ignore tests in this
directory unless the `--buildv2` option is used.
Without the `--buildv2` option, only the existing cmakev1 tests are
executed and tests in `buildv2` directory are skipped. With the
`--buildv2` option, the existing cmakev1 tests run with the cmakev2
testing application for backward compatibility testing, and all cmakev2
tests within the `buildv2` subdirectory are also executed.
Note: we cannot use the `pytest_ignore_collect` hook, because the
`--buildv2` option is not known to the pytest, so the
`config.getoption('--buildv2', False)` returns always False. We would
likely need to add the `--buildv2` option in the conftest.py in the
esp-idf root directory.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
fix: lsadjf las jflasjfl aslfsald asl fsadlf sladsal jfsadfas
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This adds a clear header to the pytest output, indicating which build
system version is currently being tested.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The test_hint_components_loading fails for buildv2 because the build
system only links the main component but not the components with the
hints.yml. Updated the test to explicitly include the new components in
the build.
The minimal build property is simply a shorthand for `set(COMPONENTS
main)`. The issue is that there is currently no check to verify whether
the `main` component actually exists or is known to the build system.
If the `main` component is not present, print an error message along
with suggestions on how to fix this inconsistency.
Closes https://github.com/espressif/esp-idf/issues/18219
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The test_rebuild_no_changes test verifies that running idf.py build
successively without any file changes results in identical build
artifacts on the second run (i.e., nothing gets rebuilt).
This test was failing in buildv2 because it expected kconfig_menus.json
to be present in build/config/ after a normal build. However, in
cmakev2, kconfig_menus.json is not generated during regular builds.
In cmakev1, kconfig_menus.json was generated globally during every
build alongside other config files (sdkconfig.h, sdkconfig.cmake, etc).
In cmakev2, kconfig_menus.json generation does not happend for
normal builds because it depends on the Kconfig menu hierarchy
and cannot be generated globally. It must be generated per-executable.
Hence, this commit updates the artefacts list for cmakev2 to not expect
the kconfig_menus.json file during a build/re-build action.
The commit adb2d5deee ("feat(cmake): Produce warnings when component dependen..")
introduced additional checks for source files and include directories
used by a component that are located outside the component's directory.
If these files and directories belong to another component, a warning is
issued. This feature has not yet been implemented in cmakev2, so related
tests are temporary disabled.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit renames the buildv2_test_app.c file to build_test_app.c in
the buildv2_test_app template to enable tests to refer to the same
source file name when writing tests for both cmakev1 and cmakev2.
Commit 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
introduced a temporary workaround for the cmakev2 testing
application by including the cmakev1 project.cmake file after the
return() statement. The issue is that some tests, such as
test_build_fail_on_build_time, append commands to the project's
CMakeLists.txt. These commands are placed after the newly added return()
statement, causing them to be ignored and resulting in test failures. To
resolve this, include the cmakev1 project.cmake within the cmake block
comment.
Fixes: 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
