diff --git a/docs/en/api-guides/build-system.rst b/docs/en/api-guides/build-system.rst index ac57d66a3e9..bec630be8b0 100644 --- a/docs/en/api-guides/build-system.rst +++ b/docs/en/api-guides/build-system.rst @@ -149,6 +149,43 @@ If using an IDE with CMake, setting the ``PYTHON`` value as a CMake cache overri To manage the Python version more generally via the command line, check out the tools pyenv_ or virtualenv_. These let you change the default Python version. +.. _build-acceleration: + +Build Acceleration +------------------ + +ESP-IDF supports several tools to accelerate the build process. These tools work as compiler launchers that wrap the actual compiler invocation. + +CCache +^^^^^^ + +CCache_ is a compiler cache that speeds up recompilation by caching previous compilations. When enabled, if a compilation is repeated with the same source code and compiler flags, CCache returns the cached result instead of recompiling. + +See the :ref:`idf_py_global_options` section for more information about how to enable or disable it. + +Configdep Wrapper +^^^^^^^^^^^^^^^^^ + +The ``esp-idf-configdep`` tool post-processes compiler-generated dependency files to reduce unnecessary rebuilds caused by ``sdkconfig.h`` changes. + +Normally, every source file that includes ``sdkconfig.h`` would be rebuilt whenever any configuration option changes, because the compiler records ``sdkconfig.h`` as a direct dependency. The Configdep tool edits the dependency files after the initial build so that source files no longer depend directly on ``sdkconfig.h``. Instead, they depend on a special file structure created by ``esp-idf-kconfig``: each configuration option has its own file, and when a given option is changed, only the corresponding file is touched. That way only the source files that actually use the changed option are rebuilt, not the entire project. + +Important: source files are not modified. They still contain ``#include `` and there is no need to alter anything in the source code for Configdep to work. The tool operates purely on the generated dependency (``.d``) files. + +This is particularly useful when there are frequent changes of a small number of config options between rebuilds. Configdep is enabled by default. + +See the :ref:`idf_py_global_options` section for more information about how to enable or disable it. + +Chaining Acceleration Tools +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When multiple acceleration tools are enabled, they are chained together as compiler launchers. The chain is applied in a specific order: + +1. **esp-idf-configdep** (if enabled) - processes dependency files and optimizes rebuilds caused by ``sdkconfig.h`` changes +2. **ccache** (if enabled) - caches compilations + +Each tool in the chain wraps the next, ultimately invoking the actual compiler. This allows combining the benefits of multiple tools. However, combining multiple tools may not always result in optimal performance, and it may be needed to experiment with different combinations of the acceleration tools to achieve the best performance for your project. + .. _example-project-structure: Example Project diff --git a/docs/en/api-guides/tools/idf-py.rst b/docs/en/api-guides/tools/idf-py.rst index ae804f7363a..4dda25fcf33 100644 --- a/docs/en/api-guides/tools/idf-py.rst +++ b/docs/en/api-guides/tools/idf-py.rst @@ -459,6 +459,8 @@ By default, the ``sdkconfig`` file is created in the project root directory. How For a complete example, see the :example_file:`Multiple Build Configurations Example `. +.. _idf_py_global_options: + Global Options ============== @@ -472,6 +474,8 @@ To list all available root level options, run ``idf.py --help``. To list options Note that some older versions of CCache_ may exhibit bugs on some platforms, so if files are not rebuilt as expected, try disabling CCache_ and rebuilding the project. To enable CCache_ by default, set the ``IDF_CCACHE_ENABLE`` environment variable to a non-zero value. +- ``--configdep`` or ``--no-configdep`` enables or disables the rebuild optimization using ``esp-idf-configdep``. This tool post-processes compiler-generated dependency files to reduce unnecessary rebuilds caused by ``sdkconfig.h`` changes. This is particularly useful when there are frequent changes of small number of config options between rebuilds. Enabled by default. To permanently enable or disable configdep, set the ``IDF_CONFIGDEP_ENABLE`` environment variable to ``1`` or ``0`` respectively. + - ``-v`` flag causes both ``idf.py`` and the build system to produce verbose build output. This can be useful for debugging build problems. - ``--cmake-warn-uninitialized`` (or ``-w``) causes CMake to print uninitialized variable warnings found in the project directory only. This only controls CMake variable warnings inside CMake itself, not other types of build warnings. This option can also be set permanently by setting the ``IDF_CMAKE_WARN_UNINITIALIZED`` environment variable to a non-zero value. - ``--no-hints`` flag disables hints on resolving errors and disable capturing output. diff --git a/docs/en/api-guides/tools/idf-tools-notes.inc b/docs/en/api-guides/tools/idf-tools-notes.inc index 37a2b631c22..36d5d27174d 100644 --- a/docs/en/api-guides/tools/idf-tools-notes.inc +++ b/docs/en/api-guides/tools/idf-tools-notes.inc @@ -89,6 +89,11 @@ Some ESP-specific instructions for running QEMU for Xtensa chips are here: https Some ESP-specific instructions for running QEMU for RISC-V chips are here: https://github.com/espressif/esp-toolchain-docs/blob/main/qemu/esp32c3/README.md +--- + +.. tool-esp-idf-configdep-notes + + --- .. tool-idf-python-notes diff --git a/docs/zh_CN/api-guides/build-system.rst b/docs/zh_CN/api-guides/build-system.rst index b3f72737849..2f41f4d8971 100644 --- a/docs/zh_CN/api-guides/build-system.rst +++ b/docs/zh_CN/api-guides/build-system.rst @@ -149,6 +149,43 @@ ESP-IDF 适用于 Python 3.10 以上版本。 如果想在命令行中更优雅地管理 Python 的各个版本，请查看 pyenv_ 或 virtualenv_ 工具，它们会帮助你更改默认的 python 版本。 +.. _build-acceleration: + +构建加速 +-------- + +ESP-IDF 支持多种用于加速构建过程的工具。这些工具充当编译器启动器，封装并调用实际的编译器。 + +CCache +^^^^^^ + +CCache_ 是一种编译器缓存工具，通过缓存先前的编译结果来加速重新编译。启用后，当使用相同的源代码和编译器标志重复编译时，CCache 会直接返回缓存结果，而无需重新编译。 + +关于如何启用或禁用 CCache，请参见 :ref:`idf_py_global_options` 章节。 + +Configdep 封装器 +^^^^^^^^^^^^^^^^^ + +``esp-idf-configdep`` 工具对编译器生成的依赖文件进行后处理，以减少由 ``sdkconfig.h`` 变更引起的不必要重建。 + +通常在任一配置选项发生变化时，凡是包含 ``sdkconfig.h`` 的源文件都会被重建，因为编译器会将 ``sdkconfig.h`` 记录为直接依赖。Configdep 工具会在初次构建后编辑这些依赖文件，使源文件不再直接依赖 ``sdkconfig.h``。取而代之的是，源文件会依赖由 ``esp-idf-kconfig`` 创建的一种特殊文件结构：每个配置选项都有各自的文件，当某个选项被修改时，只会更新对应的文件。这样一来，只有实际使用到该变更选项的源文件会被重建，而并非整个项目。 + +重要：源文件不会被修改，仍然包含 ``#include ``，并且无需更改源代码中的任何内容，Configdep 工具即可正常工作。该工具仅作用于生成的依赖文件 (``.d``) 进行工作。 + +若重建时仅有少量配置选项频繁变更，该工具尤为有用。Configdep 默认启用。 + +有关如何启用或禁用 Configdep，请参见 :ref:`idf_py_global_options` 章节。 + +串联使用加速工具 +^^^^^^^^^^^^^^^^ + +启用多个加速工具时，它们会以编译器启动器链的形式串联使用。该链按照以下顺序应用： + +1. **esp-idf-configdep** （若启用） - 处理依赖文件并优化由 ``sdkconfig.h`` 变更引起的重建 +2. **ccache** （若启用） - 缓存编译结果 + +链中的每个工具都会封装下一个工具，最终调用实际的编译器。这样可以叠加多个工具的优势。然而，组合使用多个工具并不总能带来最佳性能，因此可能需要针对具体项目尝试不同的工具组合，以获得最佳构建性能。 + .. _example-project-structure: 示例项目 diff --git a/docs/zh_CN/api-guides/tools/idf-py.rst b/docs/zh_CN/api-guides/tools/idf-py.rst index 36a62e79f9e..008eeb29d6c 100644 --- a/docs/zh_CN/api-guides/tools/idf-py.rst +++ b/docs/zh_CN/api-guides/tools/idf-py.rst @@ -459,6 +459,8 @@ ESP-IDF 支持 `CMake presets`_ 以简化多个构建配置的管理。此功能 完整示例请参阅 :example_file:`Multiple Build Configurations Example `。 +.. _idf_py_global_options: + 全局选项 ============== @@ -472,6 +474,8 @@ ESP-IDF 支持 `CMake presets`_ 以简化多个构建配置的管理。此功能 注意，某些旧版本 CCache_ 在某些平台上存在 bug，因此如果文件没有按预期重新构建，可禁用 CCache_ 并重新构建。可以通过将环境变量 ``IDF_CCACHE_ENABLE`` 设置为非零值来默认启用 CCache_。 +- ``--configdep`` 或 ``--no-configdep`` 用于启用或禁用基于 ``esp-idf-configdep`` 的重建优化。该工具会对编译器生成的依赖文件进行后处理，以减少因 ``sdkconfig.h`` 变更而导致的不必要重建。若重建时仅有少量配置选项频繁变更，这一功能尤为有用。该功能默认启用。若要永久启用 configdep，请将 ``IDF_CONFIGDEP_ENABLE`` 环境变量设置为 ``1``；若要永久禁用，则设置为 ``0``。 + - ``-v`` 会使 ``idf.py`` 和构建系统生成详细的构建输出，有助于调试构建错误。 - ``--cmake-warn-uninitialized`` （或 ``-w``）将使 CMake 只显示在工程目录中发现的变量未初始化的警告，该选项仅控制 CMake 内部的 CMake 变量警告，不控制其他类型的构建警告。将环境变量 ``IDF_CMAKE_WARN_UNINITIALIZED`` 设置为非零值，可永久启用该选项。 - ``--no-hints`` 用于禁用有关错误处理的提示并禁用捕获输出。 diff --git a/docs/zh_CN/api-guides/tools/idf-tools-notes.inc b/docs/zh_CN/api-guides/tools/idf-tools-notes.inc index da663966cc4..1836a427caf 100644 --- a/docs/zh_CN/api-guides/tools/idf-tools-notes.inc +++ b/docs/zh_CN/api-guides/tools/idf-tools-notes.inc @@ -91,6 +91,11 @@ Some ESP-specific instructions for running QEMU for Xtensa chips are here: https Some ESP-specific instructions for running QEMU for RISC-V chips are here: https://github.com/espressif/esp-toolchain-docs/blob/main/qemu/esp32c3/README.md +--- + +.. tool-esp-idf-configdep-notes + + --- .. tool-idf-python-notes diff --git a/tools/cmake/kconfig.cmake b/tools/cmake/kconfig.cmake index 892e981f6a3..ea332478f0f 100644 --- a/tools/cmake/kconfig.cmake +++ b/tools/cmake/kconfig.cmake @@ -8,6 +8,9 @@ set(ESP_MENUCONFIG_MIN_KCONFIG_VERSION "3.1.0") # (fused menuconfig + deprecated-options post-processing in a single invocation). set(MENUCONFIG_INLINE_MIN_KCONFIG_VERSION "3.9.0") +# Minimum esp-idf-kconfig version required for cdep_tree / configdep (--output cdep_tree) +set(CONFIGDEP_MIN_KCONFIG_VERSION "3.6.0") + function(__kconfig_init) idf_build_get_property(idf_path IDF_PATH) @@ -239,6 +242,12 @@ function(__kconfig_generate_config sdkconfig sdkconfig_defaults) if(output_sdkconfig) list(APPEND kconfgen_output_options --output config ${sdkconfig}) endif() + # If configdep feature is enabled in ESP-IDF and esp-idf-kconfig version supports cdep_tree, + # add output option for configdep (requires esp-idf-kconfig >= CONFIGDEP_MIN_KCONFIG_VERSION) + __check_python_package_min_version(${python} esp-idf-kconfig "${CONFIGDEP_MIN_KCONFIG_VERSION}" CONFIGDEP_SUPPORTED) + if(CONFIGDEP_SUPPORTED AND CONFIGDEP_ENABLE) + list(APPEND kconfgen_output_options --output cdep_tree ${config_dir}) + endif() execute_process( COMMAND ${prepare_kconfig_files_command}) diff --git a/tools/cmake/project.cmake b/tools/cmake/project.cmake index 04dc1da4652..1868a336559 100644 --- a/tools/cmake/project.cmake +++ b/tools/cmake/project.cmake @@ -608,16 +608,52 @@ macro(project project_name) __target_set_toolchain() + # Build compiler launcher chain + set(compiler_launcher_chain "") + # Add custom wrapper if enabled (FIRST in chain) + if(CONFIGDEP_ENABLE) + # Check that esp-idf-kconfig >= CONFIGDEP_MIN_KCONFIG_VERSION is installed + # (required for cdep_tree / configdep support; version is defined in kconfig.cmake) + __check_python_package_min_version(${PYTHON} esp-idf-kconfig + "${CONFIGDEP_MIN_KCONFIG_VERSION}" _kconfig_version_ok) + if(_kconfig_version_ok) + find_program(CONFIGDEP_FOUND esp-idf-configdep) + if(CONFIGDEP_FOUND) + message(STATUS "esp-idf-configdep will be used for faster recompilation") + list(APPEND compiler_launcher_chain "esp-idf-configdep") + else() + message(WARNING "esp-idf-configdep enabled but not found. Re-run install and export scripts.") + endif() + else() + message(WARNING "esp-idf-configdep not supported by esp-idf-kconfig " + "(>= ${CONFIGDEP_MIN_KCONFIG_VERSION} required). Re-run install and export scripts.") + endif() + endif() + + # Add ccache if enabled (SECOND in chain) if(CCACHE_ENABLE) find_program(CCACHE_FOUND ccache) if(CCACHE_FOUND) message(STATUS "ccache will be used for faster recompilation") - set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache) + list(APPEND compiler_launcher_chain "ccache") else() message(WARNING "enabled ccache in build but ccache program not found") endif() endif() + # Apply the launcher chain - CMake automatically creates semicolon-separated list + if(compiler_launcher_chain) + set(CMAKE_C_COMPILER_LAUNCHER ${compiler_launcher_chain}) + set(CMAKE_CXX_COMPILER_LAUNCHER ${compiler_launcher_chain}) + set(CMAKE_ASM_COMPILER_LAUNCHER ${compiler_launcher_chain}) + + # Debug: show what the launcher chain looks like + string(REPLACE ";" " -> " launcher_display "${compiler_launcher_chain}") + message(STATUS "Compiler launcher chain: ${launcher_display}") + else() + message(STATUS "No compiler launcher chain will be used.") + endif() + # The actual call to project() __project(${project_name} C CXX ASM) __linux_build_set_lang_version() diff --git a/tools/cmakev2/idf.cmake b/tools/cmakev2/idf.cmake index 03b45abb57f..5f401f4fb16 100644 --- a/tools/cmakev2/idf.cmake +++ b/tools/cmakev2/idf.cmake @@ -17,6 +17,11 @@ set(CMAKE_MODULE_PATH # for both cmakev1 and cmakev2. include(${CMAKE_CURRENT_LIST_DIR}/../cmake/version.cmake) +# Suppress CMake warning: "Manually-specified variables were not used by the project: CONFIGDEP_ENABLE" +# (CONFIGDEP_ENABLE is passed by idf.py but only used in the cmake v1 project() flow in project.cmake) +# FIXME: When cmakev2 will start supporting configdep, this can be removed. +set(_idf_ignore_configdep_enable "${CONFIGDEP_ENABLE}") + # The gdbinit.cmake file from cmakev1 contains a single function, # __generate_gdbinit, which is used in the generation of # project_description.json. diff --git a/tools/idf_py_actions/core_ext.py b/tools/idf_py_actions/core_ext.py index a275131f9ec..76420b816cd 100644 --- a/tools/idf_py_actions/core_ext.py +++ b/tools/idf_py_actions/core_ext.py @@ -530,6 +530,13 @@ def action_extensions(base_actions: dict, project_path: str) -> Any: 'envvar': 'IDF_CCACHE_ENABLE', 'default': False, }, + { + 'names': ['--configdep/--no-configdep'], + 'help': 'Use configdep wrapper to optimize rebuild process. Enabled by default.', + 'is_flag': True, + 'envvar': 'IDF_CONFIGDEP_ENABLE', + 'default': True, + }, { 'names': ['-G', '--generator'], 'help': 'CMake generator.', diff --git a/tools/idf_py_actions/tools.py b/tools/idf_py_actions/tools.py index 4364f6b32e9..27622c942ae 100644 --- a/tools/idf_py_actions/tools.py +++ b/tools/idf_py_actions/tools.py @@ -788,6 +788,7 @@ def ensure_build_directory( cache = _parse_cmakecache(cache_path) if os.path.exists(cache_path) else {} args.define_cache_entry.append(f'CCACHE_ENABLE={args.ccache}') + args.define_cache_entry.append(f'CONFIGDEP_ENABLE={args.configdep}') cache_cmdl = _parse_cmdl_cmakecache(args.define_cache_entry) diff --git a/tools/test_build_system/configdep_test_app/CMakeLists.txt b/tools/test_build_system/configdep_test_app/CMakeLists.txt new file mode 100644 index 00000000000..19decd6871e --- /dev/null +++ b/tools/test_build_system/configdep_test_app/CMakeLists.txt @@ -0,0 +1,9 @@ +cmake_minimum_required(VERSION 3.22) + +# placeholder_before_include_project_cmake + +include($ENV{IDF_PATH}/tools/cmake/project.cmake) + +# placeholder_after_include_project_cmake + +project(configdep_test_app) diff --git a/tools/test_build_system/configdep_test_app/README.md b/tools/test_build_system/configdep_test_app/README.md new file mode 100644 index 00000000000..f417621981d --- /dev/null +++ b/tools/test_build_system/configdep_test_app/README.md @@ -0,0 +1,6 @@ +| Supported Targets | ESP32 | ESP32-C2 | ESP32-C3 | ESP32-C5 | ESP32-C6 | ESP32-C61 | ESP32-H2 | ESP32-H21 | ESP32-H4 | ESP32-P4 | ESP32-S2 | ESP32-S3 | ESP32-S31 | +| ----------------- | ----- | -------- | -------- | -------- | -------- | --------- | -------- | --------- | -------- | -------- | -------- | -------- | --------- | + +# Configdep test application + +Minimal ESP-IDF project used by ``tools/test_build_system/test_configdep.py`` to exercise ``esp-idf-configdep`` selective rebuilds and to assert that the built firmware reflects Kconfig toggles (via strings embedded in the ELF). diff --git a/tools/test_build_system/configdep_test_app/main/CMakeLists.txt b/tools/test_build_system/configdep_test_app/main/CMakeLists.txt new file mode 100644 index 00000000000..39d1003c5d1 --- /dev/null +++ b/tools/test_build_system/configdep_test_app/main/CMakeLists.txt @@ -0,0 +1,2 @@ +idf_component_register(SRCS "configdep_main.c" "unrelated.c" + INCLUDE_DIRS "") diff --git a/tools/test_build_system/configdep_test_app/main/Kconfig.projbuild b/tools/test_build_system/configdep_test_app/main/Kconfig.projbuild new file mode 100644 index 00000000000..b22487eab66 --- /dev/null +++ b/tools/test_build_system/configdep_test_app/main/Kconfig.projbuild @@ -0,0 +1,5 @@ +menu "Configdep Test" + config TEST_CONFIGDEP_OPTION + bool "Test configdep option" + default y +endmenu diff --git a/tools/test_build_system/configdep_test_app/main/configdep_main.c b/tools/test_build_system/configdep_test_app/main/configdep_main.c new file mode 100644 index 00000000000..e2476f098b6 --- /dev/null +++ b/tools/test_build_system/configdep_test_app/main/configdep_main.c @@ -0,0 +1,22 @@ +/* + * SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD + * + * SPDX-License-Identifier: Apache-2.0 + */ +#include +#include "sdkconfig.h" +#include "freertos/FreeRTOS.h" +#include "freertos/task.h" + +extern void unrelated_function(void); + +void app_main(void) +{ +#if CONFIG_TEST_CONFIGDEP_OPTION + printf("CONFIGDEP_OPTION: ENABLED\n"); +#else + printf("CONFIGDEP_OPTION: DISABLED\n"); +#endif + unrelated_function(); + vTaskDelay(pdMS_TO_TICKS(1000)); +} diff --git a/tools/test_build_system/configdep_test_app/main/unrelated.c b/tools/test_build_system/configdep_test_app/main/unrelated.c new file mode 100644 index 00000000000..2dff8dd964f --- /dev/null +++ b/tools/test_build_system/configdep_test_app/main/unrelated.c @@ -0,0 +1,17 @@ +/* + * SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD + * + * SPDX-License-Identifier: Apache-2.0 + */ +/* + * Second TU in ``main``: only uses CONFIG_IDF_TARGET from sdkconfig—no conditional + * on the optional symbol defined for configdep_main.c (see ``Kconfig.projbuild``). + * Used by tests to snapshot this ``.obj`` alongside ``configdep_main``. + */ +#include +#include "sdkconfig.h" + +void unrelated_function(void) +{ + printf("IDF target: %s\n", CONFIG_IDF_TARGET); +} diff --git a/tools/test_build_system/test_build_system_helpers/__init__.py b/tools/test_build_system/test_build_system_helpers/__init__.py index d5166e7a1ea..0432ea96249 100644 --- a/tools/test_build_system/test_build_system_helpers/__init__.py +++ b/tools/test_build_system/test_build_system_helpers/__init__.py @@ -20,6 +20,8 @@ from .idf_utils import get_idf_build_env from .idf_utils import run_cmake from .idf_utils import run_cmake_and_build from .idf_utils import run_idf_py +from .rebuild_utils import BUILD_SNAPSHOT_EXCLUDE +from .rebuild_utils import rebuild_and_check from .snapshot import Snapshot from .snapshot import get_snapshot @@ -46,4 +48,6 @@ __all__ = [ 'file_contains', 'bin_file_contains', 'bin_files_differ', + 'BUILD_SNAPSHOT_EXCLUDE', + 'rebuild_and_check', ] diff --git a/tools/test_build_system/test_build_system_helpers/rebuild_utils.py b/tools/test_build_system/test_build_system_helpers/rebuild_utils.py new file mode 100644 index 00000000000..c40f4aacacc --- /dev/null +++ b/tools/test_build_system/test_build_system_helpers/rebuild_utils.py @@ -0,0 +1,24 @@ +# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD +# SPDX-License-Identifier: Apache-2.0 +"""Shared helpers and constants for rebuild tests (test_rebuild*.py).""" + +from .idf_utils import IdfPyFunc +from .snapshot import get_snapshot + +# Paths excluded from mtime snapshots (volatile or external-project stamps). +BUILD_SNAPSHOT_EXCLUDE = [ + 'build/log/*', + 'build/CMakeFiles/bootloader-complete', + 'build/bootloader-prefix/src/bootloader-stamp/bootloader-done', +] + + +def rebuild_and_check( + idf_py: IdfPyFunc, should_be_rebuilt: str | list[str], should_not_be_rebuilt: str | list[str] +) -> None: + """Assert that files matching *should_be_rebuilt* change and *should_not_be_rebuilt* stay the same after a build.""" + snapshot_should_be_rebuilt = get_snapshot(should_be_rebuilt) + snapshot_should_not_be_rebuilt = get_snapshot(should_not_be_rebuilt) + idf_py('build') + snapshot_should_be_rebuilt.assert_different(get_snapshot(should_be_rebuilt)) + snapshot_should_not_be_rebuilt.assert_same(get_snapshot(should_not_be_rebuilt)) diff --git a/tools/test_build_system/test_configdep.py b/tools/test_build_system/test_configdep.py new file mode 100644 index 00000000000..85a4c119936 --- /dev/null +++ b/tools/test_build_system/test_configdep.py @@ -0,0 +1,80 @@ +# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD +# SPDX-License-Identifier: Apache-2.0 +import logging +import os +from pathlib import Path + +import pytest +from test_build_system_helpers import IdfPyFunc +from test_build_system_helpers import append_to_file +from test_build_system_helpers import bin_file_contains +from test_build_system_helpers import get_snapshot +from test_build_system_helpers import replace_in_file + +CONFIGDEP_TEST_APP = 'tools/test_build_system/configdep_test_app' + +_MAIN_OBJ_DIR = ('build', 'esp-idf', 'main', 'CMakeFiles', '__idf_main.dir') +MAIN_OBJ = os.path.join(*_MAIN_OBJ_DIR, 'configdep_main.c.obj') +UNRELATED_OBJ = os.path.join(*_MAIN_OBJ_DIR, 'unrelated.c.obj') +CONFIGDEP_ELF = Path(os.path.join('build', 'configdep_test_app.elf')) + + +@pytest.mark.test_app_copy(CONFIGDEP_TEST_APP) +@pytest.mark.usefixtures('test_app_copy') +def test_configdep_selective_rebuild(idf_py: IdfPyFunc) -> None: + """Verify selective rebuild when a Kconfig option toggles. + + ``esp-idf-configdep`` scans each translation unit's source for ``CONFIG_*`` + tokens and appends matching ``.cdep`` stubs to the compiler depfile. Avoid + spelling projbuild-only symbols (even in comments) in ``unrelated.c``, or the + scanner adds a spurious ``option.cdep`` edge and Ninja rebuilds that ``.obj``. + """ + logging.info('initial build with configdep enabled (default)') + idf_py('build') + + logging.info('toggle CONFIG_TEST_CONFIGDEP_OPTION from y to n') + replace_in_file('sdkconfig', 'CONFIG_TEST_CONFIGDEP_OPTION=y', '# CONFIG_TEST_CONFIGDEP_OPTION is not set') + + snapshot_main = get_snapshot(MAIN_OBJ) + snapshot_unrelated = get_snapshot(UNRELATED_OBJ) + + logging.info('rebuild after config change') + idf_py('build') + + logging.info('configdep_main.c.obj must be rebuilt (references changed option)') + snapshot_main.assert_different(get_snapshot(MAIN_OBJ)) + + logging.info('unrelated.c.obj must not be rebuilt (no CONFIG_* token for that option in source)') + snapshot_unrelated.assert_same(get_snapshot(UNRELATED_OBJ)) + + +@pytest.mark.test_app_copy(CONFIGDEP_TEST_APP) +@pytest.mark.usefixtures('test_app_copy') +def test_configdep_no_rebuild_without_change(idf_py: IdfPyFunc) -> None: + logging.info('initial build with configdep enabled (default)') + idf_py('build') + + snapshot_main = get_snapshot(MAIN_OBJ) + snapshot_unrelated = get_snapshot(UNRELATED_OBJ) + + logging.info('rebuild with no config changes') + idf_py('build') + + logging.info('neither object file should be rebuilt') + snapshot_main.assert_same(get_snapshot(MAIN_OBJ)) + snapshot_unrelated.assert_same(get_snapshot(UNRELATED_OBJ)) + + +@pytest.mark.test_app_copy(CONFIGDEP_TEST_APP) +@pytest.mark.usefixtures('test_app_copy') +def test_configdep_elf_strings_track_sdkconfig_toggle(idf_py: IdfPyFunc) -> None: + """After rebuild, the firmware string matches the final Kconfig (host-side ELF check).""" + idf_py('build') + assert CONFIGDEP_ELF.is_file() + assert bin_file_contains(CONFIGDEP_ELF, b'CONFIGDEP_OPTION: ENABLED') + assert not bin_file_contains(CONFIGDEP_ELF, b'CONFIGDEP_OPTION: DISABLED') + + append_to_file('sdkconfig', '\n# CONFIG_TEST_CONFIGDEP_OPTION is not set\n') + idf_py('build') + assert bin_file_contains(CONFIGDEP_ELF, b'CONFIGDEP_OPTION: DISABLED') + assert not bin_file_contains(CONFIGDEP_ELF, b'CONFIGDEP_OPTION: ENABLED') diff --git a/tools/test_build_system/test_rebuild.py b/tools/test_build_system/test_rebuild.py index 355c6b69c6e..c6d4cf52904 100644 --- a/tools/test_build_system/test_rebuild.py +++ b/tools/test_build_system/test_rebuild.py @@ -1,20 +1,51 @@ # SPDX-FileCopyrightText: 2022-2026 Espressif Systems (Shanghai) CO LTD # SPDX-License-Identifier: Apache-2.0 # These tests check whether the build system rebuilds some files or not -# depending on the changes to the project. +# depending on the changes to the project. Configdep is disabled here so +# mtime assertions reflect CMake/Ninja only; see test_rebuild_configdep.py for +# the same checks with configdep enabled (after settle builds). import logging import os +import subprocess from pathlib import Path +from typing import cast import pytest from test_build_system_helpers import ALL_ARTIFACTS_BUILDV1 from test_build_system_helpers import ALL_ARTIFACTS_BUILDV2 from test_build_system_helpers import APP_BINS from test_build_system_helpers import BOOTLOADER_BINS +from test_build_system_helpers import BUILD_SNAPSHOT_EXCLUDE from test_build_system_helpers import PARTITION_BIN from test_build_system_helpers import IdfPyFunc +from test_build_system_helpers import get_idf_build_env from test_build_system_helpers import get_snapshot +from test_build_system_helpers import rebuild_and_check from test_build_system_helpers import replace_in_file +from test_build_system_helpers import run_idf_py + + +@pytest.fixture +def idf_py(monkeypatch: pytest.MonkeyPatch) -> IdfPyFunc: + """Use ``idf.py`` with esp-idf-configdep disabled for this module. + + These tests assert Ninja rebuild granularity via file mtimes. The + ``esp-idf-configdep`` wrapper can create per-option ``.cdep`` stubs after the + real compiler has written ``.obj`` files, so on the next Ninja run inputs can + look newer than outputs and trigger spurious rebuilds. That behavior is + covered separately in ``test_rebuild_configdep.py`` with a settle-build + pattern. Here we exercise the core CMake/Ninja dependency graph only. + """ + monkeypatch.setenv('IDF_CONFIGDEP_ENABLE', '0') + env = get_idf_build_env(os.environ['IDF_PATH']) + + def result(*args: str, check: bool = True, input_str: str | None = None) -> subprocess.CompletedProcess[str]: + return cast( + subprocess.CompletedProcess[str], + run_idf_py(*args, env=env, workdir=os.getcwd(), check=check, input_str=input_str), + ) + + return result @pytest.mark.usefixtures('test_app_copy') @@ -23,14 +54,7 @@ def test_rebuild_no_changes(idf_py: IdfPyFunc, request: pytest.FixtureRequest) - idf_py('build') logging.info('get the first snapshot') # excluding the 'log' subdirectory here since it changes after every build - all_build_files = get_snapshot( - 'build/**/*', - exclude_patterns=[ - 'build/log/*', - 'build/CMakeFiles/bootloader-complete', - 'build/bootloader-prefix/src/bootloader-stamp/bootloader-done', - ], - ) + all_build_files = get_snapshot('build/**/*', exclude_patterns=BUILD_SNAPSHOT_EXCLUDE) logging.info('check that all build artifacts were generated') all_artifacts = ALL_ARTIFACTS_BUILDV2 if request.config.getoption('buildv2', False) else ALL_ARTIFACTS_BUILDV1 @@ -40,33 +64,10 @@ def test_rebuild_no_changes(idf_py: IdfPyFunc, request: pytest.FixtureRequest) - logging.info('build again with no changes') idf_py('build') # if there are no changes, nothing gets rebuilt - all_build_files_after_rebuild = get_snapshot( - 'build/**/*', - exclude_patterns=[ - 'build/log/*', - 'build/CMakeFiles/bootloader-complete', - 'build/bootloader-prefix/src/bootloader-stamp/bootloader-done', - ], - ) + all_build_files_after_rebuild = get_snapshot('build/**/*', exclude_patterns=BUILD_SNAPSHOT_EXCLUDE) all_build_files_after_rebuild.assert_same(all_build_files) -def rebuild_and_check( - idf_py: IdfPyFunc, should_be_rebuilt: str | list[str], should_not_be_rebuilt: str | list[str] -) -> None: - """ - Helper function for the test cases below. - Asserts that the files matching 'should_be_rebuilt' patterns are rebuilt - and files matching 'should_not_be_rebuilt' patterns aren't, after - touching (updating the mtime) of the given 'file_to_touch' and rebuilding. - """ - snapshot_should_be_rebuilt = get_snapshot(should_be_rebuilt) - snapshot_should_not_be_rebuilt = get_snapshot(should_not_be_rebuilt) - idf_py('build') - snapshot_should_be_rebuilt.assert_different(get_snapshot(should_be_rebuilt)) - snapshot_should_not_be_rebuilt.assert_same(get_snapshot(should_not_be_rebuilt)) - - # For this and the following test function, there are actually multiple logical # tests in one test function. It would be better to have every check in a separate # test case, but that would mean doing a full clean build each time. Having a few @@ -104,7 +105,7 @@ def test_rebuild_source_files(idf_py: IdfPyFunc) -> None: # pick on .c, .cpp, .S file which includes sdkconfig.h: obj_files = [ 'build/esp-idf/esp_libc/CMakeFiles/__idf_esp_libc.dir/newlib_init.c.obj', - 'build/esp-idf/nvs_flash/CMakeFiles/__idf_nvs_flash.dir/src/nvs_api.cpp.obj' + 'build/esp-idf/nvs_flash/CMakeFiles/__idf_nvs_flash.dir/src/nvs_api.cpp.obj', 'build/esp-idf/esp_system/CMakeFiles/__idf_esp_system.dir/port/arch/xtensa/panic_handler_asm.S.obj', ] sdkconfig_files = ['build/config/sdkconfig.h', 'build/config/sdkconfig.json'] diff --git a/tools/test_build_system/test_rebuild_configdep.py b/tools/test_build_system/test_rebuild_configdep.py new file mode 100644 index 00000000000..72d401a5e9e --- /dev/null +++ b/tools/test_build_system/test_rebuild_configdep.py @@ -0,0 +1,108 @@ +# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD +# SPDX-License-Identifier: Apache-2.0 +"""Rebuild tests with esp-idf-configdep enabled (default idf.py behavior). + +esp-idf-configdep rewrites compiler .d files to depend on per-option +build/config/.../*.cdep stubs instead of sdkconfig.h. When a referenced +.cdep file does not exist yet, the wrapper creates it **after** the real +compiler has written the .obj. Those stubs then have a newer mtime than the +object. On the **next** Ninja run, those .cdep paths are prerequisites in the +merged dependency graph, so Ninja correctly treats the object as out of date and +rebuilds it—even though nothing in the source tree or sdkconfig changed. + +That one-time “settling” wave is not a CMake bug; it is an artifact of how +configdep materializes stubs relative to object timestamps. The tests in +test_rebuild.py therefore disable configdep to exercise pure CMake/Ninja +incremental behavior. Here we keep configdep **on** and run **two** full +idf.py build invocations before any mtime baseline so stub/object mtimes are +consistent; then we apply the same coarse-grained checks as the classic rebuild +tests (plus relaxed expectations where configdep legitimately skips compiling +TUs that do not reference changed CONFIG_* symbols). +""" + +import logging +import os +from pathlib import Path + +import pytest +from test_build_system_helpers import ALL_ARTIFACTS_BUILDV1 +from test_build_system_helpers import ALL_ARTIFACTS_BUILDV2 +from test_build_system_helpers import APP_BINS +from test_build_system_helpers import BOOTLOADER_BINS +from test_build_system_helpers import BUILD_SNAPSHOT_EXCLUDE +from test_build_system_helpers import PARTITION_BIN +from test_build_system_helpers import IdfPyFunc +from test_build_system_helpers import get_snapshot +from test_build_system_helpers import rebuild_and_check +from test_build_system_helpers import replace_in_file + + +def _settle_configdep_graph(idf_py: IdfPyFunc) -> None: + """Two full builds so ``.cdep`` stubs created by configdep are not newer than ``.obj`` for the next graph walk.""" + logging.info('settle (configdep): first full build') + idf_py('build') + logging.info('settle (configdep): second full build reconciles obj vs .cdep mtimes') + idf_py('build') + + +@pytest.mark.usefixtures('test_app_copy') +def test_rebuild_no_changes_with_configdep(idf_py: IdfPyFunc, request: pytest.FixtureRequest) -> None: + """No-op rebuild: with configdep on, baseline snapshot only after two settle builds (see module docstring).""" + _settle_configdep_graph(idf_py) + + logging.info('baseline snapshot after settle') + all_build_files = get_snapshot('build/**/*', exclude_patterns=BUILD_SNAPSHOT_EXCLUDE) + + logging.info('check that all build artifacts were generated') + all_artifacts = ALL_ARTIFACTS_BUILDV2 if request.config.getoption('buildv2', False) else ALL_ARTIFACTS_BUILDV1 + for artifact in all_artifacts: + assert Path(artifact).exists() + + logging.info('noop build with configdep; mtimes under baseline should match') + idf_py('build') + all_build_files_after = get_snapshot('build/**/*', exclude_patterns=BUILD_SNAPSHOT_EXCLUDE) + all_build_files_after.assert_same(all_build_files) + + +@pytest.mark.usefixtures('test_app_copy') +def test_rebuild_source_files_with_configdep(idf_py: IdfPyFunc) -> None: + """Same selective-rebuild checks as ``test_rebuild_source_files``, after configdep settle builds.""" + idf_path = Path(os.environ['IDF_PATH']) + _settle_configdep_graph(idf_py) + + logging.info('updating a component source file rebuilds only that component') + component_files_patterns = [ + 'build/esp-idf/esp_system/libesp_system.a', + 'build/esp-idf/esp_system/CMakeFiles/__idf_esp_system.dir/port/cpu_start.c.obj', + ] + other_files_patterns = [ + 'build/esp-idf/lwip/liblwip.a', + 'build/esp-idf/freertos/libfreertos.a', + *BOOTLOADER_BINS, + *PARTITION_BIN, + ] + (idf_path / 'components/esp_system/port/cpu_start.c').touch() + rebuild_and_check(idf_py, component_files_patterns, other_files_patterns) + + logging.info('changing a bootloader source file rebuilds the bootloader') + (idf_path / 'components/bootloader/subproject/main/bootloader_start.c').touch() + rebuild_and_check(idf_py, BOOTLOADER_BINS, APP_BINS + PARTITION_BIN) + + logging.info('changing the partitions CSV file rebuilds only the partition table') + (idf_path / 'components/partition_table/partitions_singleapp.csv').touch() + rebuild_and_check(idf_py, PARTITION_BIN, APP_BINS + BOOTLOADER_BINS) + + logging.info( + 'sdkconfig update: binaries and generated headers must refresh (objects are selective under configdep)' + ) + sdkconfig_files = ['build/config/sdkconfig.h', 'build/config/sdkconfig.json'] + replace_in_file('sdkconfig', '# CONFIG_FREERTOS_UNICORE is not set', 'CONFIG_FREERTOS_UNICORE=y') + rebuild_and_check(idf_py, APP_BINS + BOOTLOADER_BINS + sdkconfig_files, PARTITION_BIN) + + logging.info('Updating project CMakeLists.txt triggers app re-link at minimum') + replace_in_file( + 'CMakeLists.txt', + '# placeholder_after_include_project_cmake', + 'add_compile_options("-DUSELESS_MACRO_DOES_NOTHING=1")', + ) + rebuild_and_check(idf_py, APP_BINS, PARTITION_BIN + BOOTLOADER_BINS + sdkconfig_files) diff --git a/tools/test_idf_tools/test_idf_tools.py b/tools/test_idf_tools/test_idf_tools.py index bce1276151b..f2a4e8f2920 100755 --- a/tools/test_idf_tools/test_idf_tools.py +++ b/tools/test_idf_tools/test_idf_tools.py @@ -65,6 +65,7 @@ NINJA = 'ninja' IDF_EXE = 'idf-exe' CCACHE = 'ccache' DFU_UTIL = 'dfu-util' +ESP_IDF_CONFIGDEP = 'esp-idf-configdep' def get_version_dict(): @@ -98,6 +99,7 @@ NINJA_VERSION = version_dict[NINJA] IDF_EXE_VERSION = version_dict[IDF_EXE] CCACHE_VERSION = version_dict[CCACHE] DFU_UTIL_VERSION = version_dict[DFU_UTIL] +ESP_IDF_CONFIGDEP_VERSION = version_dict[ESP_IDF_CONFIGDEP] # There are some complex search patterns to detect download snippets @@ -383,7 +385,7 @@ class TestUsageUnix(TestUsage): self.assertIn(f'* {CLANGD}:', output) self.assertIn(f'- {CLANGD_VERSION} (recommended)', output) - required_tools_installed = 8 + required_tools_installed = 9 output = self.run_idf_tools_with_action(['install']) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) self.assert_tool_installed(output, RISCV_ELF, RISCV_ELF_VERSION, RISCV_ELF_ARCHIVE_PATTERN) @@ -393,6 +395,7 @@ class TestUsageUnix(TestUsage): self.assert_tool_installed(output, RISCV_ESP_GDB, RISCV_ESP_GDB_VERSION) self.assert_tool_installed(output, ESP_ROM_ELFS, ESP_ROM_ELFS_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -406,6 +409,7 @@ class TestUsageUnix(TestUsage): RISCV_ESP_GDB_VERSION, ESP_ROM_ELFS_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) @@ -424,6 +428,11 @@ class TestUsageUnix(TestUsage): ) self.assertIn(f'{self.temp_tools_dir}/tools/esp-rom-elfs/{ESP_ROM_ELFS_VERSION}/', output) self.assertIn(f'{self.temp_tools_dir}/tools/esp-clangd/{CLANGD_VERSION}/esp-clangd/bin', output) + self.assertIn( + f'{self.temp_tools_dir}/tools/{ESP_IDF_CONFIGDEP}/{ESP_IDF_CONFIGDEP_VERSION}/' + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}/bin', + output, + ) output = self.run_idf_tools_with_action(['list', '--outdated']) self.assertEqual('', output) @@ -451,7 +460,7 @@ class TestUsageUnix(TestUsage): self.assertIn((f'{XTENSA_ELF}: version {XTENSA_ELF_VERSION} is outdated by {new_version}'), output) def test_tools_for_esp32(self): - required_tools_installed = 6 + required_tools_installed = 7 output = self.run_idf_tools_with_action(['install', '--targets=esp32']) self.assert_tool_installed(output, XTENSA_ELF, XTENSA_ELF_VERSION, XTENSA_ELF_ARCHIVE_PATTERN) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) @@ -461,6 +470,7 @@ class TestUsageUnix(TestUsage): self.assert_tool_not_installed(output, RISCV_ESP_GDB, RISCV_ESP_GDB_VERSION) self.assert_tool_installed(output, ESP_ROM_ELFS, ESP_ROM_ELFS_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -472,6 +482,7 @@ class TestUsageUnix(TestUsage): XTENSA_ESP_GDB_VERSION, ESP_ROM_ELFS_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) @@ -490,9 +501,14 @@ class TestUsageUnix(TestUsage): ) self.assertIn(f'{self.temp_tools_dir}/tools/esp-rom-elfs/{ESP_ROM_ELFS_VERSION}/', output) self.assertIn(f'{self.temp_tools_dir}/tools/esp-clangd/{CLANGD_VERSION}/esp-clangd/bin', output) + self.assertIn( + f'{self.temp_tools_dir}/tools/{ESP_IDF_CONFIGDEP}/{ESP_IDF_CONFIGDEP_VERSION}/' + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}/bin', + output, + ) def test_tools_for_esp32c3(self): - required_tools_installed = 5 + required_tools_installed = 6 output = self.run_idf_tools_with_action(['install', '--targets=esp32c3']) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) self.assert_tool_installed(output, RISCV_ELF, RISCV_ELF_VERSION, RISCV_ELF_ARCHIVE_PATTERN) @@ -502,11 +518,18 @@ class TestUsageUnix(TestUsage): self.assert_tool_not_installed(output, XTENSA_ESP_GDB, XTENSA_ESP_GDB_VERSION) self.assert_tool_installed(output, ESP_ROM_ELFS, ESP_ROM_ELFS_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) output = self.run_idf_tools_with_action(['check']) - for tool_version in [OPENOCD_VERSION, RISCV_ELF_VERSION, RISCV_ESP_GDB_VERSION, ESP_ROM_ELFS_VERSION]: + for tool_version in [ + OPENOCD_VERSION, + RISCV_ELF_VERSION, + RISCV_ESP_GDB_VERSION, + ESP_ROM_ELFS_VERSION, + ESP_IDF_CONFIGDEP_VERSION, + ]: self.assertIn('version installed in tools directory: ' + tool_version, output) output = self.run_idf_tools_with_action(['export']) @@ -520,9 +543,14 @@ class TestUsageUnix(TestUsage): ) self.assertIn(f'{self.temp_tools_dir}/tools/esp-rom-elfs/{ESP_ROM_ELFS_VERSION}/', output) self.assertIn(f'{self.temp_tools_dir}/tools/esp-clangd/{CLANGD_VERSION}/esp-clangd/bin', output) + self.assertIn( + f'{self.temp_tools_dir}/tools/{ESP_IDF_CONFIGDEP}/{ESP_IDF_CONFIGDEP_VERSION}/' + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}/bin', + output, + ) def test_tools_for_esp32s2(self): - required_tools_installed = 7 + required_tools_installed = 8 output = self.run_idf_tools_with_action(['install', '--targets=esp32s2']) self.assert_tool_installed(output, XTENSA_ELF, XTENSA_ELF_VERSION, XTENSA_ELF_ARCHIVE_PATTERN) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) @@ -532,11 +560,18 @@ class TestUsageUnix(TestUsage): self.assert_tool_installed(output, ESP_ROM_ELFS, ESP_ROM_ELFS_VERSION) self.assert_tool_not_installed(output, RISCV_ESP_GDB, RISCV_ESP_GDB_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) output = self.run_idf_tools_with_action(['check']) - for tool_version in [OPENOCD_VERSION, XTENSA_ELF_VERSION, XTENSA_ESP_GDB_VERSION, ESP_ROM_ELFS_VERSION]: + for tool_version in [ + OPENOCD_VERSION, + XTENSA_ELF_VERSION, + XTENSA_ESP_GDB_VERSION, + ESP_ROM_ELFS_VERSION, + ESP_IDF_CONFIGDEP_VERSION, + ]: self.assertIn('version installed in tools directory: ' + tool_version, output) output = self.run_idf_tools_with_action(['export']) @@ -554,9 +589,14 @@ class TestUsageUnix(TestUsage): ) self.assertIn(f'{self.temp_tools_dir}/tools/esp-rom-elfs/{ESP_ROM_ELFS_VERSION}/', output) self.assertIn(f'{self.temp_tools_dir}/tools/esp-clangd/{CLANGD_VERSION}/esp-clangd/bin', output) + self.assertIn( + f'{self.temp_tools_dir}/tools/{ESP_IDF_CONFIGDEP}/{ESP_IDF_CONFIGDEP_VERSION}/' + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}/bin', + output, + ) def test_tools_for_esp32s3(self): - required_tools_installed = 7 + required_tools_installed = 8 output = self.run_idf_tools_with_action(['install', '--targets=esp32s3']) self.assert_tool_installed(output, XTENSA_ELF, XTENSA_ELF_VERSION, XTENSA_ELF_ARCHIVE_PATTERN) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) @@ -566,6 +606,7 @@ class TestUsageUnix(TestUsage): self.assert_tool_installed(output, ESP_ROM_ELFS, ESP_ROM_ELFS_VERSION) self.assert_tool_not_installed(output, RISCV_ESP_GDB, RISCV_ESP_GDB_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -577,6 +618,7 @@ class TestUsageUnix(TestUsage): RISCV_ESP_GDB_VERSION, ESP_ROM_ELFS_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) @@ -595,9 +637,14 @@ class TestUsageUnix(TestUsage): ) self.assertIn(f'{self.temp_tools_dir}/tools/esp-rom-elfs/{ESP_ROM_ELFS_VERSION}/', output) self.assertIn(f'{self.temp_tools_dir}/tools/esp-clangd/{CLANGD_VERSION}/esp-clangd/bin', output) + self.assertIn( + f'{self.temp_tools_dir}/tools/{ESP_IDF_CONFIGDEP}/{ESP_IDF_CONFIGDEP_VERSION}/' + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}/bin', + output, + ) def test_tools_for_esp32p4(self): - required_tools_installed = 5 + required_tools_installed = 6 output = self.run_idf_tools_with_action(['install', '--targets=esp32p4']) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) self.assert_tool_installed(output, RISCV_ELF, RISCV_ELF_VERSION, RISCV_ELF_ARCHIVE_PATTERN) @@ -607,10 +654,17 @@ class TestUsageUnix(TestUsage): self.assert_tool_not_installed(output, ESP32ULP, ESP32ULP_VERSION) self.assert_tool_not_installed(output, XTENSA_ESP_GDB, XTENSA_ESP_GDB_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertEqual(required_tools_installed, output.count('Done')) output = self.run_idf_tools_with_action(['check']) - for tool_version in [OPENOCD_VERSION, RISCV_ELF_VERSION, ESP_ROM_ELFS_VERSION, RISCV_ESP_GDB_VERSION]: + for tool_version in [ + OPENOCD_VERSION, + RISCV_ELF_VERSION, + ESP_ROM_ELFS_VERSION, + RISCV_ESP_GDB_VERSION, + ESP_IDF_CONFIGDEP_VERSION, + ]: self.assertIn(f'version installed in tools directory: {tool_version}', output) output = self.run_idf_tools_with_action(['export']) @@ -623,6 +677,17 @@ class TestUsageUnix(TestUsage): os.path.join(self.temp_tools_dir, 'tools', RISCV_ESP_GDB, RISCV_ESP_GDB_VERSION, RISCV_ESP_GDB, 'bin'), output, ) + self.assertIn( + os.path.join( + self.temp_tools_dir, + 'tools', + ESP_IDF_CONFIGDEP, + ESP_IDF_CONFIGDEP_VERSION, + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}', + 'bin', + ), + output, + ) self.assertNotIn( os.path.join(self.temp_tools_dir, 'tools', ESP32ULP, ESP32ULP_VERSION, ESP32ULP, 'bin'), output ) @@ -637,7 +702,7 @@ class TestUsageUnix(TestUsage): # a different test for qemu because of "on_request" def test_tools_for_qemu_with_required(self): - required_tools_installed = 10 + required_tools_installed = 11 output = self.run_idf_tools_with_action(['install', 'required', 'qemu-xtensa', 'qemu-riscv32']) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) self.assert_tool_installed(output, RISCV_ELF, RISCV_ELF_VERSION, RISCV_ELF_ARCHIVE_PATTERN) @@ -646,6 +711,7 @@ class TestUsageUnix(TestUsage): self.assert_tool_installed(output, XTENSA_ESP_GDB, XTENSA_ESP_GDB_VERSION) self.assert_tool_installed(output, RISCV_ESP_GDB, RISCV_ESP_GDB_VERSION) self.assert_tool_installed(output, ESP_ROM_ELFS, ESP_ROM_ELFS_VERSION) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assert_tool_installed(output, QEMU_RISCV, QEMU_RISCV_VERSION) self.assert_tool_installed(output, QEMU_XTENSA, QEMU_XTENSA_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) @@ -675,7 +741,8 @@ class TestUsageWin(TestUsage): self.assertIn(f'- {XTENSA_ELF_VERSION} (recommended)', output) self.assertIn(f'* {CLANGD}:', output) self.assertIn(f'- {CLANGD_VERSION} (recommended)', output) - required_tools_installed = 12 + + required_tools_installed = 13 output = self.run_idf_tools_with_action(['install']) self.assert_tool_installed(output, OPENOCD, OPENOCD_VERSION) self.assert_tool_installed(output, RISCV_ELF, RISCV_ELF_VERSION, RISCV_ELF_ARCHIVE_PATTERN) @@ -690,6 +757,7 @@ class TestUsageWin(TestUsage): self.assert_tool_installed(output, CCACHE, CCACHE_VERSION) self.assert_tool_installed(output, DFU_UTIL, DFU_UTIL_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -708,6 +776,7 @@ class TestUsageWin(TestUsage): CCACHE_VERSION, DFU_UTIL_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) @@ -749,6 +818,17 @@ class TestUsageWin(TestUsage): os.path.join(self.temp_tools_dir, 'tools', DFU_UTIL, DFU_UTIL_VERSION, 'dfu-util-0.11-win64'), output ) self.assertIn(os.path.join(self.temp_tools_dir, 'tools', CLANGD, CLANGD_VERSION, CLANGD, 'bin'), output) + self.assertIn( + os.path.join( + self.temp_tools_dir, + 'tools', + ESP_IDF_CONFIGDEP, + ESP_IDF_CONFIGDEP_VERSION, + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}', + 'bin', + ), + output, + ) output = self.run_idf_tools_with_action(['list', '--outdated']) self.assertEqual('', output) @@ -791,6 +871,7 @@ class TestUsageWin(TestUsage): self.assert_tool_installed(output, CCACHE, CCACHE_VERSION) self.assert_tool_not_installed(output, DFU_UTIL, DFU_UTIL_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -806,6 +887,7 @@ class TestUsageWin(TestUsage): IDF_EXE_VERSION, CCACHE_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) self.assertNotIn('version installed in tools directory: ' + DFU_UTIL_VERSION, output) @@ -856,6 +938,17 @@ class TestUsageWin(TestUsage): os.path.join(self.temp_tools_dir, 'tools', DFU_UTIL, DFU_UTIL_VERSION, 'dfu-util-0.11-win64'), output ) self.assertNotIn(os.path.join(self.temp_tools_dir, 'tools', CLANGD, CLANGD_VERSION, CLANGD, 'bin'), output) + self.assertIn( + os.path.join( + self.temp_tools_dir, + 'tools', + ESP_IDF_CONFIGDEP, + ESP_IDF_CONFIGDEP_VERSION, + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}', + 'bin', + ), + output, + ) def test_tools_for_esp32c3_win(self): required_tools_installed = 9 @@ -873,6 +966,7 @@ class TestUsageWin(TestUsage): self.assert_tool_installed(output, CCACHE, CCACHE_VERSION) self.assert_tool_not_installed(output, DFU_UTIL, DFU_UTIL_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -887,6 +981,7 @@ class TestUsageWin(TestUsage): IDF_EXE_VERSION, CCACHE_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) self.assertNotIn('version installed in tools directory: ' + DFU_UTIL_VERSION, output) @@ -931,6 +1026,17 @@ class TestUsageWin(TestUsage): os.path.join(self.temp_tools_dir, 'tools', DFU_UTIL, DFU_UTIL_VERSION, 'dfu-util-0.11-win64'), output ) self.assertNotIn(os.path.join(self.temp_tools_dir, 'tools', CLANGD, CLANGD_VERSION, CLANGD, 'bin'), output) + self.assertIn( + os.path.join( + self.temp_tools_dir, + 'tools', + ESP_IDF_CONFIGDEP, + ESP_IDF_CONFIGDEP_VERSION, + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}', + 'bin', + ), + output, + ) def test_tools_for_esp32s2_win(self): required_tools_installed = 12 @@ -948,6 +1054,7 @@ class TestUsageWin(TestUsage): self.assert_tool_installed(output, CCACHE, CCACHE_VERSION) self.assert_tool_installed(output, DFU_UTIL, DFU_UTIL_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -963,6 +1070,7 @@ class TestUsageWin(TestUsage): IDF_EXE_VERSION, CCACHE_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) @@ -1012,6 +1120,17 @@ class TestUsageWin(TestUsage): os.path.join(self.temp_tools_dir, 'tools', DFU_UTIL, DFU_UTIL_VERSION, 'dfu-util-0.11-win64'), output ) self.assertIn(os.path.join(self.temp_tools_dir, 'tools', CLANGD, CLANGD_VERSION, CLANGD, 'bin'), output) + self.assertIn( + os.path.join( + self.temp_tools_dir, + 'tools', + ESP_IDF_CONFIGDEP, + ESP_IDF_CONFIGDEP_VERSION, + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}', + 'bin', + ), + output, + ) def test_tools_for_esp32s3_win(self): required_tools_installed = 12 @@ -1030,6 +1149,7 @@ class TestUsageWin(TestUsage): self.assert_tool_installed(output, CCACHE, CCACHE_VERSION) self.assert_tool_installed(output, DFU_UTIL, DFU_UTIL_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) @@ -1046,6 +1166,7 @@ class TestUsageWin(TestUsage): IDF_EXE_VERSION, CCACHE_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn('version installed in tools directory: ' + tool_version, output) @@ -1095,6 +1216,17 @@ class TestUsageWin(TestUsage): os.path.join(self.temp_tools_dir, 'tools', DFU_UTIL, DFU_UTIL_VERSION, 'dfu-util-0.11-win64'), output ) self.assertIn(os.path.join(self.temp_tools_dir, 'tools', CLANGD, CLANGD_VERSION, CLANGD, 'bin'), output) + self.assertIn( + os.path.join( + self.temp_tools_dir, + 'tools', + ESP_IDF_CONFIGDEP, + ESP_IDF_CONFIGDEP_VERSION, + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}', + 'bin', + ), + output, + ) def test_tools_for_esp32p4_win(self): required_tools_installed = 9 @@ -1112,6 +1244,7 @@ class TestUsageWin(TestUsage): self.assert_tool_not_installed(output, XTENSA_ESP_GDB, XTENSA_ESP_GDB_VERSION) self.assert_tool_not_installed(output, DFU_UTIL, DFU_UTIL_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertEqual(required_tools_installed, output.count('Done')) output = self.run_idf_tools_with_action(['check']) @@ -1125,6 +1258,7 @@ class TestUsageWin(TestUsage): IDF_EXE_VERSION, CCACHE_VERSION, CLANGD_VERSION, + ESP_IDF_CONFIGDEP_VERSION, ]: self.assertIn(f'version installed in tools directory: {tool_version}', output) @@ -1144,6 +1278,17 @@ class TestUsageWin(TestUsage): self.assertIn( os.path.join(self.temp_tools_dir, 'tools', CCACHE, CCACHE_VERSION, 'ccache-4.12.1-windows-x86_64'), output ) + self.assertIn( + os.path.join( + self.temp_tools_dir, + 'tools', + ESP_IDF_CONFIGDEP, + ESP_IDF_CONFIGDEP_VERSION, + f'{ESP_IDF_CONFIGDEP}-{ESP_IDF_CONFIGDEP_VERSION}', + 'bin', + ), + output, + ) self.assertNotIn( os.path.join(self.temp_tools_dir, 'tools', XTENSA_ELF, XTENSA_ELF_VERSION, XTENSA_ELF, 'bin'), output ) @@ -1178,6 +1323,7 @@ class TestUsageWin(TestUsage): self.assert_tool_installed(output, CCACHE, CCACHE_VERSION) self.assert_tool_installed(output, DFU_UTIL, DFU_UTIL_VERSION) self.assert_tool_installed(output, CLANGD, CLANGD_VERSION, CLANGD_ARCHIVE) + self.assert_tool_installed(output, ESP_IDF_CONFIGDEP, ESP_IDF_CONFIGDEP_VERSION) self.assertIn('Destination: {}'.format(os.path.join(self.temp_tools_dir, 'dist')), output) self.assertEqual(required_tools_installed, output.count('Done')) diff --git a/tools/tools.json b/tools/tools.json index c113028c43a..578d26b835c 100644 --- a/tools/tools.json +++ b/tools/tools.json @@ -1150,6 +1150,79 @@ } } ] + }, + { + "description": "ESP-IDF configuration dependency tool", + "export_paths": [ + [ + "esp-idf-configdep-0.2.2", + "bin" + ] + ], + "export_vars": {}, + "info_url": "https://github.com/espressif/esp-idf-configdep", + "install": "always", + "license": "Apache-2.0", + "name": "esp-idf-configdep", + "supported_targets": [ + "all" + ], + "version_cmd": [ + "esp-idf-configdep", + "--version" + ], + "version_regex": "version: ([0-9.]+)", + "versions": [ + { + "linux-amd64": { + "sha256": "999aa895c25d5c7edb6823884d3017f146b133f7ae2c580fffe4c989d7ce8c36", + "size": 28505, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-linux-x86_64.tar.gz" + }, + "linux-arm64": { + "sha256": "0ea8f5e0de463ecfa3354a27b32ba34c4b999570e92127952fe7363f034ab7e9", + "size": 59608, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-linux-arm64.tar.gz" + }, + "linux-armel": { + "sha256": "0e1663a6f9d8c030f11feb214aab0ffc3b9f6029ebc7ce10cad974db13667632", + "size": 36797, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-linux-armel.tar.gz" + }, + "linux-armhf": { + "sha256": "98759c0a16e56ec96cc72f2d6b60d2f548194cfcc2adde11cd65c694241912b6", + "size": 34310, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-linux-armhf.tar.gz" + }, + "linux-i686": { + "sha256": "e337582b1570ac90fcfde0b724e41d62b01581ec0a755c97ac38024c5def55ac", + "size": 32114, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-linux-x86.tar.gz" + }, + "macos": { + "sha256": "8e7ad2fd91214219c276c6b371f0cd3e315f22055554d60856e88d84eb6422e9", + "size": 12755, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-macos-x86_64.tar.gz" + }, + "macos-arm64": { + "sha256": "458fa0e3193184ca6360bf35058c63e763545f0ef6cb0b659b01985a5dfafdbf", + "size": 13023, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-macos-arm64.tar.gz" + }, + "name": "0.2.2", + "status": "recommended", + "win32": { + "sha256": "6bec5711daa5196df8ec3fa2d3c2c79623e7dc6cb5e0b6cede03ef49b71ac6ac", + "size": 114156, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-win-x86.zip" + }, + "win64": { + "sha256": "1827c449c9294036f94fa82f5da65f10be1edbc2de6920ce9837cbee1393cac8", + "size": 117734, + "url": "https://github.com/espressif/esp-idf-configdep/releases/download/v0.2.2/esp-idf-configdep-0.2.2-win-x86_64.zip" + } + } + ] } ], "version": 3