Changes generated by pre-commit related to ruff-format and an update to
the copyright date.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce ESPTOOLPY_FAST_REFLASHING configuration option. It instructs
ldgen to group entity mappings for libraries deemed mutable (prone to
change) separately from those considered immutable (unlikely to change).
Organizing mutable and immutable libraries separately in the linker
script allows the linker to form larger contiguous blocks of data for
immutable libraries in the application's output sections. These blocks
are likely to stay mostly unchanged between application recompilations,
enabling them to be skipped during reflashing.
Separating mutable and immutable libraries in the linker script to
minimize changes in the output sections is insufficient. Padding is
added after the input sections of mutable libraries in the default data
and text output sections. This creates a buffer for the mutable
libraries, allowing additional changes to be made without altering the
layout of the binary image.
Additionally two optimizations currently in use can still mix data from
these libraries, leading to significant changes even within the grouped
immutable libraries.
1. constant merging
Linker will try to merge input sections that have the MERGE and
STRING flags from different libraries (object files) to perform
optimizations like tail merging. For example, adding a string
literal in a mutable library will also change the addresses of
string literals from immutable libraries in such a merged section,
causing changes in the generated code when those literals are
referenced.
Disabled with COMPILER_DISABLE_MERGE_CONSTANTS(-fno-merge-constants)
2. literal pools on Xtensa
As optimization, the linker may merge literal pools from different
libraries (object files) to improve the generated code size. This
has the same effect as constant merging, and changes in mutable
libraries may cause changes in the generated code for immutable
libraries. To get larger unchanged continuous blocks in the text
output sections for immutable libraries, we need to ensure that the
Xtensa literal pools remain close to their references and are not
merged.
Disabled with CONFIG_COMPILER_ENABLE_TEXT_SECTION_LITERALS(-mtext-section-literals)
When ESPTOOLPY_FAST_REFLASHING is enabled, these two optimizations are
disabled to achieve larger unchaged continuous blocks for the grouped
immutable libraries, even though disabling these optimizations results
in slightly larger code.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The build system keeps track of each component source. Currently
there are four types of sources:
1. "project_components" - project components
2. "project_extra_components" - components from EXTRA_COMPONENT_DIRS
3. "project_managed_components" - custom project dependencies managed by the IDF Component Manager
4. "idf_components" - ESP-IDF built-in components, typically under /components
This can be used to identify the component libraries that are likely to
change during application development and pass them to ldgen as mutable
libraries. Add all components with "project_components" as their source
as mutable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the `--debug` option to print the entity tree to stdout. This can be
useful for viewing the entire entity tree state to identify potential
issues or incorrect placements.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce a new `--mutable-libraries-file` option that accepts a file
containing the filenames of mutable libraries, each listed on a separate
line. Mutable libraries are component libraries expected to change
during development. In contrast, immutable component libraries are not
expected to change. In the generated linker script, the input sections
of mutable libraries are grouped together rather than being mixed with
those of immutable libraries. The goal is to create large continuous
areas in the ELF file's output sections that remain unchanged for
immutable libraries during application recompilation, allowing these
areas to be skipped during flashing.
The build system identifies the mutable libraries and passes them to
ldgen using the `--mutable-libraries-file` option. It maintains
information about component sources, one of which is
`project_components`. This source type identifies components that are
directly related to the project being developed and are very likely to
change.
Mappings for mutable libraries are explicitly created for all sections
in the default scheme. This happens before the entity
`(archive:object_file:symbol/input_section)` node tree with placements is
generated and is equivalent to having these mappings in the mapping
linker fragment. All placements for mappings, whether newly added or
already existing as defined in linker fragments, associated with mutable
libraries are flagged as `mutable` in the entity node tree. This flag
ensures that these placements are included in the final linker script.
Currently, ldgen only emits placements that are either significant or
forced. A placement is considered significant if, for example, it is not
already covered by a placement in parent node. For instance, `*(.iram1
.iram1.*)` placement already includes `*libapp_trace.a:(.iram1
.iram1.*)`, so the latter is not emitted by default. The `mutable` flag
ensures that placements for mutable libraries are emitted in the linker
script and placed at dedicated location.
The locations where placements for mutable libraries are specified in
the linker script are identified by a new `mutable` marker, for example,
`mutable[flash_text]`. The placements for immutable libraries remain in
the existing `mapping` marker, for example, `mapping[flash_text]`. The
`mutable` marker for each target is placed after the `mapping` marker.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Entity mappings for immutable libraries are placed in the existing
`mapping[target]` marker, while those for mutable libraries are placed
in the new `mutable[target]` marker, which comes after the `mapping`
marker. Additionally, include padding after the input sections of
mutable libraries in the default data and text output sections,
providing a headroom for the mutable libraries to grow. Padding is
currently not added, for example, in the `.iram0.data` output section,
which is not expected to change frequently. Padding for other mutable
input sections may be added in the future.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
