This fixes the issue that "idf.py partition-table" had to be run
manually in order for the partition table to be generated, when
building for linux target.
Since the introduction of PCAddressMatcher, the executable produced
by the build system is passed to elftools.elf.elffile.ELFFile.
However on macOS, native executables are not ELF files, so the
ELFFile class raises a rather unhelpful AssertionError exception.
Given that the rest of the idf_monitor.py doesn't have assumptions
that the "elf_file" argument is an ELF file (rather than just an
executable), check if the file is a real ELF file inside
PCAddressMatcher.
Previously get_idf_build_env didn't include the default environment
(os.environ) in its output, so the environment only contained the
variables returned by "idf_tools.py export".
If PATH already contains all the right paths, "idf_tools.py export"
doesn't return the PATH variable.
Therefore the environment returned by get_idf_build_env was usually
incomplete. Fix by combining the output of "idf_tools.py export" with
os.environ.
Checking that the host is macOS is not sufficient here, since the
linker is still a GNU linker when cross-compiling for a chip.
Instead, use the linker_type variable introduced in the previous
commit.
When compiling for a chip target with Clang,
CMAKE_C_COMPILER_ID="Clang" but the linker is still a GNU linker.
Therefore we can still generate the map file.
pytest_panic used to do 'from conftest import PanicTestDut'. This
stopped working when another conftest.py file was added in
tools/test_build_system/conftest.py.
In this case we can't rename conftest.py since both are necessary
for pytest to install the hooks in the respective test cases.
Fix by moving PanicTestDut into a separate module, then importing
it from there.
Unlike COMPILE_OPTIONS, COMPILE_DEFINITIONS CMake property assumes
values without the -D prefix, such as NAME or NAME=VAL.
Previously, IDF build system was passing COMPILE_DEFINITIONS build
property to CMake COMPILE_OPTIONS property, so -D prefix was not
a problem.
Now that COMPILE_DEFINITIONS CMake property is used, -D prefix has
to be removed.
(Note that this doesn't affect 'target_compile_definitions' function,
which strips -D prefix before adding the definition to the property.)
Now that the supported CMake version is >=3.16, this code can be
simplified.
The code to deduplicate the directories can be removed since this is
handled by target_link_directories.
dl.espressif.com is now using the same root certificate as github.com.
This commit replaces the previously-used ISRG X1 root certificate
with the DigiCert Root CA certificate.
As a result, even if the certificates are not installed (as it happens
on macOS with python.org installers, if the user forgets to run
'Install Certificates.command'), the download is successful.
Related to https://github.com/espressif/esp-idf/issues/4081
For usb_serial_jtag REPL only, xTaskCreate was passing a pointer to
esp_console_repl_com_t, while esp_console_repl_task was expecting
a pointer to esp_console_repl_universal_t.
The way the two structures are defined, this makes no difference, and
the pointer values are the same. Still, this could potentially break
in the future.
(I am not sure what is the distinction between repl_com (common?) and
repl_universal; it seems that `int uart_channel` could just as well
be part of esp_console_repl_com_t; alternatively, as suggested in the
previous commit, this structure could contain a callback function
pointer, which would allow `esp_console_new_repl_*` functions to
specify how stdin/stdout should be initialized by the REPL task.)
The crash occurred when calling setvbuf(stdin,...) with stdin==NULL.
This happened because esp_console_repl_task started running before
its args->uart_channel was initialized; then esp_console_repl_task
went into the code path 'uart_channel != CONFIG_ESP_CONSOLE_UART_NUM',
and tried to 'fopen("/dev/uart/0");`
Since the UART VFS is not registered when ESP_CONSOLE_USB_SERIAL_JTAG
option is enabled, fopen failed and 'stdin' was NULL.
Fix by moving the initialization of repl task arguments before the
start of the task, same as it is done for the usb_cdcacm case.
The crash started happening after the commit 287ab7566b. I haven’t
verified this, but I guess the reason why it wasn’t happening before
was that xTaskCreate was not correctly yielding to the newly created
higher-priority 'repl' task, therefore the code which was setting
the repl task arguments after xTaskCreate had time to execute.
It should be noted that the 'uart_channel' argument is a bit hacky,
in the first place. The code should be refactored to pass a callback
function to the repl task, and let this callback initialize stdin and
stdout based on the chosen console channel. Then esp_console_repl_task
does not require assumptions about the specific interface used.
Closes https://github.com/espressif/esp-idf/issues/9662
The actual output from the build tool (CMake/Ninja) may or may not
contain color escape codes, depending on various factors. The output
written to the log file should never include color escape codes,
though. This is because color escape codes in files are usually not
rendered as "color" in editors, and complicate reading. Also escape
codes would break the regular expressions used to display hints for
compilation errors.
If stdout is a TTY (meaning that the output is not redirected), tell
the build tool (GNU Make or Ninja) to enable colorized output.
GNU Make and Ninja also check if their stdout is redirected and
strip color escape sequences in that case. CLICOLOR_FORCE environment
variable overrides this behavior.
With this change, if the compiler was launched with the
-fcolor-diagnostics flag and idf.py output is not redirected, the
final output in the terminal will be colorized.
(-fcolor-diagnostics is handled at CMake level by the previous commit)
Related to https://github.com/espressif/esp-idf/issues/4162
Setting this option informs CMake that it should pass
-fcolor-diagnostics flag to the compiler.
(Colorized build system output, like from GNU Make, is produced even
without this flag.)
Note that if the build is done using Ninja and the build output is
redirected (not a TTY), Ninja will still strip the escape codes from
the output. For the case of idf.py, this is handled in the next
commit.
There are multiple changes in this commit:
1. Unify the RISC-V and ULP-FSM code paths in esp32ulp_mapgen.py.
It seems that these were originally introduced because `nm` output
for the RISC-V case contained symbol sizes, while for the ULP-FSM
no symbol sizes were reported. This makes sense, because the
ULP-FSM object files are produced from assembly source, symbol
sizes have to be added manually using the .size directive.
In the case of RISC-V, the object files are built from C sources
and the sizes are automatically added by the compiler.
Now 'posix' output format is used for both RISC-V and ULP-FSM.
2. Move BASE_ADDR out of esp32ulp_mapgen.py. This now has to be passed
from CMake, which should make it easier to modify if a new chip
with a different RTC RAM base address is added.
3. Add C++ guards to the generated header file.
4. Switch from optparse to argparse for similarity with other IDF
tools.
5. Add type annotations.
* "dummy loop to force pre-processed linker file generation" seems to
be unnecessary. It looks like the idea was copied from the
dependency of ULP-FSM preprocessed source files on the LD script.
* Can use add_dependencies instead of
set_target_properties(...LINK_DEPENDS...) which is more readable
* Use target_link_options instead of target_link_libraries, which is
supported starting from CMake 3.13. Unlike target_link_libraries,
it doesn't require manually quoting the pats.
This fixes the issue with build output not being colorized on Windows,
while the hints messages are colorized.
The issue occurred because sys.stdout and sys.stderr get overridden
by colorama.init() at runtime, but the default argument
output_stream=sys.stdout holds the reference to the"original"
sys.stdout.
colorama.init() (which, by the way, gets called via a curious chain
of imports, via idf_component_tools.manifest and tqdm package)
overrides standard streams, on Windows only. The overridden streams
contain logic to convert ANSI color codes into Windows Console API
calls to colorize the text.
Since read_and_write_stream function used the default value of
output_stream evaluated at module loading time, it was using the
original sys.stdout, not the one overridden by colorama.
One extra note is that while this does fix the coloring issue, the
solution is a bit fragile, as it relies on one of the following
(on Windows):
- colorama.init() is called (this can change if idf-component-manager
stops importing tqdm)
- Sufficiently new version of Windows 10 is used, and ANSI color codes
support is enabled in the Registry.
The actual output from the build tool (CMake/Ninja) may or may not
contain color escape codes, depending on various factors. The output
written to the log file should never include color escape codes,
though. This is because color escape codes in files are usually not
rendered as "color" in editors, and complicate reading. Also escape
codes would break the regular expressions used to display hints for
compilation errors.
If stdout is a TTY (meaning that the output is not redirected), tell
the build tool (GNU Make or Ninja) to enable colorized output.
GNU Make and Ninja also check if their stdout is redirected and
strip color escape sequences in that case. CLICOLOR_FORCE environment
variable overrides this behavior.
With this change, if the compiler was launched with the
-fcolor-diagnostics flag and idf.py output is not redirected, the
final output in the terminal will be colorized.
(-fcolor-diagnostics is handled at CMake level by the previous commit)
Related to https://github.com/espressif/esp-idf/issues/4162
Setting this option informs CMake that it should pass
-fcolor-diagnostics flag to the compiler.
(Colorized build system output, like from GNU Make, is produced even
without this flag.)
Note that if the build is done using Ninja and the build output is
redirected (not a TTY), Ninja will still strip the escape codes from
the output. For the case of idf.py, this is handled in the next
commit.
Ivan Grokhotkov