Extend the test peripheral with receipt timestamps and an auto-recycle
hook so runtime tests can observe handoff latency and sustain arrivals from
dispatch completion.
Add a standalone Unity app covering fixed first-submission deadlines,
callback-entry batch drain, shared ESP timer task fairness, periodic
timestamp light-sleep behavior, synchronous flush, dispatch latency,
deinit races, bounded inflight peaks, and timeout conversion at both
supported tick rates. Verify FIFO delivery and reject delayed-consumer and
extra-marker false positives.
Render BLE_LOG_RT_PERF lines in the performance log parser and
cross-reference the runtime and performance apps from their READMEs.
Replace the dedicated BLE Log FreeRTOS task with a deferred esp_timer
batch dispatch. The first submission anchors a one-shot 1 ms deadline;
the callback drains the queue depth captured at entry and schedules the
next fixed defer only for arrivals left behind. Later submissions cannot
move the current deadline, and no fixed batch-size cap limits throughput.
Protect queue and timer lifetime with runtime references. Drain queued
submissions before ble_log_lbm_flush_all_trans waits for transport
ownership to return, so synchronous flush does not depend on the deferred
alarm firing.
Keep the defer alarm as a light-sleep wake source and select
ESP_TIMER_IN_IRAM. Timestamp synchronization uses the shared ESP timer task
with skipped unhandled events, so it neither wakes light sleep nor replays
missed periods. Retain legacy task and trigger symbols as hidden no-op
sdkconfig compatibility options.
New ble_log_test app captures the runtime hook output through the test
peripheral and validates the version info frame: source code, BLE Log
version, a hex-valid idf commit that must be non-zero, per-lib commit
fields non-zero exactly when the matching lib is linked, and chip
model/revision matching esp_chip_info(). The README carries the
generated empty Supported Targets table to match the build-test
manifest.
The test drives transports via sustained writes instead of
ble_log_flush(): the flush window disables the module, so hook frames
written during a flush would be dropped.
Also move .build-test-rules.yml from ble_log_perf_test/ to the
test_apps/ root (one manifest entry per app, as elsewhere in ESP-IDF).
Both apps stay disabled until BLE Log test runners are available.
Verified: full esp32c6 build of the app; the build-test checker's own
parsing logic confirms the README table matches the manifest.
Replace the 2-byte BLE Log info record with a 58-byte version info
frame (BLE_LOG_VERSION 5 -> 6; the abandoned branch that claimed the
version-6 slot frees it, so the overall bump stays 5 -> 6):
- idf build commit (12 bytes), injected at build time by
register_ble_log_idf_commit() next to the other register_* helpers;
the git probe is only trusted when the IDF tree itself is a
repo/worktree, since rev-parse walks up parent directories
- controller, btdm_common, BLE Mesh and BLE Audio lib commits (10
bytes each, zero-padded), every getter guarded by the exact
condition that links its lib, so configs without the lib leave the
field zero (no link errors)
- chip model and revision from esp_chip_info() at runtime
Lib strings are copied NUL-safely instead of assuming a fixed hash
length; the mesh commit is the substring after the last space of
bt_mesh_v11_commit_str. Frame layout is pinned by a static assert.
Verified on target: esp32, esp32c3, esp32c5 and esp32h4 boards (the
h4 run covers controller + btdm_common + mesh in one build); the
audio-enabled build is blocked by pre-existing esp_ble_audio compile
errors on this base (audio symbol verified with nm instead).
Introduce a seq_cst closing gate shared by the runtime and the LBM:
submitters increment the reference count before checking the inited
flag, and deinit closes the gate and waits for the count to drain
before deleting tasks, timers, queues, or buffers. A producer either
observes shutdown or its reference is visible to the wait, which makes
ble_log_deinit safe while write APIs are still active.
Submitters no longer block on the queue while holding a reference: a
timeout-0 send that cannot queue recycles the transport so its data
survives for the next flush.
Extract ble_log_ref_count_try_acquire/wait into the utility layer and
gate every LBM writer through ble_log_lbm_ref_acquire. ble_log_deinit
now closes the LBM gate first (ble_log_lbm_close) instead of clearing
the enable flag.
Move transport ownership from the runtime to the LBM: submit and
recycle now hand the peripheral-owned flag explicitly instead of the
runtime reaching back into LBM buffers. The submit path never blocks
producers - a transport that cannot be queued is recycled immediately
so its data survives for the next flush.
Cross-context ownership accesses go through explicit atomic helpers:
release-store on recycle pairs with acquire-loads in the flush paths,
and the inflight high-water mark stays a relaxed CAS-max (a plain
volatile update races the runtime hook's statistics reads).
Add a Unity-based perf test app under ble_log/test_apps with two
separate measurement purposes:
- throughput: multi-task (write_hex x3) + LL task/HCI + ISR + compressed
writers under 2 Mbps / 20 Mbps link caps, measuring system-wide
throughput only
- cycle: single-writer write_hex / write_hex_ll / compressed writes at
link=0, measuring per-frame cycles; the compressed path is split into
the encode phase and the write_hex phase via test-only counters in
ble_log_compression.c
Add the test-only transport (CONFIG_BLE_LOG_PRPH_TEST) that models DMA
ownership transfer and link-rate backpressure.
Move the SPI2 bus, transfer-size and DMA-alignment constants shared by the
full-duplex DMA and half-duplex HD transports into
ble_log_prph_spi_common.h. Pin numbers stay Kconfig-driven but fall back to
the target's native SPI2 IOMUX pins when configured negative, and default to
-1 so both transports use the native pins out of the box; any GPIO number
still overrides.
Add BLE_LOG_PRPH_SPI_MASTER_HD, a half-duplex SPI2 transport built on the
HD DMA protocol (slave HD ver2). Reuses the SPI master pin config shared
with the full-duplex DMA transport; the HD device queue is sized 2x for
ping-pong so the next transaction is queued while the previous DMA runs.
Extract ble_log_lbm_recycle_trans() so every transport (UART DMA, SPI DMA,
dummy) recycles owned trans through one path; the HD transport builds on it.
Also prepare buffers for GDMA weighted arbitration: align transport buffers
to the 32-byte burst size, and disable the UART data burst (UHCI enforces
burst-size alignment on uhci_transmit() once weighted arbitration is enabled,
and UART bandwidth is baud-rate limited anyway).
- Add type assertion for event.type to bypass strict TS compilation errors
- Accept both 'permission.asked' and 'permission.updated' event types
- Add normalizePermissionEvent() to handle internal vs SDK event property
format differences (pattern vs patterns, type vs permission fields)
- Add RawPermissionEvent type for loose event property parsing
- Add troubleshooting section and subdirectory auto-loading note to README
Move the OpenCode companion guide into the ble_uart_service example.
Add English and Chinese Markdown guides with image assets.
Keep ESP-BLE-UART naming consistent across the example and bridge tooling.
On ESP32C2 with BT_CTRL_RUN_IN_FLASH_ONLY, the bt_default linker scheme
routes .iram1 sections to flash_text, causing BLE_LOG_IRAM_ATTR functions
to end up in flash instead of IRAM. Define a dedicated .ble_log_iram1
section for ESP32C2 and route it to iram0_bt_text unconditionally.
- Use __atomic_fetch_add for stat_mgr counters to prevent lost updates
under concurrent ISR/task access (H1)
- Use __atomic_load_n with ACQUIRE ordering for ref_count spin-loops (L1)
- Remove unnecessary BLE_LOG_IRAM_ATTR from ble_log_rt_task since it
calls flash-resident functions and cannot run during flash ops (L3)
- Add parentheses to BLE_LOG_TRANS_FREE_SPACE and BLE_LOG_MAKE_FRAME_META
macro parameters to prevent operator precedence bugs (M6)
- Reorder deinit: stop RT task before destroying peripheral driver to
prevent sending transports to a NULL dev_handle on dual-core (H2)
- Drain remaining queue items in rt_deinit to clean up transport state
- Add atomic flush_in_progress guard to prevent two concurrent
ble_log_flush() callers from deadlocking on ref_count spin-wait (H5)
- Handle scheduler-suspended and ISR context in UART redirect path
to prevent xSemaphoreTake crash during light sleep (C1)
- Check uart_driver_install return value before setting inited flag (M1)
- Always clean up SPI device handle in deinit even if acquire_bus fails (M4)
During light sleep transitions, the FreeRTOS scheduler is suspended but
xPortInIsrContext() returns false, causing xQueueSend with portMAX_DELAY
to hit a configASSERT. Add a check for taskSCHEDULER_SUSPENDED and use
a non-blocking send with rollback on queue-full to avoid resource leaks.
idf-ci discovers pyproject.toml pytest config and runs a separate
collection pass with --target all, which finds 0 target tests and
raises RuntimeError. Add to .idf_ci.toml exclude_dirs instead of
deleting tests. Restore all 227 unit tests with lint/mypy fixes.
Reflect changes from dev/ble-log-202603: ping-pong to multi-buffer
transport (4 buffers/LBM), renamed Kconfig options (LBM_TRANS_SIZE →
LBM_TRANS_BUF_SIZE with new defaults), always-enabled checksum and
enhanced statistics, UART redirection support, UHCI Out removal,
SPI Out deprecation, and updated memory estimation.
Add Kconfig options for bandwidth-optimized logging via the BLE Log
Async Output system. When enabled, controller log levels default to 2
and host/mesh log encoding is auto-selected.
- BT_LOG_CRITICAL_ONLY parent: selects BLE_LOG_ENABLED + compression
- BT_LOG_CRITICAL_ONLY_CTRL: supports both NimBLE and non-NimBLE (C3)
controllers with default level override
- BT_LOG_CRITICAL_ONLY_HOST: host-agnostic, selects compressed log
for Bluedroid conditionally
- BT_LOG_CRITICAL_ONLY_MESH: placeholder for mesh log encoding
- Guards against BT_STACK_NO_LOG conflict
Move per-chip BLE log Kconfig options (esp32c2/c5/c6/h2) into
components/bt/common/Kconfig.in for single-source-of-truth configuration.
Restructure menu as "BT Logs" with "Log Sources" sub-menu containing
controller log config options. Controller log output mode choice defaults
to BLE Log v2 when BLE_LOG_ENABLED, with legacy mode as deprecated
fallback. Migrate wrap_panic_handler and task_wdt_user_handler configs
as well since they depend on BT_LE_CONTROLLER_LOG_ENABLED.
Replace the two-step acquire() + get_trans() flow with a unified
acquire_trans() that iterates all candidate pools. When a pool's
buffers are exhausted, the lock is released and the next pool is
tried, enabling load balancing across pools instead of only
resolving lock contention.
Add BufUtilTracker to record per-LBM inflight peak data, wire it
through StatsAccumulator and StatsUpdated message for thread-safe
delivery to the UI. Buffer utilization is displayed in a dedicated
BufUtilScreen accessible via the 'm' keybinding, separate from
the frame stats screen ('d').
Also reject false INIT_DONE frames with version==0 caused by
misaligned data during parser sync, preventing spurious stats resets.
Add BUF_UTIL=5 to InternalSource, BufUtilPool enum, BufUtilResult
TypedDict, BufUtilEntry dataclass, and name resolution helpers.
Add decode branch in internal_decoder with pool/index extraction
from the packed lbm_id field. Tests cover valid decode, pool/index
extraction, and truncated payload handling.
Replace __atomic_exchange_n with __atomic_load_n in
ble_log_emit_buf_util() so inflight_peak reports the all-time peak
since init rather than resetting to zero after each report.
Migrate all LBM, RT, and peripheral backend code from 2-buffer
ping-pong to 4-buffer transport with bitmask index arithmetic.
LBM changes:
- Init sets owner on each transport buffer
- get_trans/flush/dump use (idx+1) & (cnt-1) instead of !idx
- get_trans/flush use __atomic_load_n(ACQUIRE) for prph_owned
- New ble_log_write_buf_util() emits BUF_UTIL telemetry frames,
exchange peak with 0 to avoid stale baseline after bursts
RT changes:
- Queue depth uses unified BLE_LOG_TRANS_TOTAL_CNT
- rt_queue_trans uses __atomic_store_n for prph_owned, tracks
per-LBM inflight count and peak via lock-free CAS
- ISR path captures pxHigherPriorityTaskWoken and yields
Peripheral changes:
- All tx_done callbacks: decrement inflight before releasing
prph_owned with __atomic_store_n(RELEASE)
- SPI/UART send_trans error paths: decrement inflight and
release prph_owned to allow retry on next get_trans pass
- Dummy send_trans: recycle buffer immediately (pos=0, decrement
inflight, release prph_owned) since no DMA hardware to wait for
- Redir LBM upgraded to 4-buffer with owner and reset support
- All peripherals implement ble_log_prph_reset_util_counters()
- Click-based CLI entry point (console.py) with --port, --baudrate,
--log-dir options and interactive Launch Screen when --port omitted
- Deprecation warning for --output CLI flag
- 'ls' subcommand to list saved capture files
- PyInstaller build scripts (build.sh, build.bat, build_exe.py)
- Convenience run scripts (run.sh, run.bat)
- README rewrite with installation, usage, architecture docs, and
troubleshooting guide
- User guides in English and Chinese
- Remove old single-file ble_log_console.py
- Register new scripts in executable-list.txt
TUI frontend built on Textual with thread-safe backend integration:
- Launch screen with port/baud/log-dir selection and auto-refresh
- Log view with scrollable RichLog and color-coded write methods;
Rich markup escaped to prevent crash on bracket characters
- Stats screen split into two tables by time base:
- Firmware Counters (since chip init): Written and Buffer Loss
- Console Measurements (since console start): Average Throughput
and Peak Write — each with separate frames and bytes sub-columns
- Column widths constrained with min_width/max_width for stable layout
- Division-by-zero guard when peak window is zero
- Shortcut screen overlay
- Status panel with sync state, checksum mode, and transport metrics;
speed display uses named UART_BITS_PER_BYTE constant
- Main app wiring: UART reader thread with threading.Lock for serial
access, frame parser pipeline, stats emission via StatsUpdated
messages (including funnel snapshots via public accessor for
thread-safe delivery), BackendStopped message for disconnect
- BackendStopped posted on open_serial failure (prevents stuck status)
- Select.BLANK guard for baud rate in launch screen
- Explicit None guards replace asserts in backend worker
- Wall-clock burst tracking for REDIR frames (no chip-side timestamp)
- SN gap alerts (FrameLossDetected) and traffic spike notifications
Modular backend for the BLE log console rewrite:
- Frame parser with sync state machine and checksum auto-detection
(4 modes: XOR/Sum x Full/Header-only); handles incomplete frames
during re-sync search when previously synced
- Internal frame decoder (INIT_DONE, ENH_STAT, FLUSH, INFO)
- Data models: SourceCode, FrameByteCount, FunnelSnapshot, LossType
- Stats package with composition-root StatsAccumulator orchestrating:
- TransportMetrics (RX bytes, lifetime-average throughput)
- FirmwareLossTracker / FirmwareWrittenTracker (ENH_STAT deltas
with first-report absolute value initialization)
- SNGapTracker (sliding window reorder-tolerant SN gap detection)
- PeakBurstTracker (per-source sliding window burst density)
- TrafficSpikeDetector (wire utilization spike detection)
- Wall-clock burst tracker for non-timestamped sources (REDIR)
- Torn-read guard on ENH_STAT reports (baudrate-based plausibility)
with prev-state update on discard to prevent cascading drops
- Console-local metrics (TransportMetrics, PeakBurstTracker) preserved
across INIT_DONE resets; only ENH_STAT-coupled components reset
- UART transport with port validation and exclusive serial access
- Comprehensive test suite (17 test files, 223 tests)
Add BLE_LOG_GET_FRAME_SN() on the buffer-full loss path so the SN
counter advances even when frames are dropped. This allows the console
to detect firmware-side buffer loss via SN gaps. Bump BLE_LOG_VERSION
to reflect the semantic change.
Introduce stream write API with deferred frame encapsulation for UART
redirection on port 0. Redirected data is now properly framed (header +
payload + checksum) instead of being sent as raw ASCII, preventing frame
parser sync oscillation on the receiver.
- Add stream_seal/stream_write/stream_flush in ble_log_lbm.c
- Add BLE_LOG_SRC_REDIR source and BLE_LOG_UART_REDIR_ENABLED gate
- Simplify redir_uart_tx_chars and timer callback to use stream API
- Flush pending stream data in ble_log_prph_deinit
- Make get_trans static (no external callers after refactor)
- Move UART wrap linker flags outside CONFIG_BT_ENABLED guard
- Default UART DMA peripheral when SOC_UHCI_SUPPORTED
- Default baud rate 921600 -> 3000000
Replace embedded ble_log_enh_stat_t (packed wire struct) inside
ble_log_stat_mgr_t with flat uint32_t counters. Natural 4-byte
alignment ensures each load/store compiles to a single l32i/s32i
on Xtensa/RISC-V, making individual field access atomic without
locks.
Build the packed wire format on the stack inside a critical section
in ble_log_write_enh_stat() so the full snapshot is consistent.
Remove CONFIG_BLE_LOG_PAYLOAD_CHECKSUM_ENABLED conditional compilation
-- payload checksum is always computed over the full frame.
Remove CONFIG_BLE_LOG_XOR_CHECKSUM_ENABLED conditional -- XOR checksum
is always used; delete the sum checksum dead code path.
Remove CONFIG_BLE_LOG_ENH_STAT_ENABLED conditional -- enhanced
statistics (frame/byte counters per source) are always active.
Remove incorrect select on choice symbol
BT_LE_CONTROLLER_LOG_MODE_BLE_LOG_V2 from BLE_LOG_IS_ESP_CONTROLLER.
Move ble_log_spi_out.c and ble_log_spi_out.h into a deprecated/
subdirectory under components/bt/common/ble_log/. Extract all
BT_BLE_LOG_SPI_OUT_* Kconfig options into deprecated/Kconfig.in
with an if-block to reduce depends-on repetition, sourced from
ble_log/Kconfig.in inside a "Legacy SPI Log Output (Deprecated)"
menu. Add mutual exclusion with BLE_LOG_ENABLED so that the legacy
SPI Out and the new BLE Log Module cannot be enabled simultaneously.
Update CMakeLists.txt source path and add deprecated/include to
include dirs so existing callers are unaffected. The BLE Log
Module's SPI Master DMA peripheral transport is the replacement.
Delete ble_log_uhci_out.c and its header. Remove the UHCI Out source
from CMakeLists.txt and all BT_BLE_LOG_UHCI_OUT_* Kconfig options.
Remove per-chip BT_LE_CONTROLLER_LOG_UHCI_OUT_ENABLED Kconfig and
all UHCI Out caller references in esp32c5/c6/h2 controller bt.c.
Update ble_log_console README to reference the new BLE Log Module
UART DMA config. The functionality is superseded by the BLE Log
Module's UART DMA peripheral transport.
Add ble_log_write_hci macro that encodes upstream/downstream direction
in MSB of HCI type byte. Update all 4 callsites (NimBLE + Bluedroid)
to use the new macro. Parser reads MSB to determine direction;
backward compatible (old firmware MSB=0 defaults to "sent").
Zhou Xiao