Mirrors/esp-idf
1
0
Fork 0
mirror of https://github.com/espressif/esp-idf.git synced 2026-09-22 13:01:16 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat(isp): Add example and script to check raw picture by ISP dma input

Browse Source
This commit is contained in:
gaoxu
2026-07-15 11:30:49 +08:00
parent e4f9f672b0
commit 0df7a1f2cf
9 changed files with 538 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/doxygen/Doxyfile_esp32p4
View File

@@ -18,6 +18,7 @@ INPUT += \
$(PROJECT_PATH)/components/esp_driver_isp/include/driver/isp_ccm.h \
$(PROJECT_PATH)/components/esp_driver_isp/include/driver/isp_color.h \
$(PROJECT_PATH)/components/esp_driver_isp/include/driver/isp_core.h \
$(PROJECT_PATH)/components/esp_driver_isp/include/driver/isp_dma.h \
$(PROJECT_PATH)/components/esp_driver_isp/include/driver/isp_crop.h \
$(PROJECT_PATH)/components/esp_driver_isp/include/driver/isp_demosaic.h \
$(PROJECT_PATH)/components/esp_driver_isp/include/driver/isp_gamma.h \

6
docs/en/api-reference/peripherals/isp.rst
View File

@@ -67,6 +67,7 @@ The ISP driver offers following services:
- :ref:`isp-resource-allocation` - covers how to allocate ISP resources with properly set of configurations. It also covers how to recycle the resources when they finished working.
- :ref:`isp-enable-disable` - covers how to enable and disable an ISP processor.
- :ref:`isp-dma-input` - covers how to feed image frames stored in memory into the ISP through DW-GDMA.
- :ref:`isp-af-statistics` - covers how to get AF statistics one-shot or continuously.
- :ref:`isp-awb-statistics` - covers how to get AWB white patches statistics one-shot or continuously.
- :ref:`isp-ae-statistics` - covers how to get AE statistics one-shot or continuously.
@@ -247,7 +248,7 @@ ISP DMA Input
Besides image streams from camera controllers, the ISP can also read image frames from system memory through DW-GDMA. To use DMA input, set :cpp:member:`esp_isp_processor_cfg_t::input_data_source` in :cpp:type:`esp_isp_processor_cfg_t` to :cpp:enumerator:`ISP_INPUT_DATA_SOURCE_DWGDMA`, and configure the input format, output format, and resolution according to the image frame.
DMA input is useful for feeding software-generated data, offline RAW images, or other test images in memory into the ISP. It can be used to validate an ISP pipeline without a camera sensor, reproduce issues with a specific input image, or generate inspectable output images in pytest. Call :cpp:func:`esp_isp_dma_process_frame` to send one input buffer to the ISP and write the processed image into an output buffer. The input and output buffers must be accessible by DMA; if cacheable memory is used, perform the required cache synchronization before and after the DMA transfer.
DMA input is useful for feeding software-generated data, offline RAW images, or other test images in memory into the ISP. It can be used to validate an ISP pipeline without a camera sensor, reproduce issues with a specific input image. Call :cpp:func:`esp_isp_dma_process_frame` to send one input buffer to the ISP and write the processed image into an output buffer. The input and output buffers must be accessible by DMA; if cacheable memory is used, perform the required cache synchronization before and after the DMA transfer.
ISP AF Controller
~~~~~~~~~~~~~~~~~
@@ -966,7 +967,7 @@ Application Examples
--------------------
* :example:`peripherals/isp/multi_pipelines` demonstrates how to use the ISP pipelines to process the image signals from camera sensors and display the video on LCD screen via DSI peripheral.
* :example:`peripherals/isp/dma_input` demonstrates how to feed software-generated RAW8 Bayer data into the ISP through DW-GDMA and save the processed RGB888 frames as PPM images in pytest.
* :example:`peripherals/isp/dma_input` demonstrates how to feed a RAW8 BGGR image in memory into the ISP through DW-GDMA. ``pytest_isp_dma_input.py`` saves the processed RGB888 frames as PPM images and compares them pixel by pixel with the checked-in golden image.
* `esp_video/examples <https://github.com/espressif/esp-video-components/tree/master/esp_video/examples>`_ provides some examples of enabling ISP control algorithms.
API Reference
@@ -987,5 +988,6 @@ API Reference
.. include-build-file:: inc/isp_color.inc
.. include-build-file:: inc/isp_crop.inc
.. include-build-file:: inc/isp_core.inc
.. include-build-file:: inc/isp_dma.inc
.. include-build-file:: inc/components/esp_driver_isp/include/driver/isp_types.inc
.. include-build-file:: inc/components/esp_hal_cam/include/hal/isp_types.inc

6
docs/zh_CN/api-reference/peripherals/isp.rst
View File

@@ -67,6 +67,7 @@ ISP 驱动程序提供以下服务：
- :ref:`isp-resource-allocation` - 涵盖如何通过正确的配置来分配 ISP 资源，以及完成工作后如何回收资源。
- :ref:`isp-enable-disable` - 涵盖如何启用和禁用 ISP 处理器。
- :ref:`isp-dma-input` - 涵盖如何通过 DW-GDMA 将存储在内存中的图像帧送入 ISP。
- :ref:`isp-af-statistics` - 涵盖如何单次或连续获取 AF 统计信息。
- :ref:`isp-awb-statistics` - 涵盖如何单次或连续获取 AWB 白块统计信息。
- :ref:`isp-ae-statistics` - 涵盖如何单次或连续获取 AE 统计信息。
@@ -247,7 +248,7 @@ ISP DMA 输入
除来自摄像头控制器的数据流外，ISP 还可以通过 DW-GDMA 从系统存储中读取图像帧作为输入。使用 DMA 输入时，应在 :cpp:type:`esp_isp_processor_cfg_t` 中将 :cpp:member:`esp_isp_processor_cfg_t::input_data_source` 配置为 :cpp:enumerator:`ISP_INPUT_DATA_SOURCE_DWGDMA`，并根据输入图像格式设置输入、输出格式及分辨率。
DMA 输入适用于将软件生成的数据、离线保存的 RAW 图像或其他内存中的测试图像送入 ISP 进行处理。它可用于无摄像头传感器参与时验证 ISP 流水线、复现特定输入图像的问题，或在 pytest 中生成可检查的输出图像。调用 :cpp:func:`esp_isp_dma_process_frame` 可以将一帧输入缓冲区送入 ISP，并将处理后的图像写入输出缓冲区。输入和输出缓冲区需要满足 DMA 访问要求；若使用带 cache 的内存，请在 DMA 传输前后执行必要的 cache 同步。
DMA 输入适用于将软件生成的数据、离线保存的 RAW 图像或其他内存中的测试图像送入 ISP 进行处理。它可用于无摄像头传感器参与时验证 ISP 流水线、复现特定输入图像的问题。调用 :cpp:func:`esp_isp_dma_process_frame` 可以将一帧输入缓冲区送入 ISP，并将处理后的图像写入输出缓冲区。输入和输出缓冲区需要满足 DMA 访问要求；若使用带 cache 的内存，请在 DMA 传输前后执行必要的 cache 同步。
ISP AF 控制器
~~~~~~~~~~~~~
@@ -965,7 +966,7 @@ Kconfig 选项 :ref:`CONFIG_ISP_CTRL_FUNC_IN_IRAM` 支持：
--------
* :example:`peripherals/isp/multi_pipelines` 演示了如何使用 ISP 流水线处理来自摄像头传感器的图像信号，并通过 DSI 外设在 LCD 屏幕上显示视频。
* :example:`peripherals/isp/dma_input` 演示了如何通过 DW-GDMA 将软件生成的 RAW8 Bayer 数据送入 ISP，并在 pytest 中将处理后的 RGB888 帧保存为 PPM 图片。
* :example:`peripherals/isp/dma_input` 演示了如何通过 DW-GDMA 将内存中的 RAW8 BGGR 图像送入 ISP。``pytest_isp_dma_input.py`` 会将处理后的 RGB888 帧保存为 PPM 图片，并与示例中提交的 golden 图片进行逐像素比较。
* `esp_video/examples <https://github.com/espressif/esp-video-components/tree/master/esp_video/examples>`_ 中包含自动启用 ISP 控制算法的一些示例。
API 参考
@@ -986,5 +987,6 @@ API 参考
.. include-build-file:: inc/isp_color.inc
.. include-build-file:: inc/isp_crop.inc
.. include-build-file:: inc/isp_core.inc
.. include-build-file:: inc/isp_dma.inc
.. include-build-file:: inc/components/esp_driver_isp/include/driver/isp_types.inc
.. include-build-file:: inc/components/esp_hal_cam/include/hal/isp_types.inc

18
examples/peripherals/isp/dma_input/README.md
View File

@@ -5,22 +5,20 @@
## Overview
This example generates a standard RAW8 Bayer color-bar pattern in software, writes it into a DMA-capable PSRAM input buffer, feeds it into the ISP through DW-GDMA, applies the ISP color adjustment module, and prints the RGB888 output as base64. The pytest script decodes the output into PPM images for inspection.
This example embeds a 240 x 280 RAW8 Bayer image of a real scene in flash, copies it into a DMA-capable PSRAM input buffer, feeds it into the ISP through DW-GDMA, and prints the RGB888 output as base64. The pytest script decodes the output into a PPM image and compares it with the checked-in golden image.
The data flow is:
1. A synthetic RAW8 Bayer **color-bar** pattern is generated in software into the ISP DMA input buffer.
2. The pattern is transferred into the ISP via `DW-GDMA → ISP DMA input`.
1. The embedded BGGR RAW8 image is copied from flash into the ISP DMA input buffer.
2. The image is transferred into the ISP via `DW-GDMA → ISP DMA input`.
3. The ISP processes the data (demosaic, color adjustment) and outputs RGB888 (BGR24 byte layout).
4. The RGB888 frame is base64-encoded and printed with machine-parseable markers.
5. pytest decodes the payload, swaps BGR→RGB, and saves one PPM file per frame.
5. pytest decodes the payload, swaps BGR→RGB, saves one PPM file per frame, and compares it with the golden image.
## Hardware Required
- An ESP32-P4 devkit with PSRAM (this example allocates the ISP DMA input/output buffers from PSRAM).
If you replace the software-generated pattern with a RAW image embedded in flash, copy it into a DMA-capable buffer before feeding it to the ISP DMA input path.
## How to Use
Run the test locally. It builds and flashes the example, captures the serial output, and saves one PPM artifact per frame:
@@ -30,17 +28,15 @@ cd examples/peripherals/isp/dma_input
pytest pytest_isp_dma_input.py --target esp32p4 --port PORT
```
The pytest log directory (`dut.logdir`) contains `reference.ppm` for the standard reference color-bar image and `isp_dma_input_frame00.ppm`, ... for the decoded ISP output frames. The pytest script checks that the decoded frame contains the expected standard color-bar structure and verifies that the ISP color brightness adjustment brightens the black bar.
ISP feature-specific setup is kept under `main/isp_features/<feature>/`, and pytest feature checks are kept under `pytest_features/<feature>.py`. The current color brightness check lives in `main/isp_features/color/` and `pytest_features/color_brightness.py`; new ISP feature checks can follow the same pattern while reusing the common DW-GDMA input/output and image parsing flow.
The pytest log directory (`dut.logdir`) contains `isp_dma_input_frame00.ppm`, the decoded RGB888 ISP output from the current hardware run. The repository includes [golden/golden.ppm](golden/golden.ppm), the checked-in reference image. The test compares the decoded RGB888 pixels with this reference image, making image-quality regressions visible in review and detectable in CI.
## Example Output
Each frame uses the same standard vertical color-bar input pattern with positive color brightness enabled to validate the ISP color adjustment module.
The example processes one embedded 240 x 280 BGGR RAW8 frame.
```text
Feeding 1 frames through ISP DMA input...
IMAGE_META frame=0 width=128 height=96 format=BGR24 encoding=base64 color_brightness=64
IMAGE_META frame=0 width=240 height=280 format=BGR24 encoding=base64
IMAGE_BASE64_BEGIN
IMAGE_BASE64 ...
IMAGE_BASE64_END

BIN
examples/peripherals/isp/dma_input/golden/golden.ppm Normal file
View File

Binary file not shown.

4
examples/peripherals/isp/dma_input/main/CMakeLists.txt
View File

@@ -1,3 +1,7 @@
idf_component_register(SRCS "isp_dma_main.c"
PRIV_REQUIRES esp_driver_isp esp_mm esp_psram mbedtls
INCLUDE_DIRS ".")
target_add_binary_data(${COMPONENT_LIB}
"${CMAKE_CURRENT_LIST_DIR}/assets/sensor_240x280_bggr.raw"
BINARY RENAME_TO "sensor_raw")

470
examples/peripherals/isp/dma_input/main/assets/sensor_240x280_bggr.raw Normal file
View File

File diff suppressed because one or more lines are too long

49
examples/peripherals/isp/dma_input/main/isp_dma_main.c
View File

@@ -7,7 +7,7 @@
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <string.h>
#include <assert.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
@@ -19,42 +19,15 @@
#include "driver/isp_core.h"
#include "driver/isp_color.h"
#define EXAMPLE_WIDTH 128
#define EXAMPLE_HEIGHT 96
#define EXAMPLE_WIDTH 240
#define EXAMPLE_HEIGHT 280
#define EXAMPLE_BASE64_CHUNK_LEN 384
#define EXAMPLE_BASE64_DELAY_MS 10
#define EXAMPLE_DMA_ALIGN 64
#define EXAMPLE_FRAME_COUNT 2
#define EXAMPLE_FRAME_COUNT 1
static const uint8_t s_color_bars[8][3] = {
{255, 255, 255}, // white
{255, 255, 0}, // yellow
{ 0, 255, 255}, // cyan
{ 0, 255, 0}, // green
{255, 0, 255}, // magenta
{255, 0, 0}, // red
{ 0, 0, 255}, // blue
{ 0, 0, 0}, // black
};
static void s_generate_raw8_color_bars(uint8_t *raw, uint32_t w, uint32_t h)
{
for (uint32_t y = 0; y < h; y++) {
bool even_row = ((y & 1) == 0);
for (uint32_t x = 0; x < w; x++) {
uint32_t bar = (x * 8) / w;
uint8_t r = s_color_bars[bar][0];
uint8_t g = s_color_bars[bar][1];
uint8_t b = s_color_bars[bar][2];
bool even_col = ((x & 1) == 0);
if (even_row) {
raw[y * w + x] = even_col ? b : g;
} else {
raw[y * w + x] = even_col ? g : r;
}
}
}
}
extern const uint8_t sensor_raw_start[] asm("_binary_sensor_raw_start");
extern const uint8_t sensor_raw_end[] asm("_binary_sensor_raw_end");
static void *s_alloc_dma_buffer(size_t size)
{
@@ -86,7 +59,7 @@ static void s_print_base64_payload(const unsigned char *encoded, size_t encoded_
}
printf("IMAGE_BASE64 %.*s\n", (int)chunk_len, (const char *)&encoded[offset]);
fflush(stdout);
vTaskDelay(pdMS_TO_TICKS(1));
vTaskDelay(pdMS_TO_TICKS(EXAMPLE_BASE64_DELAY_MS));
}
printf("IMAGE_BASE64_END\n");
fflush(stdout);
@@ -120,6 +93,10 @@ void app_main(void)
uint8_t *isp_out_buf = s_alloc_dma_buffer(out_size);
assert(isp_in_buf && isp_out_buf);
size_t embedded_raw_size = sensor_raw_end - sensor_raw_start;
assert(embedded_raw_size == in_size);
memcpy(isp_in_buf, sensor_raw_start, embedded_raw_size);
size_t encoded_len = 0;
int ret = mbedtls_base64_encode(NULL, 0, &encoded_len, isp_out_buf, out_size);
ESP_ERROR_CHECK((ret == MBEDTLS_ERR_BASE64_BUFFER_TOO_SMALL) ? ESP_OK : ESP_FAIL);
@@ -128,8 +105,6 @@ void app_main(void)
printf("Feeding %d frames through ISP DMA input...\n", EXAMPLE_FRAME_COUNT);
for (int frame = 0; frame < EXAMPLE_FRAME_COUNT; frame++) {
s_generate_raw8_color_bars(isp_in_buf, h_res, v_res);
ESP_ERROR_CHECK(esp_cache_msync(isp_in_buf, in_size, ESP_CACHE_MSYNC_FLAG_DIR_C2M));
ESP_ERROR_CHECK(esp_isp_dma_process_frame(isp_proc, isp_out_buf, isp_in_buf, 1000));
ESP_ERROR_CHECK(esp_cache_msync(isp_out_buf, out_size, ESP_CACHE_MSYNC_FLAG_DIR_M2C));

66
examples/peripherals/isp/dma_input/pytest_isp_dma_input.py
View File

@@ -22,19 +22,9 @@ IMAGE_CHUNK_PATTERN = (
r'(?=\r?\n|IMAGE_BASE64|IMAGE_BASE64_END|Frame )'
)
IMAGE_OUTPUT_TEMPLATE = 'isp_dma_input_frame{frame:02d}.ppm'
REFERENCE_IMAGE_NAME = 'reference.ppm'
REFERENCE_IMAGE_PATH = Path(__file__).parent / 'golden' / 'golden.ppm'
EXPECTED_PIXEL_FORMAT = 'BGR24'
EXPECTED_ENCODING = 'base64'
STANDARD_COLOR_BARS_RGB888 = (
(255, 255, 255), # white
(255, 255, 0), # yellow
(0, 255, 255), # cyan
(0, 255, 0), # green
(255, 0, 255), # magenta
(255, 0, 0), # red
(0, 0, 255), # blue
(0, 0, 0), # black
)
RGB888_BYTES_PER_PIXEL = 3
PPM_MAGIC = b'P6'
PPM_MAX_VALUE = b'255'
@@ -112,17 +102,6 @@ def decode_bgr24_base64_image(metadata: ImageMetadata, payload_lines: list[str])
return RgbImage(width=metadata.width, height=metadata.height, pixels_rgb888=_bgr24_to_rgb888(raw_bytes))
def generate_standard_color_bar_image(width: int, height: int) -> RgbImage:
pixels = bytearray(width * height * RGB888_BYTES_PER_PIXEL)
for y in range(height):
for x in range(width):
bar_index = (x * len(STANDARD_COLOR_BARS_RGB888)) // width
offset = (y * width + x) * RGB888_BYTES_PER_PIXEL
pixels[offset : offset + RGB888_BYTES_PER_PIXEL] = STANDARD_COLOR_BARS_RGB888[bar_index]
return RgbImage(width=width, height=height, pixels_rgb888=bytes(pixels))
def save_ppm_artifact(image: RgbImage, output_path: Path) -> None:
output_path.parent.mkdir(parents=True, exist_ok=True)
try:
@@ -134,14 +113,45 @@ def save_ppm_artifact(image: RgbImage, output_path: Path) -> None:
logging.info('Saved ISP DMA artifact to %s', output_path)
def assert_image_is_meaningful(image: RgbImage) -> None:
distinct_pixels = {image.pixels_rgb888[i : i + 3] for i in range(0, len(image.pixels_rgb888), 3)}
assert len(distinct_pixels) > 1, 'ISP output is a single flat color; the pipeline likely produced no real data'
def load_ppm_image(image_path: Path) -> RgbImage:
ppm_data = image_path.read_bytes()
try:
magic, dimensions, max_value, pixels_rgb888 = ppm_data.split(b'\n', 3)
width, height = (int(value) for value in dimensions.split())
except ValueError as error:
raise ValueError(f'Invalid PPM image: {image_path}') from error
if magic != PPM_MAGIC or max_value != PPM_MAX_VALUE:
raise ValueError(f'Unsupported PPM image: {image_path}')
return RgbImage(width=width, height=height, pixels_rgb888=pixels_rgb888)
def assert_image_matches_reference(result_image: RgbImage, reference_image: RgbImage) -> None:
assert (result_image.width, result_image.height) == (reference_image.width, reference_image.height), (
f'ISP output dimensions do not match reference image: '
f'{result_image.width}x{result_image.height} != {reference_image.width}x{reference_image.height}'
)
if result_image.pixels_rgb888 == reference_image.pixels_rgb888:
return
mismatch_offset = next(
offset
for offset, (actual, expected) in enumerate(zip(result_image.pixels_rgb888, reference_image.pixels_rgb888))
if actual != expected
)
pixel_index, channel = divmod(mismatch_offset, RGB888_BYTES_PER_PIXEL)
raise AssertionError(
f'ISP output does not match reference image at pixel {pixel_index}, channel {channel}: '
f'{result_image.pixels_rgb888[mismatch_offset]} != {reference_image.pixels_rgb888[mismatch_offset]}'
)
@pytest.mark.generic
@idf_parametrize('target', soc_filtered_targets('SOC_ISP_SUPPORTED == 1'), indirect=['target'])
def test_isp_dma_input_example(dut: Dut) -> None:
reference_image = load_ppm_image(REFERENCE_IMAGE_PATH)
frame_count_match = dut.expect(r'Feeding (?P<frame_count>\d+) frames through ISP DMA input...')
expected_frame_count = int(frame_count_match.group('frame_count').decode('utf-8'))
logging.info('Expecting %d ISP DMA frame(s)', expected_frame_count)
@@ -156,10 +166,6 @@ def test_isp_dma_input_example(dut: Dut) -> None:
metadata.height,
metadata.pixel_format,
)
if expected_frame == 0:
reference_image = generate_standard_color_bar_image(metadata.width, metadata.height)
save_ppm_artifact(reference_image, Path(dut.logdir) / REFERENCE_IMAGE_NAME)
dut.expect_exact('IMAGE_BASE64_BEGIN')
logging.info('Receiving base64 image payload for frame %d', metadata.frame)
payload_lines = collect_base64_payload(dut)
@@ -169,7 +175,7 @@ def test_isp_dma_input_example(dut: Dut) -> None:
logging.info('Decoded frame %d to RGB888 image', metadata.frame)
output_path = Path(dut.logdir) / IMAGE_OUTPUT_TEMPLATE.format(frame=metadata.frame)
save_ppm_artifact(result_image, output_path)
assert_image_is_meaningful(result_image)
assert_image_matches_reference(result_image, reference_image)
dut.expect_exact(f'Frame {expected_frame} done')
dut.expect_exact('ISP DMA visual demo done.')