Mirrors/esp-idf
1
0
Fork 0
mirror of https://github.com/espressif/esp-idf.git synced 2026-09-22 13:01:16 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'feat/isp_dpc_support_v6.0' into 'release/v6.0'

Browse Source 
feat(isp): support isp dpc dead pixel correction (v6.0)

See merge request espressif/esp-idf!51470
This commit is contained in:
morris
2026-09-08 15:06:44 +08:00
parent 7dc4ca8c87 7460c05a13
commit 7f7f6cccbf
24 changed files with 2234 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

184
docs/en/api-reference/peripherals/isp.rst
View File

@@ -47,7 +47,7 @@ ISP Pipeline
isp_chs [label = "Contrast &\n Hue & Saturation", width = 150, height = 70];
isp_yuv [label = "YUV Limit\n YUB2RGB", width = 120, height = 70];
isp_header -> BLC -> BF -> LSC -> Demosaic -> WBG -> CCM -> Gamma -> RGB2YUV -> SHARP -> isp_chs -> isp_yuv -> CROP -> isp_tail;
isp_header -> BLC -> DPC -> BF -> LSC -> Demosaic -> WBG -> CCM -> Gamma -> RGB2YUV -> SHARP -> isp_chs -> isp_yuv -> CROP -> isp_tail;
LSC -> HIST
Demosaic -> WBG
@@ -74,6 +74,7 @@ The ISP driver offers following services:
- :ref:`isp-hist-statistics` - covers how to get histogram statistics one-shot or continuously.
- :ref:`isp-bf` - covers how to enable and configure BF function.
- :ref:`isp-blc` - covers how to enable and configure BLC function.
- :ref:`isp-dpc` - covers how to configure static and dynamic dead pixel correction.
- :ref:`isp-lsc` - covers how to enable and configure LSC function.
- :ref:`isp-ccm-config` - covers how to configure the CCM.
- :ref:`isp-demosaic` - covers how to configure the Demosaic function.
@@ -589,6 +590,184 @@ Calling :cpp:func:`esp_isp_blc_set_correction_offset` to set the BLC correction
ESP_ERROR_CHECK(esp_isp_blc_set_correction_offset(isp_proc, &blc_offset));
.. _isp-dpc:
ISP DPC Controller
^^^^^^^^^^^^^^^^^^
Dead Pixel Correction (DPC) corrects defective pixels in RAW Bayer images before later ISP processing stages. Since adjacent Bayer pixels have different colors, DPC uses the eight same-color neighbors around the center pixel to form a 3×3 same-color neighborhood. When it detects a defective pixel, the hardware replaces the center pixel with the neighborhood median.
DPC supports two complementary correction modes:
- **Static correction** is intended for defects at fixed locations. Software uses a uniform white frame to calibrate dark pixels and a uniform black frame to calibrate bright pixels, then merges the results into a coordinate list; a previously calibrated list can also be supplied. During configuration, the driver writes the list to the hardware LUT, and the hardware replaces pixels at matching coordinates with the neighborhood median in every frame.
- **Dynamic correction** is intended for transient defects or defects at unknown locations and does not require a coordinate list. Dynamic method 1 uses the same-color neighborhood minimum, maximum, and absolute thresholds to detect bright and dark defects. Dynamic method 2 first screens the center pixel with a neighborhood-maximum ratio range, then applies a second test using a neighborhood estimate and adaptive bright and dark factors.
Both modes can be enabled together. While DPC is disabled, call :cpp:func:`esp_isp_dpc_static_configure` and :cpp:func:`esp_isp_dpc_dynamic_configure` for the correction modes to use, call :cpp:func:`esp_isp_dpc_configure` to apply common DPC settings, then call :cpp:func:`esp_isp_dpc_enable`. To change the static coordinate list, disable DPC before configuring it again.
Static Correction and Calibration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Static correction accepts 0 to 512 :cpp:type:`esp_isp_dpc_pixel_coord_t` coordinates. Each coordinate contains ``x`` and ``y`` fields. The array must be in ascending y/x order, contain no duplicates, and all coordinates must be inside the input frame. :cpp:func:`esp_isp_dpc_static_configure` converts the coordinates to the hardware LUT format during the call, so the caller may release the coordinate array after the call returns.
To calibrate the coordinate list, use a uniform white frame to find dark pixels and a uniform black frame to find bright pixels. Each call to a calibration start API internally enables DPC and accepts one corresponding input frame. Reading the result disables DPC again, which allows the next calibration pass or the final static configuration to start.
The following sequence calibrates and enables static correction. ``white_frame`` and ``black_frame`` must be uniform RAW input frames. Acquire them as follows:
- **White frame**: Fill the sensor field of view with a uniform, texture-free bright target, such as an integrating-sphere source or a defocused matte white reflector. Avoid shadows, vignetting, reflections, and saturated areas.
- **Black frame**: Block all light, for example with a lens cap or a dark enclosure. Prevent light leaks, status LEDs, and other stray light from reaching the sensor.
The way to feed each frame to the ISP and wait for it to complete depends on the input source. The following uses DMA input as an example; ``output_frame`` is the DMA output buffer.
.. code-block:: c
esp_isp_dpc_calibration_config_t white_calibration_config = {
.threshold = 0xf0,
.enable_output = true,
};
esp_isp_dpc_calibration_config_t black_calibration_config = {
.threshold = 0x0a,
.enable_output = true,
};
static esp_isp_dpc_calibration_ref_t white_ref;
static esp_isp_dpc_calibration_ref_t black_ref;
static esp_isp_dpc_calibration_ref_t merged_ref;
// Find dark defective pixels from a white frame.
esp_isp_dpc_static_calibration_start_once(isp_proc, ESP_ISP_DPC_CALIBRATION_IMAGE_WHITE, &white_calibration_config);
// Feed white_frame to the ISP and wait until processing completes.
// For example, with DMA input:
esp_isp_dma_process_frame(isp_proc, output_frame, white_frame, 1000);
esp_isp_dpc_calibration_read_result(isp_proc, 1000, &white_ref);
// Find bright defective pixels from a black frame.
esp_isp_dpc_static_calibration_start_once(isp_proc, ESP_ISP_DPC_CALIBRATION_IMAGE_BLACK, &black_calibration_config);
// Feed black_frame to the ISP and wait until processing completes.
// For example, with DMA input:
esp_isp_dma_process_frame(isp_proc, output_frame, black_frame, 1000);
esp_isp_dpc_calibration_read_result(isp_proc, 1000, &black_ref);
// Merge, sort, and remove duplicate coordinates before writing the static LUT.
const esp_isp_dpc_calibration_ref_t *calibration_refs[] = {
&white_ref,
&black_ref,
};
esp_isp_dpc_calibration_merge_result(calibration_refs, 2, &merged_ref);
esp_isp_dpc_static_config_t static_dpc_config = {
.dead_pixel_coords = merged_ref.dead_pixel_coords,
.dead_pixel_count = merged_ref.dead_pixel_count,
};
esp_isp_dpc_config_t common_dpc_config = {
.flags.update_once_configured = true,
};
esp_isp_dpc_static_configure(isp_proc, &static_dpc_config);
esp_isp_dpc_configure(isp_proc, &common_dpc_config);
esp_isp_dpc_enable(isp_proc);
:cpp:func:`esp_isp_dpc_calibration_merge_result` is a software-only utility that accepts any positive number of references. It does not access an ISP processor or hardware. It sorts all coordinates by y/x, removes duplicates, and keeps at most :c:macro:`ESP_ISP_DPC_MAX_DEAD_PIXELS` coordinates.
To merge N references, pass an array of their addresses and the number of elements to :cpp:func:`esp_isp_dpc_calibration_merge_result`:
.. code-block:: c
const esp_isp_dpc_calibration_ref_t *refs[] = {
&ref_0,
&ref_1,
&ref_2,
};
const size_t ref_count = sizeof(refs) / sizeof(refs[0]);
static esp_isp_dpc_calibration_ref_t merged_ref;
ESP_ERROR_CHECK(esp_isp_dpc_calibration_merge_result(refs, ref_count, &merged_ref));
If a factory calibration or another source already provides a defective-pixel coordinate list, the white- and black-frame calibration flow above can be skipped. Pass the coordinate array directly to :cpp:func:`esp_isp_dpc_static_configure`:
.. code-block:: c
static const esp_isp_dpc_pixel_coord_t factory_bad_pixels[] = {
{.x = 24, .y = 24},
{.x = 56, .y = 24},
{.x = 25, .y = 72},
};
esp_isp_dpc_static_config_t static_dpc_config = {
.dead_pixel_coords = factory_bad_pixels,
.dead_pixel_count = sizeof(factory_bad_pixels) / sizeof(factory_bad_pixels[0]),
};
esp_isp_dpc_config_t common_dpc_config = {
.flags.update_once_configured = true,
};
esp_isp_dpc_static_configure(isp_proc, &static_dpc_config);
esp_isp_dpc_configure(isp_proc, &common_dpc_config);
esp_isp_dpc_enable(isp_proc);
Dynamic Correction
~~~~~~~~~~~~~~~~~~
Dynamic Method 1
++++++++++++++++
Dynamic method 1 uses absolute thresholds. A pixel is a bright candidate when it is greater than ``max8 + high_threshold`` and a dark candidate when it is less than ``min8 - low_threshold``, where ``min8`` and ``max8`` are calculated from the eight same-color neighbors.
.. code-block:: c
esp_isp_dpc_dynamic_config_t dpc_config = {
.method = ESP_ISP_DPC_DYNAMIC_METHOD_1,
.method_1 = {
.high_threshold = 48,
.low_threshold = 48,
},
};
esp_isp_dpc_config_t common_dpc_config = {
.flags.update_once_configured = true,
};
esp_isp_dpc_dynamic_configure(isp_proc, &dpc_config);
esp_isp_dpc_configure(isp_proc, &common_dpc_config);
esp_isp_dpc_enable(isp_proc);
Dynamic Method 2
++++++++++++++++
Dynamic method 2 detects defective pixels in two stages.
The first stage uses the maximum value ``max8`` of the eight same-color neighbors to screen the center pixel ``pixel_center``. The normal range is ``max8 * first_stage_lower_ratio < pixel_center < max8 * first_stage_upper_ratio``: pixels within this range pass the first stage, while pixels equal to a boundary or outside the range receive a second test. The first-stage ratios use fixed-point values: ``value = integer + decimal / ISP_DPC_RATIO_MAX``. Valid values are 0.0 to 1.0. For fractional values, set ``integer`` to 0 and ``decimal`` to 0 ... ``ISP_DPC_RATIO_MAX - 1``; for example, 0.5 is ``integer = 0`` and ``decimal = 8``. For 1.0, set ``integer`` to 1 and ``decimal`` to 0.
- ``first_stage_lower_ratio`` (range ``0.0`` to ``1.0``): Lower bound of the first-stage normal range. Raising it sends more dark pixels to the second-stage test; lowering it lets more dark pixels pass the first stage.
- ``first_stage_upper_ratio`` (range ``0.0`` to ``1.0``): Upper bound of the first-stage normal range. Raising it lets more bright pixels pass the first stage; lowering it sends more bright pixels to the second-stage test. It must be greater than ``first_stage_lower_ratio``, otherwise the configuration function returns ``ESP_ERR_INVALID_ARG``.
The second stage calculates the mean ``est`` of the eight neighbors, the absolute difference ``dif = abs(est - pixel_center)`` between ``est`` and the center pixel, and their mean ``avg``. A dark-pixel candidate is detected when ``est >= pixel_center`` and ``dif > avg * dark_deviation_factor``; a bright-pixel candidate is detected when ``est < pixel_center`` and ``dif > (255 - avg) * bright_deviation_factor``. The deviation factors use fixed-point values: ``value = integer + decimal / ISP_DPC_DEVIATION_FACTOR_MAX``. Valid values are 0.0 to 1.0. For fractional values, set ``integer`` to 0 and ``decimal`` to 0 ... ``ISP_DPC_DEVIATION_FACTOR_MAX - 1``; for example, 0.5 is ``integer = 0`` and ``decimal = 16``. For 1.0, set ``integer`` to 1 and ``decimal`` to 0.
- ``dark_deviation_factor`` (range ``0.0`` to ``1.0``): Second-stage dark-pixel sensitivity. Lowering it reduces the required dark-pixel deviation and corrects dark pixels more aggressively; raising it is more conservative.
- ``bright_deviation_factor`` (range ``0.0`` to ``1.0``): Second-stage bright-pixel sensitivity. Lowering it reduces the required bright-pixel deviation and corrects bright pixels more aggressively; raising it is more conservative.
.. code-block:: c
esp_isp_dpc_dynamic_config_t dpc_config = {
.method = ESP_ISP_DPC_DYNAMIC_METHOD_2,
.method_2 = {
.first_stage_lower_ratio = {
.integer = 0,
.decimal = 8,
},
.first_stage_upper_ratio = {
.integer = 1,
.decimal = 0,
},
.bright_deviation_factor = {
.integer = 0,
.decimal = 16,
},
.dark_deviation_factor = {
.integer = 0,
.decimal = 16,
},
},
};
esp_isp_dpc_config_t common_dpc_config = {
.flags.update_once_configured = true,
};
esp_isp_dpc_dynamic_configure(isp_proc, &dpc_config);
esp_isp_dpc_configure(isp_proc, &common_dpc_config);
esp_isp_dpc_enable(isp_proc);
.. _isp-lsc:
ISP LSC Controller
@@ -982,6 +1161,9 @@ API Reference
.. include-build-file:: inc/isp_lsc.inc
.. include-build-file:: inc/isp_ccm.inc
.. include-build-file:: inc/isp_demosaic.inc
.. include-build-file:: inc/isp_dpc.inc
.. include-build-file:: inc/isp_dpc_dynamic.inc
.. include-build-file:: inc/isp_dpc_static.inc
.. include-build-file:: inc/isp_sharpen.inc
.. include-build-file:: inc/isp_gamma.inc
.. include-build-file:: inc/isp_hist.inc