Merge branch 'test/h264_on_esp32s31' into 'master'

feat(peripherals): add ESP32-S31 support to the h264 example

See merge request espressif/esp-idf!51624
This commit is contained in:
morris
2026-08-12 23:38:21 +08:00
8 changed files with 142 additions and 167 deletions
+3 -4
View File
@@ -129,11 +129,10 @@ examples/peripherals/gpio/matrix_keyboard:
examples/peripherals/h264:
enable:
- if: IDF_TARGET in ["esp32p4", "esp32s3"]
reason: only supports esp32p4 and esp32s3
- if: IDF_TARGET in ["esp32p4", "esp32s3", "esp32s31"]
reason: esp_h264 provides prebuilt codec libraries only for these targets
depends_components:
- *common_components
- esp_h264
- esp_hw_support
examples/peripherals/i2c/i2c_basic:
disable:
+20 -28
View File
@@ -1,5 +1,5 @@
| Supported Targets | ESP32-P4 | ESP32-S3 |
| ----------------- | -------- | -------- |
| Supported Targets | ESP32-P4 | ESP32-S3 | ESP32-S31 |
| ----------------- | -------- | -------- | --------- |
# H.264 Encoder-Decoder Example
@@ -8,7 +8,7 @@
This example demonstrates how to use H.264 hardware/software encoder and decoder with visual pattern generation:
- Generate colorful test patterns for video processing
- Encode video frames using H.264 codec (hardware on ESP32-P4, software on ESP32-S3)
- Encode video frames using the H.264 hardware or software encoder
- Decode the encoded frames back to original format using software decoder
- Display visual comparison between source and decoded images
@@ -19,8 +19,8 @@ The example supports multiple YUV formats and provides side-by-side colorized di
This example provides comprehensive configuration options through `idf.py menuconfig`:
### H.264 Encoder Type Selection
- **Hardware Encoder**: Available only on ESP32-P4, provides better performance and lower power consumption
- **Software Encoder**: Available on all targets (ESP32-S3, ESP32-P4), uses more CPU resources
- **Hardware Encoder**: Selected by default on targets with H.264 encoding hardware; provides better performance and lower power consumption.
- **Software Encoder**: Available on all supported targets; uses more CPU resources.
### Configurable Parameters
All parameters can be adjusted in "H.264 Example Configuration" menu:
@@ -32,16 +32,15 @@ All parameters can be adjusted in "H.264 Example Configuration" menu:
- **GOP Size**: 1-255 frames (default: 30)
- **QP Value**: 10-51 (default: 26 for hardware, 28 for software)
### Target-Specific Defaults
- **ESP32-P4**: Optimized for hardware encoding with higher performance settings
- **ESP32-S3**: Optimized for software encoding with conservative settings
### Defaults
The hardware encoder defaults to 30 fps, 512 Kbps, and QP 26. The software encoder uses conservative defaults of 15 fps, 256 Kbps, and QP 28.
## How to use example
### Prerequisites Required
This example requires:
- ESP32-P4 development board (for hardware encoding) or ESP32-S3 development board (for software encoding)
- A development board for one of the supported targets
- USB cable for programming and power supply
- Terminal that supports ANSI color codes for proper visual output
@@ -53,29 +52,22 @@ Before building, configure the example parameters:
idf.py menuconfig
```
Navigate to: `Component config` → `H.264 Example Configuration`
Navigate to: `Component config` → `H.264 Example Configuration`.
1. **Select Encoder Type**: Choose between Hardware (ESP32-P4 only) or Software encoder
1. **Select Encoder Type**: Choose between Hardware (when supported by the target) or Software encoder
2. **Adjust Parameters**: Configure video resolution, frame rate, bitrate, etc.
3. **Save and Exit**: Press 'S' to save configuration
### Build and Flash
For ESP32-P4 (with hardware encoding support):
```bash
idf.py set-target esp32p4
idf.py set-target TARGET
idf.py menuconfig # Configure as needed
idf.py build
idf.py -p PORT flash monitor
```
For ESP32-S3 (software encoding only):
```bash
idf.py set-target esp32s3
idf.py menuconfig # Software encoder will be automatically selected
idf.py build
idf.py -p PORT flash monitor
```
Replace `TARGET` with a supported target from the table. The hardware encoder is selected automatically when the target provides the H.264 encoder capability; otherwise the software encoder is selected.
(To exit the serial monitor, type ``Ctrl-]``.)
@@ -109,17 +101,17 @@ I (21475) main_task: Returned from app_main()
## Video Format Support
- **ESP_H264_RAW_FMT_I420**: Planar YUV 4:2:0 format (decoder output, software encoder input)
- **ESP_H264_RAW_FMT_O_UYY_E_VYY**: Interlaced YUV format (hardware encoder input on ESP32-P4)
- **ESP_H264_RAW_FMT_O_UYY_E_VYY**: Interlaced YUV format (hardware encoder input)
## Performance Recommendations
### For ESP32-P4 (Hardware Encoding):
- Resolution: Up to 1920x1080 supported
- Frame Rate: 30-60 fps achievable
### Hardware Encoder
- Resolution: 80x80 to 1920x1080 in this example
- Frame Rate: 30-60 fps is achievable, depending on resolution
- Bitrate: 512K-5M bps recommended
- QP: 20-30 for optimal quality/performance balance
### For ESP32-S3 (Software Encoding):
### Software Encoder
- Resolution: 320x240 or smaller recommended
- Frame Rate: 10-15 fps for stable performance
- Bitrate: 256K-1M bps recommended
@@ -129,7 +121,7 @@ I (21475) main_task: Returned from app_main()
**Configuration Issues:**
- Use `idf.py menuconfig` to verify H.264 settings before building
- Ensure hardware encoder is only selected for ESP32-P4 target
- Select the hardware encoder only when it is offered by menuconfig for the current target
**Memory allocation failures:**
- Reduce resolution or frame rate in menuconfig
@@ -137,13 +129,13 @@ I (21475) main_task: Returned from app_main()
- Check ESP-IDF memory configuration
**Encoding/decoding errors:**
- Verify the correct target is selected (ESP32-P4 for hardware)
- Verify the selected encoder is supported by the target
- Check that H.264 component is properly configured in menuconfig
- Adjust bitrate settings for your resolution/frame rate combination
**Performance Issues:**
- Lower resolution, frame rate, or bitrate for software encoding
- Use hardware encoder on ESP32-P4 for better performance
- Use the hardware encoder when it is available for better performance
- Increase QP value to reduce computational load
**Visual output issues:**
@@ -1,3 +1,3 @@
idf_component_register(SRC_DIRS "./"
idf_component_register(SRCS "esp_h264_enc_dec.c" "video_pattern.c"
INCLUDE_DIRS "./"
PRIV_REQUIRES esp_psram)
@@ -2,22 +2,22 @@ menu "H.264 Example Configuration"
choice H264_ENCODER_TYPE
prompt "H.264 Encoder Type"
default H264_ENCODER_HARDWARE if IDF_TARGET_ESP32P4
default H264_ENCODER_HARDWARE if SOC_H264_ENCODER_SUPPORTED
default H264_ENCODER_SOFTWARE
help
Select the H.264 encoder type to use.
Hardware encoder is only available on ESP32P4 and provides
Hardware encoder is available on targets with an H.264 encoder and provides
better performance and lower power consumption.
Software encoder is available on all targets but requires
more CPU resources.
config H264_ENCODER_HARDWARE
bool "Hardware Encoder (ESP32P4 only)"
depends on IDF_TARGET_ESP32P4
bool "Hardware Encoder"
depends on SOC_H264_ENCODER_SUPPORTED
help
Use hardware H.264 encoder.
This option is only available on ESP32P4 which has
dedicated H.264 hardware encoding capabilities.
This option is available on targets with dedicated
H.264 hardware encoding capabilities.
Provides better performance and lower power consumption
compared to software encoding.
@@ -25,9 +25,8 @@ menu "H.264 Example Configuration"
bool "Software Encoder"
help
Use software H.264 encoder using OpenH264 library.
Available on all supported targets (ESP32S3, ESP32P4)
but requires more CPU resources and power consumption
compared to hardware encoding.
Available on all supported targets but requires more CPU
resources and power consumption compared to hardware encoding.
endchoice
menu "H.264 Encoder Parameters"
+92 -120
View File
@@ -1,13 +1,13 @@
/**
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO LTD
* SPDX-FileCopyrightText: 2025-2026 Espressif Systems (Shanghai) CO LTD
*
* SPDX-License-Identifier: Apache-2.0
*/
#include <stdio.h>
#include <inttypes.h>
#include <stdbool.h>
#include "sdkconfig.h"
#include "esp_heap_caps.h"
#include "esp_h264_alloc.h"
#include "esp_h264_dec_sw.h"
#if CONFIG_H264_ENCODER_HARDWARE
@@ -18,164 +18,141 @@
#include "video_pattern.h"
#include "esp_log.h"
static const char *TAG = "H264_ENC_DEC";
static const char *TAG = "example";
#define FRAME_MAX_NUM 10
// Helper function to allocate aligned memory with error checking
static void *allocate_frame_buffer(size_t size, uint32_t *actual_size, const char *buffer_name)
/**
* @brief Allocate a 16-byte-aligned frame buffer in PSRAM.
*
* Keeping the source image and encoded bitstream in PSRAM leaves internal
* memory available for the system and codec allocations. The returned buffer
* is not zero-initialized: the pattern generator initializes the source frame,
* and the encoder writes the valid byte range recorded in enc_frame.length.
*/
static void *allocate_frame_buffer(size_t size, uint32_t *actual_size)
{
void *buffer = esp_h264_aligned_calloc(16, 1, size, actual_size, ESP_H264_MEM_SPIRAM);
if (!buffer) {
ESP_LOGE(TAG, "Failed to allocate %s buffer memory (%zu bytes)", buffer_name, size);
}
void *buffer = esp_h264_aligned_malloc(16, 1, size, actual_size, ESP_H264_MEM_SPIRAM);
ESP_ERROR_CHECK(buffer ? ESP_OK : ESP_ERR_NO_MEM);
return buffer;
}
// Helper function to initialize pattern info
static void init_pattern_info(pattern_info_t *pattern, uint32_t width, uint32_t height, uint32_t format_id)
/**
* @brief Initialize metadata used to generate or display a color-bar frame.
*/
static void init_pattern_info(pattern_info_t *pattern, uint16_t width, uint16_t height,
esp_h264_raw_format_t format_id)
{
pattern->res.width = width;
pattern->res.height = height;
pattern->format_id = format_id;
pattern->vertical = false;
pattern->bar_count = 16;
pattern->data_size = width * height * 3 / 2;
pattern->data_size = (uint32_t)(width * height * ESP_H264_GET_BPP_BY_PIC_TYPE(format_id));
}
/**
* @brief Decode all NAL units produced for one encoded video frame.
*
* esp_h264_dec_process() may consume only one NAL unit per call. It also
* returns no image for non-picture NAL units such as SPS and PPS, so retain
* the most recent non-empty decoded output while advancing by @c consume.
*/
static void decode_encoded_frame(esp_h264_dec_handle_t dec, const esp_h264_enc_out_frame_t *enc_frame,
esp_h264_dec_out_frame_t *dest_frame)
{
esp_h264_dec_in_frame_t dec_input = {
.raw_data = {
.buffer = enc_frame->raw_data.buffer,
.len = enc_frame->length,
},
};
esp_h264_dec_out_frame_t decoded_frame = {};
bool picture_ready = false;
while (dec_input.raw_data.len > 0) {
uint32_t remaining_len = dec_input.raw_data.len;
*dest_frame = (esp_h264_dec_out_frame_t) {};
ESP_ERROR_CHECK((esp_err_t)esp_h264_dec_process(dec, &dec_input, dest_frame));
ESP_ERROR_CHECK(dec_input.consume > 0 && dec_input.consume <= remaining_len ? ESP_OK : ESP_FAIL);
dec_input.raw_data.buffer += dec_input.consume;
dec_input.raw_data.len -= dec_input.consume;
if (dest_frame->out_size > 0) {
decoded_frame = *dest_frame;
picture_ready = true;
}
}
ESP_ERROR_CHECK(picture_ready ? ESP_OK : ESP_FAIL);
*dest_frame = decoded_frame;
}
/*
This function is used to encode and decode a single frame.
src_frame --> encoder --> enc_frame(dec_input) --> decoder --> dest_frame(out_pattern)
src_frame --> encoder --> enc_frame --> decoder --> dest_frame(out_pattern)
*/
#if CONFIG_H264_ENCODER_HARDWARE
esp_h264_err_t single_enc_dec_process(esp_h264_enc_cfg_hw_t enc_cfg, esp_h264_dec_cfg_sw_t dec_cfg)
static void single_enc_dec_process(esp_h264_enc_cfg_hw_t enc_cfg, esp_h264_dec_cfg_sw_t dec_cfg)
#else
esp_h264_err_t single_enc_dec_process(esp_h264_enc_cfg_sw_t enc_cfg, esp_h264_dec_cfg_sw_t dec_cfg)
static void single_enc_dec_process(esp_h264_enc_cfg_sw_t enc_cfg, esp_h264_dec_cfg_sw_t dec_cfg)
#endif /* CONFIG_H264_ENCODER_HARDWARE */
{
int frame_num = 0;
// Frame buffers - Fixed types to match decoder expectations
esp_h264_enc_in_frame_t src_frame = {0}; // Original input frame
esp_h264_enc_out_frame_t enc_frame = {0}; // Encoded frame output
esp_h264_dec_in_frame_t dec_input = {0}; // Decoder input frame (fixed type)
esp_h264_dec_out_frame_t dest_frame = {0}; // Decoded frame output (fixed type)
// Handles and variables
esp_h264_err_t ret = ESP_H264_ERR_OK;
esp_h264_enc_in_frame_t src_frame = {0};
esp_h264_enc_out_frame_t enc_frame = {0};
esp_h264_dec_out_frame_t dest_frame = {0};
esp_h264_enc_handle_t enc = NULL;
esp_h264_dec_handle_t dec = NULL;
// Pattern info structures
pattern_info_t in_pattern = {};
pattern_info_t out_pattern = {};
size_t frame_size = enc_cfg.res.width * enc_cfg.res.height;
size_t pixel_bits = 12; // 12 bits per pixel for YUV420
if (enc_cfg.pic_type == ESP_H264_RAW_FMT_YUYV) {
// Calculate frame size
pixel_bits = 16; // 16 bits per pixel for YUYV
}
frame_size *= pixel_bits;
size_t frame_size = (size_t)(enc_cfg.res.width * enc_cfg.res.height
* ESP_H264_GET_BPP_BY_PIC_TYPE(enc_cfg.pic_type));
// Initialize pattern configurations
// The encoder writes the source format; the software decoder always outputs I420.
init_pattern_info(&in_pattern, enc_cfg.res.width, enc_cfg.res.height, enc_cfg.pic_type);
init_pattern_info(&out_pattern, enc_cfg.res.width, enc_cfg.res.height, dec_cfg.pic_type);
// Allocate frame buffers
src_frame.raw_data.buffer = allocate_frame_buffer(frame_size, &src_frame.raw_data.len, "source frame");
if (!src_frame.raw_data.buffer) {
goto cleanup;
}
// Because of the different bitrate, the encoded frame buffer size is different.
// It uses the same buffer size as the source frame to avoid not enough buffer error.
enc_frame.raw_data.buffer = allocate_frame_buffer(frame_size, &enc_frame.raw_data.len, "encoded frame");
if (!enc_frame.raw_data.buffer) {
goto cleanup;
}
src_frame.raw_data.buffer = allocate_frame_buffer(frame_size, &src_frame.raw_data.len);
// The encoder API recommends an output buffer at least as large as its input.
enc_frame.raw_data.buffer = allocate_frame_buffer(frame_size, &enc_frame.raw_data.len);
// Setup decoder input frame (correct structure for decoder)
dec_input.raw_data.buffer = enc_frame.raw_data.buffer;
// Assign pattern pixel buffers
in_pattern.pixel = src_frame.raw_data.buffer;
// Initialize H264 encoder
#if CONFIG_H264_ENCODER_HARDWARE
ret = esp_h264_enc_hw_new(&enc_cfg, &enc);
ESP_ERROR_CHECK((esp_err_t)esp_h264_enc_hw_new(&enc_cfg, &enc));
#else
ret = esp_h264_enc_sw_new(&enc_cfg, &enc);
ESP_ERROR_CHECK((esp_err_t)esp_h264_enc_sw_new(&enc_cfg, &enc));
#endif /* CONFIG_H264_ENCODER_HARDWARE */
if (ret != ESP_H264_ERR_OK) {
ESP_LOGE(TAG, "Failed to create H264 encoder (error: %d)", ret);
goto cleanup;
}
ret = esp_h264_enc_open(enc);
if (ret != ESP_H264_ERR_OK) {
ESP_LOGE(TAG, "Failed to open H264 encoder (error: %d)", ret);
goto cleanup;
}
// Opening prepares the codec instance after it has been created.
ESP_ERROR_CHECK((esp_err_t)esp_h264_enc_open(enc));
// Initialize H264 decoder
ret = esp_h264_dec_sw_new(&dec_cfg, &dec);
if (ret != ESP_H264_ERR_OK) {
ESP_LOGE(TAG, "Failed to create H264 decoder (error: %d)", ret);
goto cleanup;
}
ret = esp_h264_dec_open(dec);
if (ret != ESP_H264_ERR_OK) {
ESP_LOGE(TAG, "Failed to open H264 decoder (error: %d)", ret);
goto cleanup;
}
ESP_ERROR_CHECK((esp_err_t)esp_h264_dec_sw_new(&dec_cfg, &dec));
ESP_ERROR_CHECK((esp_err_t)esp_h264_dec_open(dec));
ESP_LOGI(TAG, "H264 encode-decode loop started (%dx%d @ %dfps)",
enc_cfg.res.width, enc_cfg.res.height, enc_cfg.fps);
while (1) {
// Generate input pattern
gen_pattern_color_bar(&in_pattern);
// Encode frame
ret = esp_h264_enc_process(enc, &src_frame, &enc_frame);
if (ret != ESP_H264_ERR_OK) {
ESP_LOGE(TAG, "H264 encoding failed (error: %d)", ret);
break;
}
//update decoder input
dec_input.raw_data.len = enc_frame.length;
// Decode frame
ret = esp_h264_dec_process(dec, &dec_input, &dest_frame);
if (ret != ESP_H264_ERR_OK) {
ESP_LOGE(TAG, "H264 decoding failed (error: %d)", ret);
break;
}
for (int frame_num = 0; frame_num < FRAME_MAX_NUM; frame_num++) {
ESP_ERROR_CHECK(gen_pattern_color_bar(&in_pattern));
ESP_ERROR_CHECK((esp_err_t)esp_h264_enc_process(enc, &src_frame, &enc_frame));
decode_encoded_frame(dec, &enc_frame, &dest_frame);
out_pattern.pixel = dest_frame.outbuf;
// Display conversion result
draw_convert_result(&in_pattern, &out_pattern);
printf("\nFrame %d: source image | decoded image\n", frame_num);
frame_num++;
if (frame_num >= FRAME_MAX_NUM) {
break;
}
draw_convert_result(&in_pattern, &out_pattern);
}
cleanup:
// Cleanup encoder
esp_h264_enc_close(enc);
esp_h264_enc_del(enc);
// Cleanup decoder
esp_h264_dec_close(dec);
esp_h264_dec_del(dec);
// Free memory buffers
if (src_frame.raw_data.buffer) {
esp_h264_free(src_frame.raw_data.buffer);
}
if (enc_frame.raw_data.buffer) {
esp_h264_free(enc_frame.raw_data.buffer);
}
ESP_LOGI(TAG, "H264 process %s", (ret == ESP_H264_ERR_OK) ? "Completed successfully" : "Failed");
return ret;
ESP_ERROR_CHECK((esp_err_t)esp_h264_dec_close(dec));
ESP_ERROR_CHECK((esp_err_t)esp_h264_dec_del(dec));
ESP_ERROR_CHECK((esp_err_t)esp_h264_enc_close(enc));
ESP_ERROR_CHECK((esp_err_t)esp_h264_enc_del(enc));
esp_h264_free(src_frame.raw_data.buffer);
esp_h264_free(enc_frame.raw_data.buffer);
ESP_LOGI(TAG, "H264 process completed successfully");
}
void app_main(void)
@@ -207,7 +184,7 @@ void app_main(void)
};
#endif /* CONFIG_H264_ENCODER_HARDWARE */
// Always use software decoder since decoder choice was removed
// This example uses the portable software decoder for both encoder modes.
esp_h264_dec_cfg_sw_t dec_cfg = {
.pic_type = ESP_H264_RAW_FMT_I420,
};
@@ -221,13 +198,8 @@ void app_main(void)
"Software"
#endif
);
// Fixed format specifiers to use PRIu32 for uint32_t values
ESP_LOGI(TAG, "Config: GOP=%d, Bitrate=%" PRIu32 " bps, QP=%d",
CONFIG_H264_ENCODER_GOP_SIZE, CONFIG_H264_ENCODER_BITRATE, CONFIG_H264_ENCODER_QP_VALUE);
// Start encode-decode process
esp_h264_err_t ret = single_enc_dec_process(enc_cfg, dec_cfg);
if (ret != ESP_H264_ERR_OK) {
ESP_LOGE(TAG, "H264 example failed with error: %d", ret);
}
single_enc_dec_process(enc_cfg, dec_cfg);
}
@@ -1,2 +1,2 @@
dependencies:
espressif/esp_h264: "^1.1.0"
espressif/esp_h264: "^1.3.8"
+10 -4
View File
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO LTD
# SPDX-FileCopyrightText: 2025-2026 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
import pytest
from pytest_embedded import Dut
@@ -8,10 +8,16 @@ from pytest_embedded_idf.utils import idf_parametrize
@pytest.mark.octal_psram
@idf_parametrize('target', ['esp32s3'], indirect=['target'])
def test_esp_h264_esp32s3(dut: Dut) -> None:
dut.expect_exact('H264 process Completed successfully')
dut.expect_exact('H264 process completed successfully')
@pytest.mark.generic
@idf_parametrize('target', ['esp32p4'], indirect=['target'])
def test_esp_h264_esp32p4(dut: Dut) -> None:
dut.expect_exact('H264 process Completed successfully')
def test_esp_h264_hardware_encoder(dut: Dut) -> None:
dut.expect_exact('H264 process completed successfully')
@pytest.mark.generic
@idf_parametrize('target', ['esp32s31'], indirect=['target'])
def test_esp_h264_esp32s31(dut: Dut) -> None:
dut.expect_exact('H264 process completed successfully')
@@ -0,0 +1,7 @@
# SPIRAM configurations for ESP32S31
CONFIG_SPIRAM=y
CONFIG_SPIRAM_MODE_OCT=y
CONFIG_SPIRAM_SPEED_250M=y
# CPU configuration
CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_320=y