ESP-IDF project standards and conventions
I've built a bunch of ESP32 projects using Espressif's ESP-IDF framework, and every time I start a new one, I second-guess myself on basic structure: where does the config go? How do I version the firmware? What partition layout should I use? This post is documentation-to-self, but hopefully it saves someone else from the iterative debugging I've done.
Project structure and build configuration
The ESP-IDF follows a pretty strict layout. At the top level you'll have:
-
main/: your application code (main.cis the entry point) -
CMakeLists.txt: project-level build configuration -
sdkconfig.defaults: sensible defaults for menuconfig options -
partitions.csv: partition table (more on that below) -
.clangd: language server config to fix false positives on macOS
The main/CMakeLists.txt is minimal:
idf_component_register(SRCS "main.c" INCLUDE_DIRS ".")
Keep components in main/ until you have more than a few files, then break them into components/mycomponent/.
OTA update patterns: ota_0 and ota_1
Never use the factory+ota_0 partition layout. I learned this the hard way when I bricked a device. Use ota_0 and ota_1 instead, so you can always fall back to the previous firmware if an update goes wrong.
Your partitions.csv should look like:
nvs, data, nvs, 0x9000, 0x6000, otadata, data, ota, 0xf000, 0x2000, ota_0, app, ota_0, 0x20000, 0x1C0000, ota_1, app, ota_1, 0x1E0000, 0x1C0000, spiffs, data, spiffs, 0x3A0000, 0x60000,
The sizes here depend on your flash size (I'm assuming 4MB). The key is that otadata tracks which OTA partition is active, and the bootloader automatically switches between them. On a bad update, it rolls back to the previous partition on next boot.
I allocate 1.75 MB (0x1C0000) per OTA partition because my firmware is usually 600-800 KB, and that leaves plenty of headroom for growth without requiring a reflash to resize.
Firmware versioning
Every binary that gets flashed or pushed to S3 needs a version bump. Use a simple semver string:
// main/main.c #define FIRMWARE_VERSION "1.2.3" #define FIRMWARE_VERSION_MAJOR 1 #define FIRMWARE_VERSION_MINOR 2 #define FIRMWARE_VERSION_PATCH 3
Then in your OTA or setup code, log it:
ESP_LOGI(TAG, "Firmware version: %s", FIRMWARE_VERSION);
This sounds obvious, but I've spent frustrating hours trying to reproduce a bug and realizing the device is running an older build than I thought. A simple version string at startup saves that pain.
If you're pushing to S3 for OTA (see below), include the version in the build artifact name: myproject-v1.2.3.bin.
WiFi authentication: stick with WPA2_PSK
Always use WIFI_AUTH_WPA2_PSK, not WIFI_AUTH_WPA2_WPA3_PSK. The mixed mode causes mysterious auth failures on home routers, especially older ones or devices that don't advertise both protocols equally.
wifi_config_t wifi_config = { .sta = { .ssid = (uint8_t *)CONFIG_WIFI_SSID, .password = (uint8_t *)CONFIG_WIFI_PASSWORD, .threshold.authmode = WIFI_AUTH_WPA2_PSK, }, };
Even if your home network supports WPA3, the mixed mode is a footgun. If you're deploying to multiple locations or devices with varying WiFi hardware, stick with WPA2_PSK. It's universally supported and won't surprise you.
Partition sizing and resizing pain
Leave expansion room in your OTA partitions. I allocate 1.75 MB when my current firmware is 600 KB because resizing partitions later requires a USB cable and esptool.py flash_id followed by a full reflash. It's doable but annoying.
If your project grows and you need more space, you have two options: 1. Resize partitions (USB reflash required) 2. Move to a larger flash chip (harder)
Allocate generously at the start. A few MB of unused space costs nothing.
Clangd/LSP false positives on macOS
If you're using clangd or VSCode's C/C++ extension on macOS, you'll see tons of red squiggles for ESP-IDF types and macros that actually compile fine. This is because clangd doesn't know about the build configuration.
Create a .clangd file at your project root:
CompileFlags: CompilationDatabase: build/compile_commands.json
Build once (so compile_commands.json exists), then clangd will use it and the false positives disappear. This is especially important for understanding macro expansions and avoiding frustration while editing.
S3 OTA uploads: the cache-control trap
If you're pushing built binaries to S3 for OTA updates, always include --cache-control "max-age=30" when uploading:
AWS_PROFILE=yourprofile aws s3 cp build/myproject.bin s3://your-bucket/firmware/ \ --cache-control "max-age=30"
Without this, CloudFront (or any other CDN in front of S3) will serve stale versions for up to 1 hour, and your devices will pull the old firmware. I've spent hours debugging "why is the new firmware not deployed?" only to realize the CDN was serving the previous build.
Set max-age to something short (30 seconds works) so you get fresh binaries on each request, not "fresh every hour."
Battery-powered sensors: ESP-NOW vs WiFi
If you're building a battery-powered sensor, ESP-NOW uses far less power than WiFi, but requires a receiver that's always on (or close to it). Tradeoffs:
WiFi: - Higher latency and power draw (radio startup is expensive) - Works everywhere without custom infrastructure - Built-in over-the-air update support
ESP-NOW: - One-way or paired communication only; you need a gateway/receiver - Extremely low power if you can wake, transmit, and sleep in milliseconds - Mesh capability with enough peers
For a humidity sensor that transmits once per hour, ESP-NOW + a gateway is overkill. For a door sensor that needs to wake and transmit immediately, ESP-NOW is worth the complexity. For anything in between, build WiFi first and optimize to ESP-NOW later if battery life becomes critical.
That's the foundation. Every new project I start, I copy the partition table, firmware version pattern, and build wrapper from the last one, and I've saved myself countless hours of "wait, why is this partition 512K?" debugging. Hopefully this saves you from the same mistakes.
