NanoPB: Custom array converters with offset and size in C
See also: C++ version: How to handle custom array converters in C++
NanoPB is a code-size optimized Protocol Buffers implementation for embedded systems. This post shows how to handle custom array converters in C with NanoPB.
Proto definition
First, create a .proto file with repeated fields:
syntax = "proto3";
package example;
message CustomArrayMessage {
repeated uint32 values = 1;
}Generate NanoPB code
Generate the NanoPB code with a .options file to specify max count:
Create custom_array.options:
example.CustomArrayMessage.values max_count:20Then generate:
protoc --nanopb_out=. custom_array.protoThis will generate custom_array.pb.h and custom_array.pb.c.
C example with custom array converter
Here’s a complete C example implementing a custom array converter with offset and size:
#include <stdio.h>
#include <stdint.h>
#include <stddef.h>
#include <limits.h>
#include "custom_array.pb.h"
#include "pb_encode.h"
#include "pb_decode.h"
// Custom array converter context
typedef struct {
const uint32_t* data;
size_t size;
size_t offset;
size_t count;
} array_context_t;
// Encode callback for uint32 array with offset and size
bool uint32_array_encode_callback(pb_ostream_t *stream, const pb_field_t *field, void * const *arg) {
const array_context_t* ctx = (const array_context_t*)*arg;
size_t start = ctx->offset;
size_t end = (ctx->offset + ctx->count < ctx->size) ?
ctx->offset + ctx->count : ctx->size;
for (size_t i = start; i < end; i++) {
if (!pb_encode_tag_for_field(stream, field))
return false;
if (!pb_encode_varint(stream, ctx->data[i]))
return false;
}
return true;
}
int main() {
// Buffer for encoded message
uint8_t buffer[256];
size_t message_length;
// Create array with 10 elements
uint32_t values[10] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
// --- ENCODING ---
example_CustomArrayMessage message = example_CustomArrayMessage_init_zero;
// Set up context: encode elements 3-7 (offset=2, count=5)
array_context_t ctx = {
.data = values,
.size = 10,
.offset = 2,
.count = 5
};
// Set up callback
message.values.funcs.encode = uint32_array_encode_callback;
message.values.arg = &ctx;
// Create stream for encoding
pb_ostream_t ostream = pb_ostream_from_buffer(buffer, sizeof(buffer));
// Encode the message
if (!pb_encode(&ostream, example_CustomArrayMessage_fields, &message)) {
printf("Encoding failed: %s\n", PB_GET_ERROR(&ostream));
return 1;
}
message_length = ostream.bytes_written;
printf("Encoded %zu bytes (elements 3-7)\n", message_length);
// Print hex dump
printf("Encoded data: ");
for (size_t i = 0; i < message_length; i++) {
printf("%02x ", buffer[i]);
}
printf("\n");
// Print what was encoded
printf("Encoded values: ");
for (size_t i = 2; i < 2 + 5 && i < 10; i++) {
printf("%u ", values[i]);
}
printf("\n");
return 0;
}Compile command
Compile the example with nanopb. NanoPB is typically used by including the source files directly in your project:
gcc -o custom_array_example custom_array_example.c custom_array.pb.c pb_common.c pb_encode.c pb_decode.c -I.Note: NanoPB source files (pb_common.c, pb_encode.c, pb_decode.c) need to be compiled directly with your project. You can obtain these from the NanoPB GitHub repository.
Key points
- Context structure: Use a struct to pass offset, count, and data pointer
- Offset logic: Start encoding at specified offset
- Count limit: Encode at most count elements after offset
- Callback pattern: Use pb_callback_t with custom encode function
- Flexibility: Can encode any slice of an array without copying
- Memory efficiency: No need to create temporary arrays
- KKS-Firmware pattern: Based on ArrayConverterWithOffsetAndSize from KKS-Firmware
When to use custom array converters
- When you need to encode a subset of a large array
- When you want to avoid copying data
- When implementing pagination or chunking
- When array layout doesn’t match protobuf requirements
- When you need custom encoding logic for arrays
Expected output
Encoded 10 bytes (elements 3-7)
Encoded data: 08 03 08 04 08 05 08 06 08 07
Encoded values: 3 4 5 6 7 Note that only elements 3, 4, 5, 6, 7 are encoded (5 elements starting at offset 2).
Advanced: Generic template-like implementation
For a more generic C implementation (simulating templates):
#include <stddef.h>
#include <limits.h>
#define MAX_OFFSET_SIZE (SIZE_MAX / 2)
typedef struct {
const void* data;
size_t element_size;
size_t size;
size_t offset;
size_t count;
bool (*encode_element)(pb_ostream_t*, const pb_field_t*, const void*);
} generic_array_context_t;
bool generic_array_encode_callback(pb_ostream_t *stream, const pb_field_t *field, void * const *arg) {
const generic_array_context_t* ctx = (const generic_array_context_t*)*arg;
size_t start = ctx->offset;
size_t end = (ctx->offset + ctx->count < ctx->size) ?
ctx->offset + ctx->count : ctx->size;
const uint8_t* data = (const uint8_t*)ctx->data;
for (size_t i = start; i < end; i++) {
const void* element = data + (i * ctx->element_size);
if (!ctx->encode_element(stream, field, element)) {
return false;
}
}
return true;
}Real-world usage
This pattern is used in KKS-Firmware for:
- Encoding only active channels from a larger channel array
- Sending partial sensor data to save bandwidth
- Implementing differential updates
- Handling sparse arrays efficiently
Differences from C++
The C version differs from C++ in several ways:
- No templates: Use structs and function pointers instead
- Manual context management: Must pass context explicitly
- No std::array: Use plain C arrays with size tracking
- No std::algorithm: Implement iteration manually
- Same callback pattern: Both use pb_callback_t
## More NanoPB posts
- [Basic scalar types in C++](/2026/05/09/nanopb-cpp-basic-scalar-types/)
- [Basic scalar types in C](/2026/05/09/nanopb-c-basic-scalar-types/)
- [String types in C++](/2026/05/09/nanopb-cpp-string-types/)
- [String types in C](/2026/05/09/nanopb-c-string-types/)
- [Bytes types in C++](/2026/05/09/nanopb-cpp-bytes-types/)
- [Bytes types in C](/2026/05/09/nanopb-c-bytes-types/)
- [Optional fields in C++](/2026/05/09/nanopb-cpp-optional-fields/)
- [Optional fields in C](/2026/05/09/nanopb-c-optional-fields/)
- [Repeated fields/arrays in C++](/2026/05/09/nanopb-cpp-repeated-fields/)
- [Repeated fields/arrays in C](/2026/05/09/nanopb-c-repeated-fields/)
- [Enums in C++](/2026/05/09/nanopb-cpp-enums/)
- [Enums in C](/2026/05/09/nanopb-c-enums/)
- [Nested messages in C++](/2026/05/09/nanopb-cpp-nested-messages/)
- [Nested messages in C](/2026/05/09/nanopb-c-nested-messages/)
- [Oneof/union types in C++](/2026/05/09/nanopb-cpp-oneof-types/)
- [Oneof/union types in C](/2026/05/09/nanopb-c-oneof-types/)
- [Custom array converters in C++](/2026/05/09/nanopb-cpp-custom-array-converters/)