NanoPB: Custom array converters with offset and size in C++
See also: C version: How to handle custom array converters in C
NanoPB is a code-size optimized Protocol Buffers implementation for embedded systems. This post shows how to handle custom array converters in C++ with NanoPB, based on the KKS-Firmware implementation.
Proto definition
First, create a .proto file with repeated fields:
syntax = "proto3";
package example;
message CustomArrayMessage {
repeated uint32 values = 1;
}Generate NanoPB code
Generate the NanoPB code with a .options file to specify max count:
Create custom_array.options:
example.CustomArrayMessage.values max_count:20Then generate:
protoc --nanopb_out=. custom_array.protoThis will generate custom_array.pb.h and custom_array.pb.c.
C++ example with custom array converter
Here’s a complete C++ example implementing a custom array converter with offset and size:
#include <stdio.h>
#include <algorithm>
#include <array>
#include <limits>
#include "custom_array.pb.h"
#include "pb_encode.h"
#include "pb_decode.h"
// Custom array converter with offset and size template parameters
template<class ITEM_CONVERTER, class CONTAINER, size_t OFFSET=0, size_t SIZE=std::numeric_limits<size_t>::max()/2>
class ArrayConverterWithOffsetAndSize {
public:
static bool encodeCallback(pb_ostream_t *stream, const pb_field_t *field, const CONTAINER &container) {
size_t i = 0;
// Use for loop to iterate through container
for (const auto &item : container) {
i++; // First index we check is 1
if(i <= OFFSET) { // <=, not "<", since we ++ before this line
continue;
}
if(i > OFFSET + SIZE) {
break;
}
// Encode the item
if (!ITEM_CONVERTER::encodeCallback(stream, field, item)) {
return false;
}
}
return true;
}
static bool decodeCallback(pb_istream_t *stream, const pb_field_t *field, CONTAINER &container) {
// Add item to container
// Note: This is a simplified example - real implementation depends on container type
return false; // Not implemented in this example
}
};
// Simple uint32 converter
class UInt32Converter {
public:
static bool encodeCallback(pb_ostream_t *stream, const pb_field_t *field, uint32_t value) {
if (!pb_encode_tag_for_field(stream, field))
return false;
return pb_encode_varint(stream, value);
}
};
int main() {
// Buffer for encoded message
uint8_t buffer[256];
size_t message_length;
// Create array with 10 elements
std::array<uint32_t, 10> values = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
// --- ENCODING ---
example_CustomArrayMessage message = example_CustomArrayMessage_init_zero;
// Use custom converter: encode elements 3-7 (OFFSET=2, SIZE=5)
using CustomConverter = ArrayConverterWithOffsetAndSize<UInt32Converter, std::array<uint32_t, 10>, 2, 5>;
// Set up callback
message.values.funcs.encode = [](pb_ostream_t *stream, const pb_field_t *field, void * const *arg) {
const std::array<uint32_t, 10>* container = (const std::array<uint32_t, 10>*)*arg;
return CustomConverter::encodeCallback(stream, field, *container);
};
message.values.arg = &values;
// Create stream for encoding
pb_ostream_t ostream = pb_ostream_from_buffer(buffer, sizeof(buffer));
// Encode the message
if (!pb_encode(&ostream, example_CustomArrayMessage_fields, &message)) {
printf("Encoding failed: %s\n", PB_GET_ERROR(&ostream));
return 1;
}
message_length = ostream.bytes_written;
printf("Encoded %zu bytes (elements 3-7)\n", message_length);
// Print hex dump
printf("Encoded data: ");
for (size_t i = 0; i < message_length; i++) {
printf("%02x ", buffer[i]);
}
printf("\n");
// Print what was encoded
printf("Encoded values: ");
for (size_t i = 2; i < 2 + 5 && i < values.size(); i++) {
printf("%u ", values[i]);
}
printf("\n");
return 0;
}Compile command
Compile the example with nanopb. NanoPB is typically used by including the source files directly in your project:
g++ -o custom_array_example custom_array_example.cpp custom_array.pb.c pb_common.c pb_encode.c pb_decode.c -I.Note: NanoPB source files (pb_common.c, pb_encode.c, pb_decode.c) need to be compiled directly with your project. You can obtain these from the NanoPB GitHub repository.
Key points
- Template parameters: OFFSET and SIZE control which elements to encode
- Offset logic: Skip first OFFSET elements (1-based indexing in the loop)
- Size limit: Encode at most SIZE elements after the offset
- Callback pattern: Use pb_callback_t with custom encode/decode functions
- Flexibility: Can encode any slice of an array without copying
- Memory efficiency: No need to create temporary arrays
- KKS-Firmware pattern: Based on ArrayConverterWithOffsetAndSize from KKS-Firmware
When to use custom array converters
- When you need to encode a subset of a large array
- When you want to avoid copying data
- When implementing pagination or chunking
- When array layout doesn’t match protobuf requirements
- When you need custom encoding logic for arrays
Expected output
Encoded 10 bytes (elements 3-7)
Encoded data: 08 03 08 04 08 05 08 06 08 07
Encoded values: 3 4 5 6 7 Note that only elements 3, 4, 5, 6, 7 are encoded (5 elements starting at offset 2).
Advanced: Decoding support
To support decoding, extend the converter:
static bool decodeCallback(pb_istream_t *stream, const pb_field_t *field, CONTAINER &container) {
uint64_t value;
if (!pb_decode_varint(stream, &value))
return false;
// Add to container (implementation depends on container type)
// For std::array, you might track a separate index
return true;
}Real-world usage
This pattern is used in KKS-Firmware for:
- Encoding only active channels from a larger channel array
- Sending partial sensor data to save bandwidth
- Implementing differential updates
- Handling sparse arrays efficiently
## More NanoPB posts
- [Basic scalar types in C++](/2026/05/09/nanopb-cpp-basic-scalar-types/)
- [Basic scalar types in C](/2026/05/09/nanopb-c-basic-scalar-types/)
- [String types in C++](/2026/05/09/nanopb-cpp-string-types/)
- [String types in C](/2026/05/09/nanopb-c-string-types/)
- [Bytes types in C++](/2026/05/09/nanopb-cpp-bytes-types/)
- [Bytes types in C](/2026/05/09/nanopb-c-bytes-types/)
- [Optional fields in C++](/2026/05/09/nanopb-cpp-optional-fields/)
- [Optional fields in C](/2026/05/09/nanopb-c-optional-fields/)
- [Repeated fields/arrays in C++](/2026/05/09/nanopb-cpp-repeated-fields/)
- [Repeated fields/arrays in C](/2026/05/09/nanopb-c-repeated-fields/)
- [Enums in C++](/2026/05/09/nanopb-cpp-enums/)
- [Enums in C](/2026/05/09/nanopb-c-enums/)
- [Nested messages in C++](/2026/05/09/nanopb-cpp-nested-messages/)
- [Nested messages in C](/2026/05/09/nanopb-c-nested-messages/)
- [Oneof/union types in C++](/2026/05/09/nanopb-cpp-oneof-types/)
- [Oneof/union types in C](/2026/05/09/nanopb-c-oneof-types/)
- [Custom array converters in C](/2026/05/09/nanopb-c-custom-array-converters/)