Building a circular buffer in C for real-time data streams

A circular buffer, sometimes called a ring buffer or cyclic queue, is a fixed-size data structure that treats memory as if the last position loops back to the first. When the write pointer reaches the end of the underlying array, it wraps around to the beginning, overwriting stale samples only when the buffer is genuinely full. This predictable behaviour makes circular buffers a favourite for embedded developers, audio engineers, and anyone who needs guaranteed-latency ingest of streaming samples.

In Australia, real-time data handling shows up in systems that the rest of the world rarely sees up close. Outback mining operations in the Pilbara stream drill telemetry across thousands of kilometres of radio links to control rooms in Perth. Adelaide's renewable-heavy grid, dominated by wind farms on the Yorke Peninsula, fires inverter readings into substations at millisecond cadence. Bushfire detection networks operated by the CSIRO rely on similar pipelines, and even small studios streaming ABC radio content need jitter-free buffers to keep codecs happy. A circular buffer written in C remains a sensible building block for all of them.

C suits this work because it gives the programmer direct control over memory layout and pointer arithmetic, both of which matter when you care about cache locality and nanosecond-scale latency. Languages with garbage collection introduce pauses that simply cannot be tolerated in control loops for drones, medical devices, or audio drivers. C also compiles cleanly onto microcontrollers used by hobbyists and by local manufacturers prototyping industrial sensors in Brisbane and Melbourne labs. A well-written buffer in C is portable, predictable, and easy to audit, which is why it keeps appearing in firmware reviewed by teams at institutions such as UNSW and the Defence Science and Technology Group in Edinburgh, South Australia.

This walkthrough builds a working circular buffer step by step. It covers the core data structure, modular arithmetic for wrap-around, thread safety choices, fixed-point considerations for embedded targets, testing under real-time pressure, and a couple of extensions useful for telemetry and audio pipelines.

Memory layout and core structure

The simplest circular buffer stores elements in a contiguous array together with two indices: one for the head (where the next write lands) and one for the tail (where the next read begins). A count field tracks how many slots are currently occupied, which removes the classic ambiguity between an empty and a full buffer that arises when head and tail point at the same location.

A typical declaration in C looks like this:

typedef struct {
    uint8_t  *data;
    size_t    capacity;
    size_t    head;
    size_t    tail;
    size_t    count;
} ring_buffer_t;

Keeping the structure compact helps the cache, because every push or pop touches only a handful of fields. For audio work at 48 kHz stereo, a 1024-sample buffer fits comfortably in a single cache line on modern ARM cores used in NBN-connected premises gateways and consumer DACs. For larger telemetry frames from mining rigs, you might bump capacity into the tens of thousands, accepting that pushes will occasionally spill across cache lines.

Choosing the element width is part of the design. A buffer of uint8_t suits raw bytes from a serial port, while a buffer of float or int16_t suits processed samples. Some shops prefer a void-pointer layout so the same structure can hold any payload, at the cost of an extra dereference on every access. Local preference varies: Perth-based audio teams often lean toward typed arrays for clarity, while embedded teams in Canberra working on defence contracts favour the generic version for reuse across projects.

Wrap-around with modular arithmetic

Wrap-around is what gives the circular buffer its name, and the standard trick is to compute (index + 1) % capacity. With a power-of-two capacity, the modulo can be replaced with a bitwise AND against capacity - 1, which compiles down to a single instruction on most CPUs. That micro-optimisation matters when a buffer drains a thousand times per second from a weather radar feed near Hobart or from a phased-array receiver at a University of Adelaide research campus.

Two push helpers illustrate the pattern:

bool rb_push(ring_buffer_t *rb, uint8_t value) {
    if (rb->count == rb->capacity) return false; // full
    rb->data[rb->head] = value;
    rb->head = (rb->head + 1) & (rb->capacity - 1);
    rb->count++;
    return true;
}

Note the use of & rather than %. The compiler must be able to prove that capacity is a power of two at the call site, which is best enforced with a _Static_assert when the buffer is created. Without that guard, the bitwise AND silently corrupts data on every wrap.

The pop mirror looks similar, advancing tail instead of head and decrementing count. Returning a boolean lets the caller decide whether to retry, log, or drop the sample. Dropping is usually correct for telemetry from sensors that report once per minute, while retrying is usually correct for audio underrun recovery in a DAW plug-in running on a laptop in Surry Hills.

Thread safety and lock-free choices

Single-threaded code is easy. The moment a producer and a consumer run on different cores, the buffer becomes a shared resource. The simplest fix is a mutex around every push and pop, which works but reintroduces latency spikes that real-time systems try to avoid. For high-throughput paths, lock-free designs built on C11 atomics give better worst-case timing.

The standard single-producer single-consumer (SPSC) ring buffer uses an atomic head and tail, with the consumer reading the producer's index and vice versa. Acquire and release semantics on the atomics make the writes to the data array visible in the right order. On an Apple silicon laptop or a Ryzen-based workstation used in a Melbourne design studio, a well-tuned SPSC buffer can sustain tens of millions of operations per second without ever blocking.

For multi-producer scenarios, things get harder. A common approach is to give each producer its own private buffer and have a downstream stage merge the streams, which mirrors how Bureau of Meteorology ingest systems handle overlapping radar sweeps. Another approach uses compare-and-swap loops to claim slots, which is portable but slower and trickier to reason about.

Strategy	Worst-case latency	Throughput	Implementation effort
Mutex around push/pop	Unbounded under contention	Limited by lock handover	Low
SPSC lock-free with C11 atomics	Bounded by cache coherence	Very high	Medium
MPSC with CAS slot claiming	Bounded by retry loops	Moderate	High
Per-producer private buffers merged downstream	Bounded by merge stage	High	Medium

The table summarises the trade-offs. Picking the wrong row tends to show up as audio glitches, dropped telemetry, or runaway CPU usage on the producer side.

Fixed-point and embedded considerations

When the target is a microcontroller without a floating-point unit, every sample must be stored as an integer. Telemetry payloads from remote weather stations in the Snowy Mountains or from cattle-tracking collars on stations outside Alice Springs often arrive as 16-bit signed values scaled by a fixed factor. Choosing int16_t or int32_t over float saves flash, RAM, and cycles.

Alignment matters too. Many DMA engines on STM32 and NXP parts prefer 32-bit-aligned buffers. Declaring the storage with __attribute__((aligned(4))) or the C11 alignas keyword avoids cache-line splits and silent corruption on some ARM revisions. A statically allocated buffer also makes the linker map simpler, which the firmware team reviewing the binary will appreciate.

Power-of-two sizing helps again, not just for the modulo trick but because it lets you use bit shifts to convert between sample counts and byte counts. A 2048-sample buffer of 16-bit audio is exactly 4096 bytes, which is also the page size on most embedded MMUs and a common DMA burst length.

Testing under real-time pressure

A buffer that passes unit tests can still fail in production. Real-time code needs stress tests that mimic the actual cadence and jitter of incoming data. One useful technique is to drive the buffer from a hardware timer or a high-priority thread and measure worst-case latency, not average latency. The Australian Cyber Security Centre publishes guidance on this kind of measurement for critical infrastructure, and the same ideas apply to firmware.

A small driver that fills the buffer, drains it, and checks every element catches most regressions. Add a fault-injection mode that randomly drops push attempts, and you start to see whether downstream code copes with gaps. For audio, an off-target test that streams a sine wave through the buffer and checks the output for clicks or dropouts is worth the few hours it takes to set up.

Property-based testing is another lever. Asserting invariants like 0 <= count <= capacity and head == (initial_head + pushes) % capacity after every operation catches off-by-one bugs that hand-written tests miss. The kind of statistical reasoning covered in a practical introduction to Bayesian statistics review is useful for estimating how many random iterations you need to feel confident the buffer is solid.

Extending the design for audio and telemetry

Once the basic ring works, two extensions cover most real-world needs. The first is a peek API that lets the consumer look ahead by N samples without consuming them, which is essential for audio visualisers that need a window of past samples for FFTs. The second is a batch API that pushes or pops several samples in one call, which avoids the overhead of locking or atomic operations per sample when handling telemetry bursts.

For audio specifically, a double buffer where the consumer and producer swap roles on every period keeps glitches out of the output stream. DAW plug-ins and game audio engines have used this trick for decades. For telemetry, wrapping the buffer with a small protocol layer that prepends sequence numbers and CRCs turns raw bytes into something a receiver can validate, which is how most NBN-side smart meters report usage back to retailers.

A useful checklist before shipping:

• Verify the capacity is a power of two with a static assertion.
• Benchmark worst-case latency, not just throughput.
• Run the buffer under the highest expected load for at least an hour.
• Test the empty and full boundary conditions explicitly.
• Profile memory access patterns to confirm cache behaviour matches the design.

Common pitfalls to avoid:

• Using signed indices and accidentally letting tail wrap into negative territory.
• Forgetting to make head and tail volatile or atomic when shared with an ISR.
• Mixing modulo and bitwise wrap-arithmetic in the same buffer.
• Allocating the storage on the heap inside a real-time thread, which can stall on a missing page.
• Treating the buffer as thread-safe without proving the memory order on every architecture.

The patterns above give any C programmer a solid starting point, whether the destination is a Raspberry Pi attached to a rooftop inverter in Fremantle or a bare-metal microcontroller driving sensors on a remote cattle station in the Kimberley. The hello ML blog publishes plenty of similar walkthroughs for data structures and algorithms, and the techniques transfer cleanly into interview prep for software roles advertised by Australian employers in fintech, defence, and resources.