docs/Writerside/topics/buffer.h.md

# Buffer

This buffer implementation can be used to read from or write to memory like you would do with a stream.

This allows the use of `cx_stream_copy()` (see [](streams.h.md)) to copy contents from one buffer to another,
or from a file or network streams to the buffer and vice versa.

More features for convenient use of the buffer can be enabled, like automatic memory management,
automatic resizing of the buffer space, or automatic flushing of contents.

The functions `cxBufferRead()` and `cxBufferWrite()` are `cx_read_func` and `cx_write_func` compatible,
which in turn have a compatible signature to `fread()` and `fwrite()`.
However, due to the different pointer type, the function pointers do not type check out of the box.
For convenience, the macros `cxBufferReadFunc` and `cxBufferWriteFunc` are defined, which perform the necessary cast.

## Example

In the following example we use a `CxBuffer`, the `bprintf()` macro from the [UCX printf header](printf.h.md),
and a [UCX map](map.h.md) to create a simple HTTP GET request.

```C
#include <cx/buffer.h>
#include <cx/printf.h>
#include <cx/map.h>

void send_get_request(
	void *net_target,
	cx_write_func net_write,
	const char *host,
	const char *path,
	CxMap *header
) {
	// initialize buffer and use stack memory for small requests
	char stackmem[128];
	CxBuffer buf;
	cxBufferInit(
		&buf, stackmem, sizeof(stackmem),
		cxDefaultAllocator,
		CX_BUFFER_COPY_ON_EXTEND // move to heap when request is larger
	);
	
	// basic request data
	cx_bprintf(&buf, "GET %s HTTP/1.1\r\n", path);
	cx_bprintf(&buf, "Host: %s\r\n", host);
	
	// add headers
	CxMapIterator iter = cxMapIterator(header);
	cx_foreach(CxMapEntry*, entry, iter) {
		cxstring name = cx_strn(entry->key->data, entry->key->len);
		cx_bprintf(&buf, "%" CX_PRIstr ": %s\r\n",
			CX_SFMT(name), entry->value
		);
	}
	
	// blank line terminates
	cxBufferWrite("\r\n", 1, 2, &buf);
	
	// write request to the network stream and destroy the buffer
	net_write(buf.space, 1, buf.size, net_target);
	cxBufferDestroy(&buf);
}
```

## Create

```C
#include <cx/buffer.h>

int cxBufferInit(CxBuffer *buffer, void *space, size_t capacity,
        const CxAllocator *allocator, int flags);

CxBuffer *cxBufferCreate(void *space,size_t capacity,
        const CxAllocator *allocator, int flags);

// available flags:
#define CX_BUFFER_DEFAULT
#define CX_BUFFER_FREE_CONTENTS
#define CX_BUFFER_AUTO_EXTEND
#define CX_BUFFER_COPY_ON_WRITE
#define CX_BUFFER_COPY_ON_EXTEND
```

For creating a UCX buffer you have two options: initialize a pre-allocated structure, or allocate and initialize a new structure in one call.

For the first option, you can call `cxBufferInit()` with the `buffer` argument pointing to an uninitialized `CxBuffer` structure.
You can pass a pre-allocated `space`, the desired `capacity`, and an `allocator`.
If `space` is `NULL`, the `allocator` is used to allocate enough space to match the desired `capacity`.

The `flags` argument is a bit-wise-or combination of the constants listed below.

The function `cxBufferCreate()` is equivalent, except that it uses the `allocator` to allocate a new `CxBuffer` structure, instead of initializing an existing one.

If any allocation fails, `cxBufferInit()` returns non-zero, or `cxBufferCreate()` returns `NULL`, respectively.

| Flag                     | Description                                                                                                                                                                                                   |
|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| CX_BUFFER_DEFAULT        | Alias for zero, meaning no flags are set.                                                                                                                                                                     |
| CX_BUFFER_FREE_CONTENTS  | When the buffer structure is [destroyed](#destroy), the memory for the contents is also automatically deallocated.                                                                                            |
| CX_BUFFER_AUTO_EXTEND    | When a write operation would exceed the buffers capacity, the buffer is grown automatically with a call to `cxBufferMinimumCapacity()`.                                                                       |
| CX_BUFFER_COPY_ON_WRITE  | Any write operation will result in allocating a copy of the data, first. This is particularly useful, when a buffer is initialized with read-only memory.                                                     |
| CX_BUFFER_COPY_ON_EXTEND | Only when a write operation would exceed the buffers capacity, the copy-on-write operation is performed. This is useful, when a buffer is initialized with read-write memory which is allocated on the stack. |


## Destroy

```C
#include <cx/buffer.h>

void cxBufferDestroy(CxBuffer *buffer);

void cxBufferFree(CxBuffer *buffer);
```

The above functions destroy the buffer and deallocate the buffer's memory when the `CX_BUFFER_FREE_CONTENTS` flag is set.

The function `cxBufferDestroy()` is to be used when the buffer was initialized with `cxBufferInit()`,
and the function `cxBufferFree()` is to be used when the buffer was created with `cxBufferCreate()`.
The only difference is, that `cxBufferFree()` additionally deallocates the memory of the `CxBuffer` structure. 

## Capacity Management

```C
#include <cx/buffer.h>

int cxBufferMinimumCapacity(CxBuffer *buffer, size_t capacity);

void cxBufferShrink(CxBuffer *buffer, size_t reserve);
```

The function `cxBufferMinimumCapacity()` guarantees a buffer capacity of _at least_ `capacity`.
The actual capacity will be a power of two until the system's page size is reached.
Then, the new capacity will be a multiple of the page size.
The function returns non-zero if increasing the capacity was attempted unsuccessfully.

The function `cxBufferShrink()` can be used to shrink the capacity of the buffer to its current size
plus a number of `reserve` bytes.
If the current capacity is not larger than the size plus the reserve bytes, the function will do nothing.

> If the buffer is in a copy-on-write state, `cxBufferMinimumCapacity()` will perform the copy-on-write action
> before reallocating the space.
> The function `cxBufferShrink()` on the other hand does _nothing_ when the buffer is in a copy-on-write state,
> because it does not make much sense to copy the memory just to have it in a smaller memory region.

## Write

```C
#include <cx/buffer.h>

size_t cxBufferWrite(const void *ptr, size_t size, size_t nitems,
        CxBuffer *buffer);

int cxBufferPut(CxBuffer *buffer, int c);

size_t cxBufferPutString(CxBuffer *buffer, const char *str);

int cxBufferTerminate(CxBuffer *buffer);

size_t cxBufferAppend(const void *ptr, size_t size, size_t nitems,
        CxBuffer *buffer);
```

The primary function for writing to a buffer is `cxBufferWrite()`
which writes up to `nitems` with `size` bytes each from the memory pointed to by `ptr` to the buffer.

When the capacity of the buffer is not sufficient and the `CX_BUFFER_AUTO_EXTEND` is not set in the buffer,
items that do not fit into the buffer are discarded.
The function always returns the actual number of items successfully written.
This equals the number of bytes if and only if `size=1`.

The function `cxBufferPut()` is a `putc()`-like wrapper for `cxBufferWrite()` which writes the character `c`,
converted to an `unsigned char` to the buffer.
Just like `putc()` this function returns the written character on success, and `EOF` on failure,
but it does _not_ set `errno` on failure.

The function `cxBufferPutString()` is a convenience function,
that uses stdlib `strlen()` to compute the length of `str` and then invokes `cxBufferWrite()`.

All of the above functions advance the buffers position by the number of bytes written,
and cause the _size_ of the buffer to grow, if necessary, to contain all written bytes.
On the other hand, `cxBufferTerminate()` writes a zero-byte at the current position,
effectively creating a zero-terminated string whose size equals the buffer size.

The function `cxBufferAppend()` writes the data to end of the buffer (given by its size) regardless of the current position,
and it also does _not_ advance the position.
If the write operation triggered a [flush](#flushing), however, the position will be shifted left alongside the shifted buffer contents.
In case the data at which the current position points gets flushed, the new position will be zero.

## Read

```C
#include <cx/buffer.h>

size_t cxBufferRead(void *ptr, size_t size, size_t nitems,
        CxBuffer *buffer);

int cxBufferGet(CxBuffer *buffer);

bool cxBufferEof(const CxBuffer *buffer);
```

The function `cxBufferRead()` reads `nitems` number of items of `size` bytes each from the `buffer`
and stores them into the memory pointed to by `ptr`, which must be sufficiently large to hold the contents.
The function returns the actual bytes read, which might be lower, if the desired number of items is not available.

The function `cxBufferGet()` is a `fgetc()`-like function which returns the next byte in the buffer converted to an `int`.
Similar to `fgetc()` it returns `EOF` when there are no more bytes in the buffer.

When all bytes from the buffer have been read, the function `cxBufferEof()` returns true. 

All read functions advance the position in the buffer by the number of read bytes.

> When you want to read from a buffer that you previously used for writing, do not forget to set the position
> in the buffer to zero, e.g. by calling `cxBufferSeek()`.

## Reset and Clear

```C
#include <cx/buffer.h>

void cxBufferReset(CxBuffer *buffer);

void cxBufferClear(CxBuffer *buffer);
```

The function `cxBufferReset()` sets both size and position of the buffer to zero,
and `cxBufferClear()` additionally uses `memset()` to set every byte in the buffer to zero.

> When clearing the buffer, only the "live" data, i.e. bytes with indices `[0..size)`, are cleared.
> If you which to clear the entire buffer's memory, you would need to set the size to the capacity, first.

> If the `CX_BUFFER_COPY_ON_WRITE` flag is set, `cxBufferClear()` behaves exactly like `cxBufferReset()`,
> because writing to the contents is disallowed.
>{style="note"}

## Random Access

```C
#include <cx/buffer.h>

int cxBufferSeek(CxBuffer *buffer, off_t offset, int whence);
```

The function `cxBufferSeek()` is a `fseek()`-like function that sets the current position in the buffer
relative to the start (when `whence` is `SEEK_SET`), the current position (when `whence` is `SEEK_CUR`),
or end (when `whence` is `SEEK_END`) of the buffer.

If the resulting position is negative, or larger than the size (i.e. the first unused byte),
this function returns non-zero and sets `errno` to `EINVAL`.

> Note that the behavior of `cxBufferSeek()` when seeking beyond the end of the buffer is not exactly the same as for POSIX `fseek()`.

## Shift Contents

```C
#include <cx/buffer.h>

int cxBufferShift(CxBuffer *buffer, off_t shift);

int cxBufferShiftRight(CxBuffer *buffer, size_t shift);

int cxBufferShiftLeft(CxBuffer *buffer, size_t shift);
```

The function `cxBufferShift()` moves the contents within the buffer by the specified `shift` offset,
where a negative offset means a shift to the left, and a positive offset means a shift to the right.
It also adjusts the current position within the buffer, and in case of a right shift also the size, by the same offset.

Data that is shift to the left, is always discarded when the new position of a byte would be smaller than zero.
If the new position would be smaller than zero, it is set exactly to zero.

When data is shift to the right, the behavior depends on the `CX_BUFFER_AUTO_EXTEND` flag.
If set, the function extends the buffer's capacity before moving the data.
Otherwise, the function discards all data that would exceed the buffer's capacity, and both the size and the position are equal to the capacity
(which means, `cxBufferEof()` returns `true` after the operation). 

The functions `cxBufferShiftRight()` and `cxBufferShiftLeft()` accept a larger (but in both cases positive) shift offset,
which usually does not make much sense on a 64-bit platform where `off_t` is already large enough to represent any reasonable offset.
You may, however, still use those function to express more explicitly in your code in which direction you want the contents to be shifted.

## Flushing

```C
#include <cx/buffer.h>

typedef struct cx_buffer_flush_config_s {
    size_t threshold;
    size_t blksize;
    size_t blkmax;
    void *target;
    cx_write_func wfunc;
} CxBufferFlushConfig;

int cxBufferEnableFlushing(CxBuffer *buffer,
        CxBufferFlushConfig config);

size_t cxBufferFlush(CxBuffer *buffer);
```

With the function `cxBufferEnableFlushing()` you can configure a flushing strategy for the contents of the buffer.

Flushing, once enabled, may happen in the following cases:
1. when data is written to the buffer, the capacity is insufficient, and `CX_BUFFER_AUTO_EXTEND` is _not_ enabled
2. when data is written to the buffer, and the required capacity exceeds the `threshold` configuration
3. when `cxBufferFlush()` is called explicitly

> By combining the `CX_BUFFER_AUTO_EXTEND` flag and the `threshold` setting,
> you can create a buffer that initially starts with a small capacity, grows up to a certain threshold,
> and starts flushing only when that threshold is exceeded.

Flushing happens by invoking the `wfunc` up to `blkmax` times, writing up to `blksize` bytes from the buffer to the `target` with each call.
The target might not accept all bytes (i.e. the `wfunc` return value indicates that fewer items have been written than requested),
in which case the remaining data remains in the buffer.
That means, the buffer is effectively [shifted](#shift-contents) left by the number of successfully flushed bytes.

> When you write large amounts of data to a buffer, multiple flush cycles might happen.
> After the first flush operations completed, the reclaimed space in the buffer is filled first, but if that
> is not sufficient, another flush may be triggered within the same invocation of the write operation.
> 
> That means, as much data is written to the buffer and/or flushed as possible, until neither the flush target, nor the buffer accept more data.
>{style="note"}

> The function `cxBufferFlush()` simply returns zero when flushing was not enabled via `cxBufferEnableFlushing()`.

<seealso>
<category ref="apidoc">
<a href="https://ucx.sourceforge.io/api/buffer_8h.html">buffer.h</a>
</category>
</seealso>