Fri, 23 May 2025 12:44:24 +0200
make test-compile depend on both static and shared
the shared lib is not needed for the tests,
but when run with coverage, gcov will be confused
when outdated line information is available from
a previous shared build
1145
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
1 | # Data Streams |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
2 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
3 | Stream copy functions provide a way to copy all - or a limited amount of - data from one stream to another. |
1166
03bbdf402675
write introduction for the Utilities Section
Mike Becker <universe@uap-core.de>
parents:
1147
diff
changeset
|
4 | |
1145
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
5 | Since the read/write functions of a [UCX buffer](buffer.h.md) are fully compatible with stream read/write functions, |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
6 | you can, for example, easily transfer data from a file or network stream to a UCX buffer or vice versa. |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
7 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
8 | ## Overview |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
9 | ```C |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
10 | #include <cx/streams.h> |
1141 | 11 | |
1210
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
12 | size_t cx_stream_copy(void *src, void *dest, |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
13 | cx_read_func rfnc, cx_write_func wfnc); |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
14 | |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
15 | size_t cx_stream_ncopy(void *src, void *dest, |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
16 | cx_read_func rfnc, cx_write_func wfnc, size_t n); |
1145
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
17 | |
1210
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
18 | size_t cx_stream_bcopy(void *src, void *dest, |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
19 | cx_read_func rfnc, cx_write_func wfnc, |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
20 | char *buf, size_t bufsize); |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
21 | |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
22 | size_t cx_stream_bncopy(void *src, void *dest, |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
23 | cx_read_func rfnc, cx_write_func wfnc, |
2ad0cf0f314b
complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents:
1166
diff
changeset
|
24 | char *buf, size_t bufsize, size_t n); |
1145
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
25 | ``` |
1141 | 26 | |
1145
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
27 | ## Description |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
28 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
29 | All functions in the stream copy family use the `rfnc` to read data from `src` |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
30 | and use the `wfnc` to write the data to `dest`. |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
31 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
32 | The `cx_stream_copy()` function always uses internal stack memory as a temporary buffer for the read bytes. |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
33 | The `cx_stream_bcopy()` function uses either a pre-initialized buffer `buf` of length `bufsize` |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
34 | or, if `buf` is `NULL`, an internal heap-allocated buffer. |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
35 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
36 | The `cx_stream_ncopy()` function behaves like `cx_stream_copy()` except, that it reads at most `n` bytes |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
37 | (and the same is true for `cx_stream_bncopy()` and `cx_stream_bcopy()`). |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
38 | |
1141 | 39 | |
1242
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
40 | > When you are reading from a stream where you cannot track the position, there is the possibility that |
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
41 | > data gets lost when the destination does not accept all the bytes read from the source. |
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
42 | > While the stream copy functions do report how many bytes were _successfully_ copied |
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
43 | > to the destination, this might - in certain cases - not be the exact number of read items. |
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
44 | > |
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
45 | > To mitigate the risk, you should make sure that the destination can always accept all read bytes and |
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
46 | > a possible bottleneck is only introduced by the source. |
bbb734f73392
downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents:
1210
diff
changeset
|
47 | >{style="note"} |
1145
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
48 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
49 | > The size of the internal _stack_ buffer in `cx_stream_copy()` and `cx_stream_ncopy()` can be |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
50 | > set during compilation via the `CX_STREAM_COPY_BUF_SIZE` macro. |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
51 | > Similarly, the size for the implicitly allocated _heap_ buffer in can be |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
52 | > configured via the `CX_STREAM_BCOPY_BUF_SIZE` macro. |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
53 | > Refer to the [build instructions](install.md#compile-time-options) for the details. |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
54 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
55 | ## Example |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
56 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
57 | The following example shows, how to read the contents of a file into a buffer: |
1141 | 58 | ```c |
59 | FILE *inputfile = fopen(infilename, "r"); | |
60 | if (inputfile) { | |
61 | CxBuffer fbuf; | |
62 | cxBufferInit(&fbuf, NULL, 4096, NULL, CX_BUFFER_AUTO_EXTEND); | |
63 | cx_stream_copy(inputfile, &fbuf, | |
64 | (cx_read_func) fread, | |
65 | cxBufferWriteFunc); | |
66 | fclose(inputfile); | |
67 | ||
68 | // ... do something meaningful with the contents ... | |
69 | ||
70 | cxBufferDestroy(&fbuf); | |
71 | } else { | |
72 | perror("Error opening input file"); | |
73 | } | |
74 | ``` | |
1142
9437530176bc
add symbols that need documentation as TODOs
Mike Becker <universe@uap-core.de>
parents:
1141
diff
changeset
|
75 | |
1145
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
76 | |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
77 | <seealso> |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
78 | <category ref="apidoc"> |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
79 | <a href="https://ucx.sourceforge.io/api/streams_8h.html">streams.h</a> |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
80 | </category> |
1a8fe7b7dd8a
add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents:
1143
diff
changeset
|
81 | </seealso> |