docs/Writerside/topics/streams.h.md

Fri, 23 May 2025 12:44:24 +0200

author
Mike Becker <universe@uap-core.de>
date
Fri, 23 May 2025 12:44:24 +0200
changeset 1327
ed75dc1db503
parent 1242
bbb734f73392
permissions
-rw-r--r--

make test-compile depend on both static and shared

the shared lib is not needed for the tests,
but when run with coverage, gcov will be confused
when outdated line information is available from
a previous shared build

1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
1 # Data Streams
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
2
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
3 Stream copy functions provide a way to copy all - or a limited amount of - data from one stream to another.
1166
03bbdf402675 write introduction for the Utilities Section
Mike Becker <universe@uap-core.de>
parents: 1147
diff changeset
4
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
5 Since the read/write functions of a [UCX buffer](buffer.h.md) are fully compatible with stream read/write functions,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
6 you can, for example, easily transfer data from a file or network stream to a UCX buffer or vice versa.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
7
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
8 ## Overview
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
9 ```C
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
10 #include <cx/streams.h>
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
11
1210
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
12 size_t cx_stream_copy(void *src, void *dest,
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
13 cx_read_func rfnc, cx_write_func wfnc);
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
14
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
15 size_t cx_stream_ncopy(void *src, void *dest,
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
16 cx_read_func rfnc, cx_write_func wfnc, size_t n);
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
17
1210
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
18 size_t cx_stream_bcopy(void *src, void *dest,
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
19 cx_read_func rfnc, cx_write_func wfnc,
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
20 char *buf, size_t bufsize);
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
21
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
22 size_t cx_stream_bncopy(void *src, void *dest,
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
23 cx_read_func rfnc, cx_write_func wfnc,
2ad0cf0f314b complete the printf documentation and fix code formatting
Mike Becker <universe@uap-core.de>
parents: 1166
diff changeset
24 char *buf, size_t bufsize, size_t n);
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
25 ```
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
26
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
27 ## Description
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
28
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
29 All functions in the stream copy family use the `rfnc` to read data from `src`
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
30 and use the `wfnc` to write the data to `dest`.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
31
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
32 The `cx_stream_copy()` function always uses internal stack memory as a temporary buffer for the read bytes.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
33 The `cx_stream_bcopy()` function uses either a pre-initialized buffer `buf` of length `bufsize`
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
34 or, if `buf` is `NULL`, an internal heap-allocated buffer.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
35
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
36 The `cx_stream_ncopy()` function behaves like `cx_stream_copy()` except, that it reads at most `n` bytes
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
37 (and the same is true for `cx_stream_bncopy()` and `cx_stream_bcopy()`).
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
38
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
39
1242
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
40 > When you are reading from a stream where you cannot track the position, there is the possibility that
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
41 > data gets lost when the destination does not accept all the bytes read from the source.
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
42 > While the stream copy functions do report how many bytes were _successfully_ copied
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
43 > to the destination, this might - in certain cases - not be the exact number of read items.
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
44 >
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
45 > To mitigate the risk, you should make sure that the destination can always accept all read bytes and
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
46 > a possible bottleneck is only introduced by the source.
bbb734f73392 downgrade info regarding stream_copy from warning to note
Mike Becker <universe@uap-core.de>
parents: 1210
diff changeset
47 >{style="note"}
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
48
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
49 > The size of the internal _stack_ buffer in `cx_stream_copy()` and `cx_stream_ncopy()` can be
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
50 > set during compilation via the `CX_STREAM_COPY_BUF_SIZE` macro.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
51 > Similarly, the size for the implicitly allocated _heap_ buffer in can be
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
52 > configured via the `CX_STREAM_BCOPY_BUF_SIZE` macro.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
53 > Refer to the [build instructions](install.md#compile-time-options) for the details.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
54
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
55 ## Example
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
56
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
57 The following example shows, how to read the contents of a file into a buffer:
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
58 ```c
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
59 FILE *inputfile = fopen(infilename, "r");
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
60 if (inputfile) {
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
61 CxBuffer fbuf;
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
62 cxBufferInit(&fbuf, NULL, 4096, NULL, CX_BUFFER_AUTO_EXTEND);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
63 cx_stream_copy(inputfile, &fbuf,
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
64 (cx_read_func) fread,
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
65 cxBufferWriteFunc);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
66 fclose(inputfile);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
67
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
68 // ... do something meaningful with the contents ...
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
69
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
70 cxBufferDestroy(&fbuf);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
71 } else {
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
72 perror("Error opening input file");
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
73 }
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
74 ```
1142
9437530176bc add symbols that need documentation as TODOs
Mike Becker <universe@uap-core.de>
parents: 1141
diff changeset
75
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
76
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
77 <seealso>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
78 <category ref="apidoc">
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
79 <a href="https://ucx.sourceforge.io/api/streams_8h.html">streams.h</a>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
80 </category>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
81 </seealso>

mercurial