docs/Writerside/topics/streams.h.md

Thu, 06 Feb 2025 20:09:55 +0100

author
Mike Becker <universe@uap-core.de>
date
Thu, 06 Feb 2025 20:09:55 +0100
branch
docs/3.1
changeset 1166
03bbdf402675
parent 1147
52802c36b261
permissions
-rw-r--r--

write introduction for the Utilities Section

relates to #451

1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
1 # Data Streams
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
2
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
3 Stream copy functions provide a way to copy all - or a limited amount of - data from one stream to another.
1166
03bbdf402675 write introduction for the Utilities Section
Mike Becker <universe@uap-core.de>
parents: 1147
diff changeset
4
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
5 Since the read/write functions of a [UCX buffer](buffer.h.md) are fully compatible with stream read/write functions,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
6 you can, for example, easily transfer data from a file or network stream to a UCX buffer or vice versa.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
7
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
8 ## Overview
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
9 ```C
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
10 #include <cx/streams.h>
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
11
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
12 size_t cx_stream_copy(
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
13 void *src, void *dest,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
14 cx_read_func rfnc, cx_write_func wfnc
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
15 );
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
16 size_t cx_stream_ncopy(
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
17 void *src, void *dest,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
18 cx_read_func rfnc, cx_write_func wfnc,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
19 size_t n
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
20 );
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
21
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
22 size_t cx_stream_bcopy(
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
23 void *src, void *dest,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
24 cx_read_func rfnc, cx_write_func wfnc,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
25 char *buf, size_t bufsize
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
26 );
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
27 size_t cx_stream_bncopy(
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
28 void *src, void *dest,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
29 cx_read_func rfnc, cx_write_func wfnc,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
30 char *buf, size_t bufsize,
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
31 size_t n
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
32 );
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
33 ```
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
34
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
35 ## Description
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
36
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
37 All functions in the stream copy family use the `rfnc` to read data from `src`
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
38 and use the `wfnc` to write the data to `dest`.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
39
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
40 The `cx_stream_copy()` function always uses internal stack memory as a temporary buffer for the read bytes.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
41 The `cx_stream_bcopy()` function uses either a pre-initialized buffer `buf` of length `bufsize`
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
42 or, if `buf` is `NULL`, an internal heap-allocated buffer.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
43
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
44 The `cx_stream_ncopy()` function behaves like `cx_stream_copy()` except, that it reads at most `n` bytes
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
45 (and the same is true for `cx_stream_bncopy()` and `cx_stream_bcopy()`).
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
46
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
47
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
48 <warning>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
49 When you are reading from a stream where you cannot track the position, there is the possibility that
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
50 data gets lost when the destination does not accept all the bytes read from the source.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
51 While the stream copy functions do report how many bytes were <emphasis>successfully</emphasis> copied
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
52 to the destination, this might - in certain cases - not be the exact number of read items.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
53
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
54 To mitigate the risk, you should make sure that the destination can always accept all read bytes and
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
55 a possible bottleneck is only introduced by the source.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
56 </warning>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
57
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
58 > The size of the internal _stack_ buffer in `cx_stream_copy()` and `cx_stream_ncopy()` can be
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
59 > set during compilation via the `CX_STREAM_COPY_BUF_SIZE` macro.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
60 > Similarly, the size for the implicitly allocated _heap_ buffer in can be
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
61 > configured via the `CX_STREAM_BCOPY_BUF_SIZE` macro.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
62 > Refer to the [build instructions](install.md#compile-time-options) for the details.
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
63
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
64 ## Example
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
65
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
66 The following example shows, how to read the contents of a file into a buffer:
1141
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
67 ```c
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
68 FILE *inputfile = fopen(infilename, "r");
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
69 if (inputfile) {
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
70 CxBuffer fbuf;
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
71 cxBufferInit(&fbuf, NULL, 4096, NULL, CX_BUFFER_AUTO_EXTEND);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
72 cx_stream_copy(inputfile, &fbuf,
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
73 (cx_read_func) fread,
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
74 cxBufferWriteFunc);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
75 fclose(inputfile);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
76
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
77 // ... do something meaningful with the contents ...
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
78
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
79 cxBufferDestroy(&fbuf);
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
80 } else {
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
81 perror("Error opening input file");
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
82 }
a06a2d27c043 create new page structure
Mike Becker <universe@uap-core.de>
parents:
diff changeset
83 ```
1142
9437530176bc add symbols that need documentation as TODOs
Mike Becker <universe@uap-core.de>
parents: 1141
diff changeset
84
1145
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
85
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
86 <seealso>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
87 <category ref="apidoc">
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
88 <a href="https://ucx.sourceforge.io/api/streams_8h.html">streams.h</a>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
89 </category>
1a8fe7b7dd8a add stream.h docs and reworks hash_key.h docs
Mike Becker <universe@uap-core.de>
parents: 1143
diff changeset
90 </seealso>

mercurial