Bytesrw v0.2.0-15-g33db5e3

Manuals

The following manuals are available:

• The quick start should do so.
• The tutorial is a conceptual overview of byte stream readers and writers.
• The Bytesrw cookbook has a few conventions and byte stream recipes.
• The design notes explains design choices made by the library.

Library bytesrw

This library has the base definition of byte stream reader and writers as an extension of the Stdlib.Bytes module.

• Bytesrw Extended Stdlib.Bytes module. Open to use it.

• Bytesrw.Bytes.Slice Byte slices.
• Bytesrw.Bytes.Stream Byte streams.
• Bytesrw.Bytes.Reader Byte stream readers.
• Bytesrw.Bytes.Writer Byte stream writers.

The following modules rely only on the Stdlib:

• Bytesrw_utf UTF streams.
• Bytesrw_hex Hexadecimal tools.

Libraries bytesrw.{blake3,crypto,md,tls,unix,xxhash,zlib,zstd}

Each of these modules lives in its corresponding library. Compression and hashing libraries depend on their canonical C library and encryption on Mbed TLS. If a library is missing, install the corresponding conf opam package and reinstall bytesrw.

• Bytesrw_blake3 BLAKE3 hashes (via conf-libblake3)
• Bytesrw_crypto Cryptographic primitives and streams (via conf-mbedtls)
• Bytesrw_md SHA-1 and SHA-2 hashes (via conf-libmd)
• Bytesrw_sysrandom Cryptographically secure pseudorandom byte streams and entropy.
• Bytesrw_tls TLS encrypted streams (via conf-mbedtls)
• Bytesrw_unix Blocking Unix file descriptor byte stream readers and writers.
• Bytesrw_xxhash XXH3-64 and XXH3-128 hashes (via conf-xxhash)
• Bytesrw_zlib deflate, zlib and gzip streams (via conf-zlib)
• Bytesrw_zstd zstd streams (via conf-zstd)

Quick start

This example compresses standard input to standard output with zstd using either a compressing byte stream reader (pull) or a compressing byte stream writer (push).

cat << 'EOF' > quickstart.ml
open Bytesrw

let stdio_compress_reads () =
  try
    let stdin = Bytes.Reader.of_in_channel In_channel.stdin in
    let stdout = Bytes.Writer.of_out_channel Out_channel.stdout in
    let params = Bytesrw_zstd.Cctx_params.make ~checksum:true () in
    let zstdr = Bytesrw_zstd.compress_reads ~params () stdin in
    Bytes.Writer.write_reader ~eod:true stdout zstdr;
    Ok ()
  with
  | Bytes.Stream.Error e -> Bytes.Stream.error_to_result e
  | Sys_error e -> Error e

let stdio_compress_writes () =
  try
    let stdin = Bytes.Reader.of_in_channel In_channel.stdin in
    let stdout = Bytes.Writer.of_out_channel Out_channel.stdout in
    let params = Bytesrw_zstd.Cctx_params.make ~checksum:true () in
    let zstdw = Bytesrw_zstd.compress_writes ~params () ~eod:true stdout in
    Bytes.Writer.write_reader ~eod:true zstdw stdin;
    Ok ()
  with
  | Bytes.Stream.Error e -> Bytes.Stream.error_to_result e
  | Sys_error e -> Error e

let main () =
  Result.fold ~ok:(Fun.const 0) ~error:(fun e -> prerr_endline e; 1) @@
  if Array.exists (String.equal "-w") Sys.argv
  then stdio_compress_writes ()
  else stdio_compress_reads ()

let () = if !Sys.interactive then () else exit (main ())
EOF

It can be compiled and used with:

ocamlfind ocamlopt -package bytesrw,bytesrw.zstd -linkpkg quickstart.ml
./a.out < quickstart.ml | zstd -d

The tutorial is a short conceptual overview of byte stream readers and writers. The cookbook has a few more tips and code snippets.