Module B00.Memo

Build memoizer.

A memoizer ties together and environment, an operation cache, a guard and an executor.

Memoizer

type feedback = [
| `Fiber_exn of exn * Stdlib.Printexc.raw_backtrace
| `Fiber_fail of string
| `Miss_tool of Tool.t * string
| `Op_cache_error of Op.t * string
| `Op_complete of Op.t * [ `Did_not_write of B0_std.Fpath.t list ]
]

The type for memoizer feedback.

type t

The type for memoizers. This ties together an environment, a aguard, an operation cache and an executor.

val create : ?⁠clock:B0_std.Time.counter -> ?⁠cpu_clock:B0_std.Time.cpu_counter -> feedback:(feedback -> unit) -> cwd:B0_std.Fpath.t -> Env.t -> Guard.t -> Reviver.t -> Exec.t -> t
val memo : ?⁠hash_fun:(module B0_std.Hash.T) -> ?⁠env:B0_std.Os.Env.t -> ?⁠cwd:B0_std.Fpath.t -> ?⁠cache_dir:B0_std.Fpath.t -> ?⁠trash_dir:B0_std.Fpath.t -> ?⁠max_spawn:int -> ?⁠feedback:([ feedback | File_cache.feedback | Exec.feedback ] -> unit) -> unit -> (t, string) Stdlib.result

memo is a simpler create

  • hash_fun defaults to Op_cache.create's default.
  • max_spawn defaults to Exec.create's default.
  • env defaults to Os.Env.current
  • cwd defaults to Os.Dir.cwd
  • cache_dir defaults to Fpath.(cwd / "_b0" / ".cache")
  • trash_dir defaults to Fpath.(cwd / "_b0" / ".trash")
  • feedback defaults formats feedback on stdout.
val clock : t -> B0_std.Time.counter

clock m is m's clock.

val cpu_clock : t -> B0_std.Time.cpu_counter

cpu_clock m is m's cpu clock.

val env : t -> Env.t

env m is m's environment.

val reviver : t -> Reviver.t

reviver m is m's reviver.

val guard : t -> Guard.t

guard m is m's guard.

val exec : t -> Exec.t

exec m is m's executors.

val trash : t -> Trash.t

trash m is m's trash.

val hash_string : t -> string -> B0_std.Hash.t

hash_string m s is Reviver.hash_string (reviver m) s.

val hash_file : t -> B0_std.Fpath.t -> (B0_std.Hash.t, string) Stdlib.result

hash_file m f is Reviver.hash_file (reviver m) f. Note that these file hashes operations are memoized.

val stir : block:bool -> t -> unit

stir ~block m runs the memoizer a bit. If block is true blocks until the memoizer is stuck with no operation to perform.

val finish : t -> (unit, B0_std.Fpath.Set.t) Stdlib.result

finish m finishes the memoizer and deletes the trash. This blocks until there are no operation to execute like stir does. If no operations are left waiting this returns Ok (). If there are remaining waiting operations it aborts them and returns Error fs with fs the files that never became ready and where not supposed to be written by the waiting operations.

val delete_trash : block:bool -> t -> (unit, string) Stdlib.result

delete_trash ~block m is Trash.delete trash m.

val ops : t -> Op.t list

ops m is the list of operations that were submitted to the memoizer

Operation groups

val group : t -> string

group m is m's group.

val with_group : t -> string -> t

group m g is m but operations performed on m have group g.

Fibers

type 'a fiber = ('a -> unit) -> unit

The type for memoizer operation fibers.

val fail : ('b, Stdlib.Format.formatter, unit, 'a) Stdlib.format4 -> 'b

fail fmt ... fails the fiber with the given error message.

val fail_error : ('a, string) Stdlib.result -> 'a

fail_error fails the fiber with the given error.

Files and directories

val file_ready : t -> B0_std.Fpath.t -> unit

ready m p declares path p to be ready, that is exists and is up-to-date in b. This is typically used with source files and files external to the build (e.g. installed libraries).

val wait_files : t -> B0_std.Fpath.t list -> unit fiber

wait_files m files k continues with k () when files become ready. FIXME Unclear whether we really want this though this is kind of a reads constraint for a pure OCaml operation, but then we got read.

val read : t -> B0_std.Fpath.t -> string fiber

read m file k reads the contents of file file as s when it becomes ready and continues with k s.

val write : t -> ?⁠stamp:string -> ?⁠reads:B0_std.Fpath.t list -> ?⁠mode:int -> B0_std.Fpath.t -> (unit -> (string, string) Stdlib.result) -> unit

write m ~reads file w writes file with data w () and mode mode (defaults to 0o644) when reads are ready. w's result must only depend on reads and stamp (defaults to "").

val copy : t -> ?⁠mode:int -> ?⁠linenum:int -> src:B0_std.Fpath.t -> B0_std.Fpath.t -> unit

copy m ~mode ?linenum ~src dst copies file src to dst with mode mode (defaults to 0o644) when src is ready. If linenum is specified, the following line number directive is prependend in dst to the contents of src:

#line $(linenum) "$(src)"
val mkdir : t -> B0_std.Fpath.t -> bool fiber

mkdir m dir k creates directory dir and continues with k created at which point file dir is ready and created indicates if the directory was created by the operation.

val delete : t -> B0_std.Fpath.t -> unit fiber

delete m p deletes (trashes in fact) path p and continues with k () when the path p is free to use.

Memoizing tool spawns

type tool

The type for memoized tools.

type cmd

The type for memoized tool invocations.

val tool : t -> Tool.t -> B0_std.Cmd.t -> cmd

tool m t is tool t memoized. Use the resulting function to spawn the tool with the given arguments.

val tool_opt : t -> Tool.t -> (B0_std.Cmd.t -> cmd) option

tool_opt m t is like tool, except None is returned if the tool cannot be found.

val spawn : t -> ?⁠stamp:string -> ?⁠reads:B0_std.Fpath.t list -> ?⁠writes:B0_std.Fpath.t list -> ?⁠env:B0_std.Os.Env.t -> ?⁠cwd:B0_std.Fpath.t -> ?⁠stdin:B0_std.Fpath.t -> ?⁠stdout:Op.Spawn.stdo -> ?⁠stderr:Op.Spawn.stdo -> ?⁠success_exits:Op.Spawn.success_exits -> ?⁠k:(int -> unit) -> cmd -> unit

spawn m ~reads ~writes ~env ~cwd ~stdin ~stdout ~stderr ~success_exits cmd spawns cmd once reads files are ready and makes files writes ready if the spawn succeeds and the file exists. The rest of the arguments are:

  • stdin reads input from the given file. If unspecified reads from the standard input of the program running the build. Warning. The file is not automatically added to reads, this allows for example to use Os.File.null.
  • stdout and stderr, the redirections for the standard outputs of the command, see stdo. Path to files are created if needed. Warning. File redirections are not automatically added to writes; this allows for example to use Os.File.null.
  • success_exits the exit codes that determine if the build operation is successful (defaults to 0, use [] to always succeed)
  • env, environment variables added to the build environment. This overrides environment variables read by the tool in the build environment except for forced one. It also allows to specify environment that may not be mentioned by the running tool's environment specification.
  • cwd the current working directory. Default is cwd. In general it's better to avoid using relative file paths and tweaking the cwd. Construct your paths using the absolute directory functions and make your invocations independent from the cwd.
  • k, if specified a fiber invoked once the spawn has succesfully executed with the exit code.
  • stamp is used for caching if two spawns diff only in their stamp they will cache to different keys. This can be used to memoize tool whose outputs may not entirely depend on the environment, the cli stamp and the the content of read files.

TODO. More expressive power could by added by:

  1. Support to refine the read and write set after the operation returns.

Note. If the tool spawn acts on a sort of "main" file (e.g. a source file) it should be specified as the first element of reads, this is interpreted specially by certain build tracer.

Future values

module Fut : sig ... end

Future values.