Module B0

module B0: sig .. end
Software construction care.

B0 is a set of OCaml libraries and command line tools to configure, build and deploy generic software projects using modular and extensible descriptions written in OCaml. It provides a fully integrated and customizable software construction experience from development to deployment.

See the manual or the build API.

Open the module to use it, it defines only types, modules and result combinators in your scope.

63dfbb8 — homepage



Preliminaries


type 'a result = ('a, [ `Msg of string ]) Pervasives.result 
The type for B0 results of type 'a.
val (>>=) : ('a, 'b) Pervasives.result ->
('a -> ('c, 'b) Pervasives.result) -> ('c, 'b) Pervasives.result
r >>= f is f v if r = Ok v and r otherwise.
val (>>|) : ('a, 'b) Pervasives.result -> ('a -> 'c) -> ('c, 'b) Pervasives.result
r >>| f is f >>= fun v -> Ok (f v).
module R: sig .. end
Result value combinators.
module Fmt: sig .. end
Format combinators.
val strf : ('a, Format.formatter, unit, string) Pervasives.format4 -> 'a
strf is Format.asprintf.
module String: sig .. end
Strings.
module Conv: sig .. end
Textual and type-safe binary value conversions.
module Tty: sig .. end
ANSI terminal interaction.
module Log: sig .. end
The B0 program log.
module Def: sig .. end
Named value definitions.
module Hmap: sig .. end
Type-safe, serializable, heterogeneous value maps.
module Fpath: sig .. end
File paths.
module Cmd: sig .. end
Command lines.
module OS: sig .. end
OS interaction.
module Time: sig .. end
Timing.
module Hash: sig .. end
Hashing.
module Stamp: sig .. end
Freshness stamps.

Builds


type build 
The type for builds. See B0.Build.t.
module Env: sig .. end
Build environment.
module Conf: sig .. end
Build configuration.
module Pkg: sig .. end
Packages.
module Unit: sig .. end
Build units.
module Tool: sig .. end
Build tools.
module Outcome: sig .. end
Build outcomes.
module Cache: sig .. end
Build cache.
module Build: sig .. end
Builds

Organizing and deploying builds

These concepts are not core to the build system they allow to organize multiple, independent builds and their deployment.

module Variant: sig .. end
Build variants.

Codec

FIXME. Move that out of the core build API to driver libraries or b0/d0 or B0_care.

module Sexp: sig .. end
S-expression decoder.
module Json: sig .. end
JSON text encoder.

Manual

Quick start

TODO show a few examples, using B0_ocaml care and the naked system. For now have a look at the examples directory in the distribution.

Conceptual overview

Build model

Effectively a B0 build system is an OCaml program that executes arbitrary external commands in parallel and whose effects on the file system are memoized across program runs with an on-disk cache.

There is no notion of build rule in B0: you simply generate and execute program commands using arbitrary OCaml functions. This allows to define modular and rich data structures for describing builds that are "compiled" down, on each build, to parallel invocations of memoized commands.

Configuration

Next to this simple build model B0 adds a configuration mecanism under the form of a typed, persisted, key-value store which builds can consult and depend on.

Since outputs from previous builds are kept in the cache, build configurations can be switched over and back almost instantaneously without loosing the earlier CPU cycles.

The configuration layer is also cross-compilation ready: any configuration key value can differ for the build and host operating system and the build system of B0 keeps tracks of build tools that are build and used by the build system to make sure they are built with the build OS toolchain. For programmers of the build system, cross compilation is oblivious in B0, it happens without the user having to perform anything special.

More on configuration.

Build units

Build units are statically known and named entities with metadata that gather sets of related build operations. Typical build units are sequences of commands that build a library, an executable, etc. Dependencies can be defined statically among units on dynamically on builds.

Build units structure builds in well identified fragments, allowing to run them independently and perform coarse grained actions on their build outcomes.

Packages

Packages are statically known named entities with metadata that represent a set of build units. They are units of deployment, they allow to build a set of units in isolation from the other and define deployment over them. They have no special functionality expect to structure the build units.

Build variants

The basic build library and model allows build operations to act anywhere and can be used as such. However to structure the builds, the notion of build variant is added on top of that. Build variant allow builds with different configurations to live next to each other or be performed in containers or on remote machines. They define a basic directory layout in which the build occurs and setup the build environment in which the configuration occurs and build tools are looked up.

More on variants.

Deployments

Deployments abstract the general process of extracting part of the sources and/or build artefacts of your software to a new location.

Examples of deployments are: installing build artefacts in a system (FIXME unclear), pushing build artefacts to a remote server or device, making source or binary distribution tarballs and pushing them to a location, interacting with package manager repositories.

More on deployments.

The b0 and d0 tools

The b0 and d0 tool allow to build projects that are described by writing one or more B0.ml OCaml files in a source tree or a composition thereof.

More on description files.

A tour of the _b0 directory

Generally the layout of the build directory is as follows:

The structure of a build variant n is as follows:

The structure of a deployment n is as follows:

Configuration

A configuration is a set of typed key-value bindings consulted by descriptions and build procedures to adjust their outcomes to the build environment and desires of the end user.

A configuration key the user did not explicitely set has a default value, specified in the description at key creation time. This value is either constant or discovered via a function.

A key can belong to at most one group which is simply a named set of related keys. Groups are used to easily select a subset of keys for user interaction. For example on b0 key get command invocations, using the -g ocaml option will report the value of all configuration keys that declared themselves to be part of the ocaml group.

Configuration presets are named sets of key-value bindings defined in descriptions. They are a convenience to set key subsets in bulk in configurations.

Last and stored configuration

The configuration used by the last build is persisted in the build outcome and called the last configuration. It is immutable and contains only the key-value pairs of the configuration that were accessed by the last build. It can be accessed via the b0 key get --last command.

The mutable stored configuration is the configuration to be used by the next build. It can be acted upon via the b0 key get and b0 key set commands.

Key value terminology

A configuration key has different values depending where and in which context it is looked up:

During a build the effective value of keys is looked up using the stored configuration. As a side effect new key-value pairs may be added to the stored configuration for keys whose default value is used and discovered during the build. This modified stored configuration is persisted at the end of the build.

Build variants and variant schemes

A build variant is a build performed in a particular environment with a particular configuration.

The build environment is defined by a variant scheme which is responsible for setting up the environment for the variant. For example this can be: configuring and setting up the environment for an opam switch, spin a container or ssh to a remote machine.

Build variants are identified by a name n which is used to operate on the variant. The build directory of a variant n is isolated from the others in _b0/v/n. Variants are created via:

b0 variant create [SCHEME] [-n NAME]
or implicitely on the first b0 build if there's no existing variant (see The initial variant). If you don't specify a variant name on creation a unique one will be automatically derived from the scheme name. If you don't specify a scheme, the default scheme (likely the nop scheme) will be used.

b0 allows variants to exist and be acted upon side by side, use b0 variant list to list them. Most b0 commands act on the variant specified explicitely via the -w or --variant argument or on the default variant as reported by b0 variant get. If there is no default variant or if it doesn't exist commands might error.

The nop variant scheme

The variant scheme B0.Variant.Scheme.nop available under the name nop is the simplest variant scheme. It does nothing, it runs builds in the environment where the build tool b0 itself is run.

The default variant and variant schemes

The default variant can be consulted, set or cleared via:

b0 variant get [--effective | --stored]
b0 variant set [--clear | VARIANT]
If the B0_VARIANT environment variable is defined, it's value will define the default. The default variant is automatically set to a newly created variant this can be prevented with the -c option:
b0 variant create -c SCHEME  # Do not set the new variant as the default

The initial variant

If no variant exists and there is no default variant when b0 build (or equivalently b0) is run, a variant is created using the default variant scheme. So on a fresh checkout of your project:

b0
automatically creates a variant, set it as the default one and builds your project.

Description values

B0 descriptions are made of a grab bag of OCaml values, configuration keys, build units, packages, variants, variant schemes, deployments, etc. In order to operate on these values from end-user interfaces (e.g. the b0 and d0 tools), the following must be guaranteed:

  1. Values and their names need to be defined during the toplevel initialization phase of the program without being conditioned by external factors B0 may not be aware of (FIXME implement B0.Def locking).
  2. Values names need to be unique to ensure all the values are accessible and can be operated on.

As far as 1. is concerned, B0 relies on the discpline of B0.ml file writers. They should define all their description values through toplevel let definitions and never conditionalize their existence or the definition of their components. For examples this should NOT be done:

let myprogram =
  (* NEVER DO THIS *)
  let default = Conf.const (if Sys.win32 then "tool.exe" else "tool"in
  Conf.(key "myprogram" ~default)
As far as 2. is concerned. B0 handles this automatically. In two manners:

Deployments

Deployements are handled via the d0 tool. They do not necessarily need a build to exist but can request for builds of specific packages to exist. They occur through a sequence of steps, all of which are configurable and made available through deployment schemes.

  1. Pre-stage check and build requirements.
  2. Stage function, prepare deploy artefacts in the deployment staging directory.
  3. Post-stage check.
  4. Pre-push check.
  5. Deployment push, push the staged artefacts.
  6. Post-push check.

Descriptions files

A description file is either:

  1. A B0.b0 file that describes how to compile a description.
  2. A B0.ml OCaml file in a directory without a B0.b0 file.

If your description is simple or uses only the default B0 library then a simple B0.ml description will do. If not, a B0.b0 file is an s-expression based configuration file that describes how to compile a self-contained and isolated description. It can specify additional (and conditional) sources and libraries to use, compilation flags and control how subdescriptions (see below) are looked up.

Root description and directory

b0 supports file hierarchies that contain more than one description file. In general, to ease build setup understanding, it is better to keep a single description per project.

However multiple descriptions allow to merge the description of multiple parallel and interdependent projects into a root description that is built in a root directory.

We first explain formally how an invocation of b0 finds the root directory, examples follow. Given the root directory we can proceed to describe which descriptions belong to the root description.

When started in a directory dir, b0, unless invoked with --root option, finds a root directory for the build as follows:

  1. Starting with dir (included) and moving up in the hierarchy, find the first start directory with a description file (a B0.b0 or B0.ml file). If there is no such directory there is no root directory and no build description.
  2. From start move to the parent directory up and:

Here's an example of a file hierarchy with multiple descriptions:

d
└── root
    ├── B0.b0
    ├── B0.ml
    ├── p1
    │   ├── B0.b0
    │   └── B0.ml
    ├── p2
    │   ├── B0.ml
    │   ├── hop
    │   │   └── B0.ml
    │   └── sub
    │       ├── a
    │       │   └── B0.ml
    │       └── b
    └── src
        ├── bin
        └── lib

In the example above starting a driver in d/root, d/root/src/bin, d/root/p1, d/root/p2/sub/b will all find the root directory d/root. However starting a driver in d/root/p2/sub/a will find the root directory d/root/p2/sub/a as there is no description in root/p2/sub. Adding an empty file d/root/p2/sub/B0.b0 would allow to find d/root.

Given a root directory with (a possibly empty) description, b0 gathers and merge the descriptions files of all direct subdirectories and recursively into the root description. The subs key of B0.b0 files can be used to control which direct subdirectories are looked up. The OCaml sources of different sub descriptions cannot refer to each other directly; they are properly isolated and linked in any (but deterministic) order.

Assuming no B0.b0 file makes use of the subs key in the above example, the root description in root takes into account all descriptions files except d/root/p2/sub/a/B0.ml. Here again adding an empty file d/root/p2/sub/B0.b0 would allow to take it into account.

B0.b0 description files

A B0.b0 description file is a possibly empty sequence of s-expressions of the form (key value). Here's an annoted example:

(b0-version 0)       ; Mandatory, except if the file is empty
(libs (b0_cmdliner)) ; Always compile with the external b0_cmdliner library

; Describe the sources that make up the description in dependency order.
; As a convention if you split your build in many build files put them
; in a B0.d/ directory. If the [srcs] key is absent and a B0.ml file
; exists next to the B0.b0 file it is always automatically added as if
; ("B0.ml" () "B0.ml file") was appended at the end of srcs.
(srcs
  ; If the source path has no suffix looks up both for an .ml and mli file
  ((B0.d/util () "Utility module")

   ; The following source needs the b0_jsoo library and is only added to
   ; the description if the library is found to be installed.
   (B0.d/with_jsoo.ml (b0_jsoo) "Description with jsoo support")))

(compile (-w -23)) ; Disable warning 23 for compiling the description

Key parsing and semantics

An B0.b0 file without keys and without a B0.ml file sitting next to it is an empty and valid description.

If a key is defined more than once, the last one takes over; other than that the key order is irrelevant. Except for keys that start with x-, unknown keys trigger parse warnings.

Relative file paths. Relative file paths are relative to the description file directory.

Library lookup. FIXME. Library lookup is currently quite restricted and done according to the following name mapping:

Dependency resolution on the libraries is not performed and cmi files have to be in the corresponding libname directory.

Key reference

B0.b0 key merges

When multiple B0.b0 file are used, their specification is merged with the root description. During this process the key values of subdescriptions are either:



Recipes and menagerie

Writing conf discovery

Error only if really needed. Otherwise log with warning and default to a reasonable deafult value. Build units can still abort if they can't use the value.