typegist design notes

TODO

• Meta placement. Must be nailed down before any release. See this section.

• For now we just have one dimensional bigarrays as arrays. Should we have the other dimensions, or just Genarray ? Where should we put it from a generic point of view ? Type.Gist.arraylike is decidely 1D. A few alternatives:

  1. One case for Array1 and one case for Genarray. and Type.Gist.Arraylike.to_array_module reshapes Genarray to Array1
  2. Only a Genarray case and Type.Gist.Arraylike.to_array_module reshapes Genarray to Array1
  3. Only Array1 (what we have now) except we add an int array that has the dimensions of a Genarray from which it was reshaped
• Arraylike. Distinguish mutability ? More convenience for construction ?
• Maplike, is possible to have a direct case for Map ? I doubt (no way to capture the module argument Ord as a parameter when writing the type Map.Make(Ord).t). The current scheme requires a functor application to embed a map. It's quite ugly.
• Setlike ?
• Add a standard error mecanism to allow abstract representations to only partially map (just having options is obscure).
• Except for the variants the representation types are kept abstract. This entails however a loss of polymorphism for empty variant cases. In the Gist module itself we can have a polymorphic description of None we can't with Type.Gist.(case "None" @@ ctor None)). Should open the representation of Type.Gist.Field.t ?
• Move field default values to a standard Meta key or keep them ? Do a review once a codec with default values on absence has been written.
• Type.Gist.Rec and Type.Gist.rec' rename to Unlazy and unlazy ?
• Type.Gist.String and Type.Gist.Bytes it's unclear whether we want the char gist.
• Typegist.Fun.Generic.{equal,compare} clarify and document the behaviour on Typegist.Type.Gist.maplike, Typegist.Type.Gist.arraylike and variant cases. The implementation on Hashtbl.t are inefficient, the stdlib doesn't help much.
• Should we add Stdlib.Seq to Typegist.Type.Gist.sum ? Feels a bit odd to mix in corecursion in there
• Representation of variants. The internal data structure is a bit too naive. Both Fun.Generic.compare and Typegist.Fun.Generic.random are quite inefficient.
• Add optional parameters to easily specify Meta.Doc ?
• The semantics of Typegist.Type.Gist.Product.name depends on the context. Should we split name into case_name and type_name
• Clarify the meta of fields. Note all other metas are tied to types and their values. Fields are not types.

Meta placement

Difficult to find the right design. Basically we can have it everywhere like now, or only in the toplevel cases or in a single case at the toplevel. This may lead to different attachement interpretations though.

Tried to simply shift them in every toplevel case and let, for example, the gist of a field define the meta of the field but it is problematic with a field whose gist is a recursion step (at a meta to the rec step ?). Also our intepretation in the pretty printer example is different. The meta for the field redefines the whole field printing. While the pretty printer of the field's gist redefines the value. Is it important for other processors ?

In general it looks easier if we don't have to redefine the meta of existing gists to be able to specify metadata at a given point (e.g. the metadata of fields). The drawback is that processors need to document a bit more precisely which meta is relevant for them and we get more lookups (the field's meta and its gist).

Also tried to have meta in the parent case for cases which are variants (Type.Gist.Scalar, Type.Gist.Scalar. etc.) but in the end I shifted them back into the subvariants themselves despite the types looking heavier (Typegist.Type.Gist.scalar, Typegist.Type.Gist.sum, etc.). It makes the types meta-aware without needing to carry something on the side. The drawback is that generic meta lookup on a gist is likely slower.

The current scheme ends up making a lots of lookups. For this having only meta as a toplevel case is likely better. It also makes the AST more lightweight. Given the current generic function examples, except maybe for the pretty printer, that would be entirely sufficient. But once we get to for example, type-directed uis. things may look a bit different, especially at the field level. It also means that generic functions need to thread and handle the scope of meta, that looks like a bad idea.