Module Evidence

TODO

• Maybe we should rather first have a lightweight DSL of typed projections. And a Var is just a named typed projection. With respect to this the O module DSL may not be such a good idea since we switch to polycompare.
• na. Add a per variable specific na value that is not considered ? Seems footgunish, this will only be considered by DSL functions. However it's nice to have a notion of na to suppress outliers without dropping the whole observation. Option looks painful. On ints. We could use min_int. Mabye not a good idea… We should also define a consistent strategy for the treatment of nan values, is it really a good idea to silently ignore them ?
• Tools or nan handling/discovering.
• Var.type'.Ordinal, rather have 'a -> int and consider them as is_numeric ? Not convenient from an OCaml perspective.
• Var.Type provide a case to allow to inject arbitrary arithmetic types via first class modules ?
• Summary statistics, rather return Invalid_argument on non numeric types ?
• Structural changes remain painful as they invalidate projectors. How much is it needed during analysis ? Is an untyped scheme (TBD) for tidying enough ?
• Var.Type.value_set and Var.Type.value_map without the polymorphic constraint on t these are only useful locally. Maybe remove.
• What about adding columns via a side hetergenous keymap in Dataset.t.
• Dataset make optional obs arguments required ?
• Dataset.Var.count return the typed variables with the result ?
• What about simplifying to Float, Nominal, Any ?
• Not sure about the whole thing, maybe go back to first reason on `Dataset.Var.* functions on a simple sequence.
• Dataset except for obs, does it make sense to exist if we have an immutable sequence, e.g. Pvec. There is the non empty invariant, what happens if we drop it ?
• Are we not just seeking a DSL for projections and order definition ?
• Negative thoughts. Negative indices are bad, magnitudes are not ? That Remains to be shown. Maybe negativity is hard to comprehend once it becomes symbolic (vs. literal constants).
• show do terminal plots ?
• Dataset.Var.group ~by:Var.t -> with values (facet), single dim. Can we generalize ? Variable for a subset of projections out of a sequence of var via Obs.o2, Obs.o3…
• Obs.set, raise Invalid_argument if var is not part of the product.
• Dataset.Csv.decode, fix limitations. Allow obs rebind to variables of csv, implement schema sniffing.
• Think on how to best support time and date time. Likely via Ordinal but maybe built-in a few goodies.