# Design and Architecture

This library is relatively large and complex client side application, and is
only made possible to develop and maintain by adhering to certain design
principles and architectural guidelines. This document discusses the design of
the tables and why these principles were chosen.

## Overview

The application is composed of a number of interacting sub-units with their own
local state and interfaces (eg. cells, headers, buttons, dialogues, etc). These
are developed as separate units of code in commonjs modules and composed into a
single application using Browserify. Browserify was chosen as it is an excellent
choice for breaking down a large application into small, digestible chunks, and
it is superior to Require.js (its only real competitor) in allowing the code to
be further consumed by other parties. It was an important design consideration
that the various sub-units of this application could be re-used separately in
other contexts (eg. the item previews, or the export dialogue). Browserify makes
this easy. The application structure is heavily influenced by Backbone, upon
which it is built.

The code is organised into the following logical divisions:
  * Models: Abstractions defining the state of components. These live in `src/models`
  * Views: Controllers binding UI actions to model behaviour. These live in
    `src/views`. These components also explicitly define the sub-component
    structure of the application, forming a tree of components.
  * Templates: Functions producing HTML markup from data. These live in
    `templates`. Views are responsible for extracting data from models and
    passing it to templates to generate HTML.
  * Utilities: General logic that may be used in multiple places. These live in
    `src/utils`.
  * Messages: Text strings. Separate from templates (which describe DOM
    structure), messages are a mechanism for getting human readable text
    strings. These are designed to be configurable with customisation and
    internationalisation in mind. These are located in `src/messages` and can be
    set at run-time after the application is loaded.
  * Options: A central configuration store, accessible to all components and
    available at run-time to the outside world. Located at `src/options`.

Where possible the code is meant to be as declarative as possible. This means
that readers of the code should be able to see what the code means just from
looking at the code, rather than holding the whole system in their heads - ie.
we are writing for humans and our own sanity and not just for the browser. Some
decisions that flow from this include:
  * The views are the canonical place to look to see how things fit together -
    their job is to bind state to appearance and actions to behaviour so they
    are where these connections are defined. As such the views should name the
    models they expect (in the `::Model` property of the class), the template
    that renders that model (in the `::template` property) and cleanly show what
    transformations are performed (in the `::getData` method).
  * DOM events should never cause a component to manipulate the DOM directly in
    the event handler. DOM events should affect model state, which can, in
    separate handlers, cause DOM manipulation if so required. The flow should be
    `DOM Event` -> `State` -> `State Event` -> `DOM manipulation`. This
    indirection has several benefits: it makes it easier to see what is going on
    (each direction of the binding from DOM <-> State has its own handler), and
    it means that some DOM manipulation can be avoided (if it does not change
    the state).
  * Localising as much of the DOM manipulation in the template is to be
    preferred, ie. prefer to re-render on any state changes rather than
    twiddling the DOM. Rendering even a moderately sized template is fast, and the best
    code structure is to do all State -> DOM binding in the template and fully
    re-render when anything relevant changes. This produces minimal classes
    (with just model, template, getData projection and event binding
    definitions) that are very comprehensible and means things can never get out
    of synch (if you have DOM twiddling you have to be sure to call that
    twiddler on every initial render and re-render and make sure it is
    consistent with the template state). Reasons to do twiddling include if it
    would overwrite a form input (which can be annoying and frustrate the user)
    or in tight loops (eg. the cells do twiddling to avoid re-rendering > 100
    cells on every small change).
  * The flow of concerns means that templates should only deal with presentation
    (never state) and models should only deal with state (never presentation) -
    ie. strong separation of concerns. The views are the place where these two
    things are composed. Hence it follows that models can never refer to views
    and are pure data-structures that never need access to a DOM to run. Models
    should be fully testable in node. Templates should be pure projections of
    data and never encode layers of data for reading out of the DOM (eg. setting
    data attributes so that handlers can access them). Data attributes are a
    major code smell and indicate that another child-component level is needed
    to bind that linkage of state and behaviour.

To make this easier some structure is provided to make life easier:

## Core-View

All views in the application extend `Backbone.View` through a subclass called
`CoreView` (located at `src/core-view`). This provides several important helpers
and lays out a component life-cycle (in this sense it is not dissimilar from
Marionette, but more specilised for this use case).

### Public API

The API for consuming a view is the standard Backbone one - all views are
instantiated with `new` and an options object (ie. all view constructors take a
single argument). Once constructed the method `render` must be called and the
component must be inserted into the DOM. eg:

```js
var view = new Component(opts);
document.querySelector(x).appendChild(view.render().el);
```

`CoreView::render` always returns `this` and never throws.

### Parameters and initialization

Views may (and *should*) specify their parameters through with the `parameters`
property. This means that those reading your class can immediately and
consistenly see how to instantiate the component. Parameters listed in this way
are required, and failing to provide them, or passing a `null` value will cause
an error to be thrown on initialisation. Optional parameters can be listed in
`::optionalParameters`, which may be omitted or set to `null` - which will be
ignored.

Views can (and *should*) define assertions that check that their parameters meet
their expectations as soon as possible. These are termed invariants and there
are two ways of defining them. One is to associate each parameter with an
assertion about its type (see `core/type-assertions`) which is an excellent way
to document your assumptions about your code. More complex assertions may be
provided using the invariants system, eg:

```js
  // We must be provided with a model and a foo.
  parameters: ['model', 'foo'],

  // We require that foo is a Foo, and the model is a model
  parameterTypes: {
    model: types.Model,
    foo: types.InstanceOf(Foo, 'Foo')
  },

  // ... some class

  // mapping from tests to failure messages
  invariants: function () {
    isAdult: "User is not old enough, is only " + this.model.get('age')
  },

  // Each test is a method that returns true if acceptable.
  isAdult: function () {
    return this.model.get('age') >= 18;
  }
```

If no further initialization is required beyond capturing the parameters then
there is no need to override `initialize`. Authors should seek to avoid
overriding `::initialize` where possible, and where they must, then *always*
call super - if `super` is not called then things like parameter setting, model
construction and invariant checking will not be done and things *will* break.
You have been warned.
 
### Model vs. State

Components can have multiple models/collections. An example of this is all
components that receive an `imjs.Query` object as a parameter, which can
function as a model. In addition to whatever they are provided with, all views
are guaranteed to have a `::model` and a `::state` property. These are both
`Backbone.Model` instances, but they are intended to differ in the following ways:

* `model` can be passed as a parameter, whereas each component receives a
  state automatically.
* The `model` is meant to be the canonical representation of the data, whereas
  the state is meant to be the transient and derivative projection of that
  data.
* Models are expected to be shared and passed around, state is meant to be
  private. If state is passed to sub-components it should be the model of the
  sub-component.
* Models are meant to have a defined shape (each view should have an
  appropriate Model class) with specific properties that are
  always present. State is more free-form and can have any properties required.
* In the template the model is exploded (eg. `model.foo` is accessed as `foo`)
  and the state is accessed via `state` (eg. `state.foo` is accessed as
  `state.foo`).

For example, some components receive a model identifying them, and a collection
identifying a set of active components. The logic for these components is:

1. On initialisation the component checks if its model is in the active set,
   and if so sets its `state.active` to `true`.
2. When rendering, the template reads from `state.active`.
3. When the user activates the component the component adds the model to the
   active set.
4. When the user deactivates the component the component removes the model
   from the active set.
5. When the active set changes the component checks if the model is in the
   active set and sets the `state.active` property accordingly.
6. When the `state.active` property changes, the component re-renders (go to
   2.)

This system provides a way for keeping states in synch while maintaining simple
projections from DOM to state and back. It also minimises re-renderings, since
we don't have to re-render all components that share the same active set, just
the ones whose membership changes.

### Event Sets

To keep everything nice and declarative components can define event sets (like
the main `events` property of a basic `Backbone.View`, which handles DOM
events) for other targets, the default ones being `modelEvents`, `stateEvents`
and `collectionEvents`, which listen to events on `::model`, `::state` and
`::collection` respectively. Not every view will have a collection, but if
`collectionEvents` is defined, then it will be checked to exist. These can be
thought of as the other direction of binding: `events` binds from DOM -> Model,
and the other events bind from state -> DOM. The most important action is
`CoreView::reRender`, which calls `render` only if `render` has already been
called. It is thus safe to initialise the state to values we are listening to
since no render will be triggered until the component is mounted.

### The Minimal CoreView

A nice clean component is easy to tell at a glance - it should be short, with
obvious methods and declarative bindings and no calls to super. A simple view
could look something like this:

```js
var SimpleView = CoreView.extend({

    // a model that has .favourite :: bool and .likes :: int attributes
    Model: SimpleViewModel, // see CoreModel below info on swap and toggle

    template: Templates.template('simple-view'),

    // DOM -> State bindings
    events: function () {
        'change .favourite': function (e) { this.model.toggle('favourite'); },
        'click .like': function (e) { this.model.swap('likes', increment); }
    },

    // State -> DOM bindings
    modelEvents: function () {
        'change:favourite change:likes': this.reRender
    }
});

// Helper that increments a value.
function increment (n) { return n + 1; }
```

### Child Management

An important part of components is child management. This is handled by the
following methods:

1.  `CoreView::renderChild(id, child, container = this.el)`:

    Renders a child component, appending it to the container (or `this.el` if
    not provided) and storing a reference to it against the id. `child` should
    be an instance of a `CoreView`.

2.  `CoreView::removeChild(id)`:

    Removes the child by the given id (if it exists). Does nothing if the child
    doesn't exist. This is always called by `renderChild`, and it is called for
    every existing child during `remove`, thus removing all trees of
    sub-children.

3. `CoreView::renderChildAt(selector, child)`:

   Specialisation of `renderChild` - renders a child and sets its element to
   that of the value of `this.$(selector)`.

These methods should be called in `renderChildren` or `postRender` which are the
parts of the life-cycle when the template has rendered the DOM of the current
component. The child can then be accessed at `::children[id]`, although that
should seldom be necessary. An important implication of this is that
child-components should be prepared to be torn-down and rebuilt as frequently as
their parents are re-rendered, and that the higher up the tree one is, the more
stable one will be. Or conversely, re-rendering a component with lots of
sub-components can get expensive.

Best practices are to isolate re-rendering. If a change in state affects that
heading in a component, but not the body, then the header and body should be
separate child components that have separate rendering cycles, even if they
share the same model.

## CoreModel

Analogous to the the CoreView there is a CoreModel extension to
Backbone.Model. This provides some helpful extensions to the `Backbone.Model`
API, and more importantly it has sensible clean up in `#destroy` that is
appropriate for non-REST resource backed models.

The extensions are detailed below:

* `CoreModel#swap(name, transformation)` - Replace an attribute with the
  result of invoking a transformation on its current value. eg:
  
```coffee
increment = (x) -> x + 1
model.swap 'clicks', increment
```

* `CoreModel#toggle(name)` - a specialisation of swap; toggle a boolean value.

## Messages

An internationalisation and customisation interface which abstracts all human
language text strings behind a set of getters. A single instance of the text
string store is made available via a call to `require('imtables/messages')` and
it is injected into the templates by default (see `CoreView#getBaseData` for
details), where it is available bound to the name `Messages`.

This mechanism uses the text-replacement-as-function system where text strings
are specified as function or template strings which are expected to return
*unescaped* text. Some messages take parameters - they should be obvious by
inspecting the default messages provided.

Messages are logically divided into bundles, prefixed with some name-spacing
identifier which should indicate the component they belong to. E.g. the columns
manager loads the bundle from `src/messages/columns` which loads messages
prefixed with `columns.` into the message store. This is designed to make it
clear where to look to edit messages, without having to have one enormous
message file. Views that need a message bundle are expected to load it via a
`require` call. Related families of views that all use the same bundle (the
column manager, the list dialogue, etc etc) should ideally all load their bundle
individually - this has two advantages: (a) it makes testing easier if we need to
isolate components in unit tests and (b) it re-inforces the nature of the view
as the declaration of dependencies - looking at the source file will tell the
developer where the relevant resources are.

The ability to set messages is exported as part of the public API of imtables.

## Options

TODO

## Formatters

TODO

## Styling

The style for the tables is also modular, and is written in less to enable
this. The main entry point is `less/main.less` which defines which modules are
loaded. Εach module is expected to be named in such a way as to reflect the
component it styles. Where possible values such as colours, widths, etc should
be abstracted as variables to be kept in `less/variables.less` so that they can
be re-used and configured.

The style for imtables is based on Bootstrap 3.2.x, with a number of
customisations (although it is still recognisably bootstrappy). This was chosen
as it is a very mature and well supported UI style system, and many sites use
it. If you use Bootstrap on your site it is easy to configure imtables to
integrate perfectly into your visual identity.

There are two strategies for developing a custom style sheet to make the tables
integrate better into your site:

1. Produce a custom imtables style sheet:

   We provide a script which is installed with imtables
   (`create-imtables-style`). This script takes as its argument the path
   (absolute or relative) of an extension file and writes a new style-sheet with
   these extensions and overrides. An extension file is just a regular less
   style-sheet; normally all you will need to do is change variable values
   (colours, border radii, etc), but it can be as complex as you like.

2. Consume imtables as a component within your style sheet.

   If you are using Bootstrap then you should be using less to produce your
   style sheets (you should be using less anyway, frankly), and if you do that
   then you can consume imtables as a module just as you would any other system.
   Just add `node_modules` to your less search path and then load imtables
   styles - eg:

```less
@import "imtables/main.less"; // Import everything
@import "./my-imtables-overrides.less"; // Your customisations
```

## Questions

### Why not just use DataTables?

I tried. Hard. The first version of this code was built on DataTables, but I had
to tear it down and start again. DataTables has a number of limitations
including:
  * You can't render complex cells easily, and we needed things like subtables,
    and cells with their own state, and formatting logic and cells that replace
    other cells, etc.
  * Same goes for headers - and this is very tricky as we needed to dynamically
    change the header definition depending on the formatters loaded and the
    outer-join status. Also headers have lots of sub-components such as the
    column summaries.
  * The event system is very limited, and we needed fine grained access to
    things like cell selection.
  * While it has improved massively, styling DataTables used to be very
    challenging.
  * It is fundamentally tied in to a global jQuery which makes it a rather poor
    candidate for embedding on other pages, which was a primary design goal.

Do keep re-investigating this regularly. The best code is the code we don't have
to write or maintain, so if we can delete half this repo that would be great.
