# cypherfmt - architecture/technical overview
This document provides some context on how the formatter contained in this directory works. It does not have
an explicit name, but we prefer `cypherfmt` to be consistent with the naming of other formatters, and to indicate
that it is the "official one" (by default, since no good alternatives exist).

The formatter was written as a thesis project, and the thesis describing it can be found here:
- [An Evaluation of Approaches to Code Formatting](https://lup.lub.lu.se/student-papers/search/publication/9188816)

The thesis project covers some academic elements that are not directly relevant, but understanding the context of how this formatter was developed
might be of interest. It also covers some interesting details about styling preferences and technical details that will
not be covered here, such as the styling survey we conducted.

The formatting algorithm is heavily inspired by the following paper on Pretty printing:
- [Prettier paper](https://homepages.inf.ed.ac.uk/wadler/papers/prettier/prettier.pdf)

Some terms and concepts have also been adopted from the following sources:
- [dartfmt blog post](https://journal.stuffwithstuff.com/2015/09/08/the-hardest-program-ive-ever-written/)
- [scalafmt thesis](https://geirsson.com/assets/olafur.geirsson-scalafmt-thesis.pdf)


Reading any of these sources is generally not necessary in order to understand and start contributing to the formatter.
If you do want to dig deeper into the underlying details, read them in the order they were presented.

### PR descriptions
Another good source of information about the architecture are the PRs for the formatter, for instance this one:
https://github.com/neo4j/cypher-language-support/pull/460

Throughout development, we generally tried to be overly verbose about descriptions. So if you're looking for details on
how exactly it works, that's generally a good place to start.

## High-level overview of the flow from query -> formatted query
1. The query is parsed into a CST using the ANTLR-generated parser.
2. An ANTLR visitor class walks through the CST (see `formatting.ts`), does some preprocessing (e.g. grouping related constructs), and adds
"simple formatting" (formatting without dynamic line breaks)
3. The processed CST is handed off to the "layout engine" (see `layoutEngine.ts`) which applies the "layout-based" approach to dynamic line breaks,
which, in short, produces formatting output similar to that of the Prettier JS/TS formatter. See the next section for more details.

### Visualization
![Formatter architecture](./architecture.png)
## Layout engine
The layout engine, found in `layoutEngine.ts`, is what applies dynamic line breaks to the query, and it also might shuffle some comments around.
It uses the groupings generated by the visitor, and tries to apply the following algorithm (in pseudo-code):
```pseudo
function fit(group):
  if (group fits on one line):
    print group with no line breaks
  else:
    for each subgroup of group:
      split(subgroup)

function layoutEngine(groups):
  for each group in groups:
    fit(group)
```
A visualization of this is provided in the architecture image in the previous section.


## Syntax errors
The formatter has some support for formatting queries that contain syntax errors, but it is generally quite limited.
Generally, if the ANTLR parser can parse the query, the formatter will be able to make a decent attempt at formatting it.
If it can't (for instance when there are two erroneous tokens in a row), it will likely fail instead.

Some more details on how this is done on a code level can be found in this PR: https://github.com/neo4j/cypher-language-support/pull/412


## Internal types
The formatter uses some internal types that have been inspired by the sources mentioned in the introduction. The most important ones
are `chunk`s (from dartfmt) and `group`s (from the Prettier paper). Understanding these types, in particular groups, is key to understanding the 
formatting algorithm.

### Chunk
A chunk is similar to ANTLR Tokens, but differs in the following ways
1. A chunk can consist of multiple tokens, if those tokens should always be kept together.
An example would be the opening parenthesis of a function call, e.g. `function()`; `function(` would become one chunk,
since these tokens should never be separated.
2. Chunks contain metadata for the layout engine, such as where groups start and end, and when indentation should be applied. It might also
force certain breaking rules, such as always breaking after certain chunks.
3. Chunks may have comments attached to them, specifically comments that are at the end of a line. These comments may be moved around.

### Group
A group represents a collection of chunks that are semantically related. For instance, a group might contain all of the 
arguments in a function call, a node pattern, or nested groups. Groups are what build the structure for the layout engine,
so that it starts splitting lines as "far out" as possible, and avoids splitting nested language constructs.

For more context on what a group is, refer to the Prettier paper.
#### Example

```cypher
// Cypher query
RETURN a = 1, b = 1
// Groups visualized
[RETURN [a = 1,] [b = 1]]
```
