async.js
========

**A clean, composable way to manipulate sequences of values with asynchronous functions
**

Dealing with control flow in heavily asynchronous code can be a big challange. Without any helper functions the code can easily degenerate into a christmas tree shape because of the callback of a callback of a callback syndrome.

    asncFunction1(function(err, result) {
        asncFunction2(function(err, result) {
            asncFunction3(function(err, result) {
                asncFunction4(function(err, result) {
                    asncFunction5(function(err, result) {
                        // do something useful
                    })
                })
            })
        })
    })
    
With async.js this can be written as

    async.list([
        asncFunction1,
        asncFunction2,
        asncFunction3,
        asncFunction4,
        asncFunction5,
    ]).call().end(function(err, result) {
        // do something useful
    });
    
It gets even worse if an asynchronous function has to be applied to a sequence of values. There is a nice example of this in the  howtonode.org article [Control Flow in Node Part III](http://howtonode.org/control-flow-part-iii):

    // Here is the async version without helpers
    function loaddir(path, callback) {
      fs.readdir(path, function (err, filenames) {
        if (err) { callback(err); return; }
        var realfiles = [];
        var count = filenames.length;
        filenames.forEach(function (filename) {
          fs.stat(filename, function (err, stat) {
            if (err) { callback(err); return; }
            if (stat.isFile()) {
              realfiles.push(filename);
            }
            count--;
            if (count === 0) {
              var results = [];
              realfiles.forEach(function (filename) {
                fs.readFile(filename, function (err, data) {
                  if (err) { callback(err); return; }
                  results.push(data);
                  if (results.length === realfiles.length) {
                    callback(null, results);
                  };
                });
              });
            }
          });
        });
      });
    }
    
This code reads the contents of a directory, filters out all directory and returns the contens of all files as an array. Without any helpers the whole control flow management totally obscures the intent of the code. With async.js it is possible to move the control flow aspects of the code out of the function and rewrite it like this:

    function loaddir(path, callback) {
        async.readdir(".")
            .stat()
            .filter(function(file) {
                return file.stat.isFile()
            })
            .readFile("utf8")
            .filter(function(file) {
                return file.data
            })
            .toString(callback)
    }
    

Features
========

Core
----

JavaScript Array like abstraction, which supports almost all JavaScript array functions with asynchronous versions of

- filter
- map
- reduce
- forEach
- some
- every

All functions are chainable.

node.js file system plugin
--------------------------

Most of the functions from the `fs` module in node.js are exposed as chainable filters.

- `stat` (lstat, fstat)
- `unlink`
- `mkdir`/`rmdir`
- `realpath`
- `open`/`close`
- `exists` (from `path.exists`)
- `chmod`
- `readFile`/`writeFile`

Constructors

- `files`: convert a list of files into a file sequence
- `readdir`: create a file sequence from the files in a directory
- `walkfiles`: recursive file walker
- `glob`: Unix style file matching. Creates a file sequence of all matching files.

Utility functions to work with the file system

- `abspath`: convert a relative to an absolute path
- `copyfile`: copy a single file
- `copytree`: copy whole diroctory tree
- `rmtree`: remove whole direcory tree (like `rm -rf`)

Utils plugin
------------

Generic filters:

- `delay`: wait a given time before returning the next value
- `timeout`: If the source doesn't respond in a given time an error is returned.
- `inspect`: print the source value as JSON
- `print`: print the source value as string

How does it work?
=================

## Construcotrs

Constructors are the sources of async streams. They return an `async.Generator` instance which generates a sequence of values.

## Filters

Filters work on the sequence generated by a source generator. They can remove, add and modify values in the sequence. Filters return themselves generator instances.

## Drivers

Driver methods drive the actual execution of the iteration. The hole system is based on the principle of pulling the values from the end. Until any of the drivers is called no value will be generated.


TODO
====

- Add documentation about the fs plugin
- Add documentation about the unit testing framework
- improve test coverage of the core functionality
- add node package manager support (npm, kiwi)
- create browser version

Core API
========

## Constructors

### async.range([start=0], [stop], [step=1], [construct=async.Generator])

Creates a generator generating an arithmetic progression of integers.
`range(i, j)` returns [i, i+1, i+2, ..., j-1] and `start` defaults to 0.
When step is given, it specifies the increment (or decrement). If `stop` is not given an infinite number of values if generated. This generates all positive odd numbers:

    async.range(1, null, 2)

The optional `construct` argument must be a sub class of `async.Generator` and can be used to tell the function to return instances of the class.


### async.list(arr, [construct=async.Generator])

Creates a generator, which returns the elements of the given array. This generates e.g. the values `1`, `2` and `3`:

    async.list([1, 2, 3])

The optional `construct` argument must be a sub class of `async.Generator` and can be used to tell the function to return instances of the class.


## async.Generator(source)

Generators are implementing the 'Iterator' design pattern in an asynchronous way. It has a `next(callback)` method, which passes on each call the next generated value to the given callback. Generators can be chained and all construcors and filters return new `Generator` instances.

The `source` is either another `async.Generator` or a generator function. All filters in the generator operate on the values generated by the source.

    var i=0;
    var gen = new async.Generator(function(callback) {
        if (i>4)
            callback(async.STOP)
        else
            callback(null, i++) // no error
    })
    
`gen` in this example generates the sequence [0, 1, 2, 3, 4]. The first argument of the callback can be used to indicate an error the the end of the iteration, whicle the second argument is the generated value.

### Constant: async.STOP

Special error value, which indicates the end of the iteration.

### Method: next(callback)

Calls the callback with the next generated value:

    gen.next(function(err, value) {
        //
    })
    
The generated value is passed in the `value` argument. If an error occured while computing the value the `err` argument is set. In this case value the meaning of value is undefined. `async.STOP` can be passed as special error value to indicate the end of the iteration.

Usually this method is only needed to write custom filters.

### Filter: map(mapper)

Applies the `mapper` function to all values of the generator's source and generates the result of the mapping.

    async.range(0, 3)
        .map(function(item, next) {
            next(null, item + 1)
        })
    
This will add one to each incoming item and thus generate the sequence [1, 2, 3].

The mapper can also be synchronous and simply return the result. In this case the mapper must not have the `next` argument.


### Filter: filter(filter)

Calls filter for each of the generator's source values. If the `filter` returns `false` for a passed value, the value is dropped from the sequence, otherwise the value is forwarded.

    async.list([1, 2, 3, 4])
        .filter(function(item, next) {
            next(null, item % 2 == 0)
        })
        
This will filter out all odd values and generate the sequence [2, 4].


### Filter: slice(begin, end)

Counts the generator's source values starting with 0 and skips all values with an index smaller then `begin`. If the index is >= `end` the sequence is stopped.

    async.range(0)
        .slice(1, 4)
        
The `async.range` call generates an infinite sequence of all integers and the `slice` call transforms this into the sequence [1, 2, 3] by dropping the first value and stopping the sequence after the 4th value.

### Filter: reduce(reduce, [initialValue])

Sum up the number from 1 to 5:

    async.range(1, 5)
        .reduce(function(previousValue, currentValue) {        
            return previousValue + currentValue;
        })
        
Sum up the number from 1 to 5 but with a first 'initialValue' of 10:

    async.range(1, 5)
        .reduce(function(previousValue, currentValue, index, next) {
            next(null, previousValue + currentValue);
        }, 10)


### Filter: forEach(fn)

Alias for `each`.

### Filter: each(fn)

Calls `fn` for each of the source generator's values. The returned generator passes on the source values.

    async.range(1, 10)
        .each(function(item, next) {
            console.log(item);
        })
        
Prints the numbers from 1 to 9. Otherwise the source sequence is not changed.

### Filter: some(condition)

Check if any of the values in the source sequence match the condition (i.e. the `condition` callback returns true). The returned generator generates a one elmement sequence, which contains the boolean value of the result.

Check if the source generates an even number:

    async.list([1, 8, 3, 5])
        .some(function odd(item) {
            return item % 2 == 0
        })

This generates [true].

### Filter: every(condition)

Check if all of the values in the source sequence match the condition (i.e. the `condition` callback returns true). The returned generator generates a one elmement sequence, which contains the boolean value of the result.

Check if the source generates only even numbers:

    async.list([1, 8, 3, 5])
        .every(function odd(item) {
            return item % 2 == 0
        })

This returns [false]

### Filter: call([context])

Calls each of source values in sequence, which have to be functions. The functions can either be synchronous and return a value of be asynchronous and call the passed callback on completion. The generated sequence are the function's return values. The optional `context` argument defines the `this` context of the called functions.

    async.list([
        function sync() {
            return "juhu"
        },    
        function async(next) {
            next(null, "kinners")
        }
    ]).call()
    
This calls the two functions in sequence and generates the sequence ["juhu", "kinners"].

### Filter: concat(...)

Concatenates a variable number of generators with this generator. 

    async.range(1, 4)
        .concat(async.range(10, 13))
        
This returns a generator, which generates the values of the first `range` followed by the values of the second `range`: [1, 2, 3, 10, 11, 12]


### Filter: zip(...)

Returns a generator, which generates on each `next()` an arroy of the next values of this generator and each of the passed arguments. The length of the generated sequence is truncated to the lenght of the shortest input sequence. This function expects a variable list of generators as arguments.

    async.range(1, 4)
        .zip(async.range(10, 14))
        
This will generate [[1, 10], [2, 11], [3, 13]]

### Filter: sort([compare])

Performs an Array.prototype.sort on the source values and returns a generator, which generates the result as one element sequence. Since this requires reading all source elements, the source must not generate an infinite number of values. Generates the read values in sorted order.

    async.list([4, 2, 3, 9])
        .sort()

This generates the sequence [2, 3, 4, 9]

### Filter: join([separator])

Performs an Array.prototype.join on the source values and returns a generator, which generates the result as one element sequence. Since this requires reading all source elements, the source must not generate an infinite number of values.

    async.list([1, 4, 5])
        .join(" - ")

This will generate ["1 - 4 - 5"]

### Filter: reverse()

Reverses the elements generated by the generator's source. Since this requires reading all source elements, the source must not generate an infinite number of values.

async.list([1, 4, 5])
    .revers()

This will generate [5, 4, 1]


### Driver: end([callback])

Perform the iteration until the generator's source either returns `async.STOP` or indicates an error. The value of the last generated value is passed to the callback.


    async.list([1, 8, 3, 5])
        .every(function odd(item) {
            return item % 2 == 0
        })
        .end(function(err, allEven) {
            console.log("All values are even: " + allEven)
        }) 

This will print the last generated value (the result of `very`) on the console.

### Driver: toArray([breakOnError=true], callback)

Perform the iteration until the generator's source either returns `async.STOP`. If `breakOnError` is true the iteration is stopped on the first error. Otherwise the iteration continues and all errors are collected in an error array. An array of all generated values is passed to the callback.

    async.list([1, 8, 3, 5])
        .map(function odd(item, next) {
            next(err, item * 10)
        })
        .toArray(function(err, values) {
            console.log(values)
        }) 

The last callback will be called with arr set to [10, 80, 30, 50].
