# stream-csv-as-json

> Micro-library of Node.js stream components for creating custom CSV processing pipelines with a minimal memory footprint. Parse CSV files far exceeding available memory using a SAX-inspired streaming token API. Companion to `stream-json`; uses the same token protocol. One dependency: `stream-json`.

## Install

npm i stream-csv-as-json

## Quick start

```js
import fs from 'node:fs';
import chain from 'stream-chain';
import {parser} from 'stream-csv-as-json';
import asObjects from 'stream-csv-as-json/as-objects.js';

const pipeline = chain([
  fs.createReadStream('data.csv'),
  parser(),
  asObjects()
]);

pipeline.on('data', token => console.log(token));
```

## API

### parser

`parser(options)` — factory returning a flushable function that produces `{name, value}` tokens. Accepts CRLF, LF, and bare CR row terminators; strips a leading UTF-8 BOM; throws on malformed quoted values.

- `parser.asStream(options)` — returns a Duplex stream.
- Options: `packStrings`/`packValues` (default: true), `streamStrings`/`streamValues` (default: true), `separator` (default: `','`).

```js
import parser from 'stream-csv-as-json/parser.js';
fs.createReadStream('data.csv')
  .pipe(parser.asStream())
  .on('data', token => console.log(token.name, token.value));
```

### Main module

The default export creates a parser Duplex stream with `emit()` applied:

```js
import make from 'stream-csv-as-json';
const stream = make();
stream.on('startArray', () => { /* row start */ });
stream.on('stringValue', val => { /* field value */ });
```

### asObjects

`asObjects(options)` — factory returning a flushable function that converts array tokens to object tokens using the first row as field names. The header collector auto-detects the upstream parser's mode (stream tokens vs packed `stringValue`), so it works with any parser configuration.

- `asObjects.asStream(options)` — returns a Duplex stream.
- `asObjects.withParser(options)` — creates a pipeline with the CSV parser.
- `asObjects.withParserAsStream(options)` — same, wrapped as a Duplex.
- Options: `packKeys` (default: true), `streamKeys` (default: true), `fieldPrefix` (default: `'field'`).
- `useStringValues` / `useValues` — deprecated no-ops kept for backward compatibility.

```js
import asObjects from 'stream-csv-as-json/as-objects.js';

chain([parser(), asObjects(), token => console.log(token)]);
```

### stringer

`stringer(options)` — factory returning a flushable function converting CSV token stream back to CSV text.

- `stringer.asStream(options)` — returns a Duplex stream.
- Options: `useStringValues`/`useValues` (default: false), `separator` (default: `','`), `rowTerminator` (default: `'\r\n'` per RFC 4180; override with `'\n'` for Unix-style output).

```js
import stringer from 'stream-csv-as-json/stringer.js';
chain([parser(), stringer(), fs.createWriteStream('output.csv')]);
```

## Token protocol

The parser emits `{name, value}` tokens: `startArray`, `endArray`, `startString`, `endString`, `stringChunk`, `stringValue`. After `asObjects`: `startObject`, `endObject`, `startKey`, `endKey`, `keyValue`.

## Common patterns

### Stream a huge CSV as objects

```js
import fs from 'node:fs';
import chain from 'stream-chain';
import {parser} from 'stream-csv-as-json';
import asObjects from 'stream-csv-as-json/as-objects.js';

chain([
  fs.createReadStream('huge.csv'),
  parser(),
  asObjects(),
  token => { if (token.name === 'endObject') processRow(); }
]);
```

### Compressed CSV processing

```js
import fs from 'node:fs';
import zlib from 'node:zlib';
import chain from 'stream-chain';
import {parser} from 'stream-csv-as-json';
import asObjects from 'stream-csv-as-json/as-objects.js';

chain([
  fs.createReadStream('data.csv.gz'),
  zlib.createGunzip(),
  parser(),
  asObjects()
]);
```

### CSV round-trip

```js
import fs from 'node:fs';
import chain from 'stream-chain';
import {parser} from 'stream-csv-as-json';
import stringer from 'stream-csv-as-json/stringer.js';

chain([
  fs.createReadStream('input.csv'),
  parser(),
  stringer(),
  fs.createWriteStream('output.csv')
]);
```

## Links

- Docs: https://github.com/uhop/stream-csv-as-json/wiki
- npm: https://www.npmjs.com/package/stream-csv-as-json
- Full LLM reference: https://github.com/uhop/stream-csv-as-json/blob/master/llms-full.txt
