1. json(1)
2. json tool manual
3. json(1)

NAME

json - JSON love for your command line

SYNOPSIS

something-generating-JSON-on-stdout | json [OPTIONS] [LOOKUPS]

json -f FILE [OPTIONS] [LOOKUPS...]

DESCRIPTION

json is a fast command-line tool for working with JSON content from the command line. Among its features: streaming stdin/stdout or working with JSON files, pretty-printing with control over output formats, JSON validation, filtering, modification, in-place JSON file modification, field extraction, tabular output, skipping HTTP header blocks for use with REST API responses, JSON stream ('\n'-separated JSON objects) processing.

Read on for details and examples. The FEATURE sections describe json features roughly in the order of processing.

FEATURE: HTTP Header Stripping

json roots are as a tool to assist working with REST APIs. Often results being parsed include HTTP headers, as from curl -i, with a JSON payload. By default json passes through HTTP header blocks. Use -H to strip a leading HTTP header block.

FEATURE: Grouping

(Added in json v4.) Use '-g' or '--group' to group adjacent objects into a single JSON array or to concatenate adjacent arrays into a single array. E.g.:

$ echo '{"a":1}
{"b": 2}' | json -g
[
  {
    "a": 1
  },
  {
    "b": 2
  }
]

$ echo '["one"]
["two"]' | json -g
[
  "one",
  "two"
]

"Adjacent" objects means objects separated by a newline, or by no space at all. Adjacent arrays means separate by a newline. These conditions are chosen as a balance between (a) not being ambiguous to parse with a simple regex and (b) enough to be useful for common cases.

Compatibility note: In json v3 and earlier, this used to be called "auto-arrayification" and was implicit. In json v4 and v5 grouping of adjacent arrays separated by no space was allowed. That was dropped in v6 (see issue #55). See the COMPATIBILITY section below.

FEATURE: Streaming

Grouping can be helpful for "one JSON object per line" formats or for things such as:

$ cat *.json | json -g ...

However, when the size of the input is large practically one must do stream processing. As of json v5.1, json -ga will stream. An extreme example is:

$ yes '{"foo":"bar"}' | json -ga

But a more practical example would be a large file of newline-separated JSON objects, such as a Bunyan log file:

$ cat foo.log | json -ga req.method req.url res.headers.x-response-time
GET /ping 1
POST /images 43
...

FEATURE: Merging

(Added in json v4.) Use '--merge' or '--deep-merge' to merge adjacent JSON objects in the input. Keys in the last object win.

$ echo '{"one":"un","two":"deux"}
{"one":"uno","three":"tres"}' | json --merge
{
  "one": "uno",
  "two": "deux",
  "three": "tres"
}

This could be useful for merging multiple config files, e.g.:

$ cat /opt/app/etc/defaults.json \
    /etc/app/config.json \
    ~/.app/config.json | json --merge
...

FEATURE: Validation

Since v1.2.0 json will give position information and context for JSON syntax errors (SyntaxError). This can be handy for validating data and config files:

$ cat config.json | json
json: error: input is not JSON: Unexpected ',' at line 17, column 5:
            , { "name": "smartos64-1.4.7"
        ....^
{
    "use-proxy": false
...
$ echo $?
1

Processing and output of the input JSON can be suppressed with the -n, --validate option:

$ cat config.json | json --validate
json: error: input is not JSON: Unexpected ',' at line 17, column 5:
            , { "name": "smartos64-1.4.7"
        ....^

Together with the -q you can get silent, exit-status-only, JSON validation:

$ cat config.json | json -nq
$ echo $?
1

FEATURE: Execution

(Added in json v7.) Use the -E CODE option to execute (JavaScript) code on the input JSON.

$ echo '{"name":"trent","age":38}' | json -E 'this.age++'
{
  "name": "trent",
  "age": 39
}

If input is an array, this will automatically process each item separately:

$ echo '[{"age":38},{"age":4}]' | json -E this.age++
[
  {
    "age": 39
  },
  {
    "age": 5
  }
]

That can be overriden with -A:

$ echo '[{"age":38},{"age":4}]' | json -A -E 'this[0].age = "unknown"'
[
  {
    "age": "unknown"
  },
  {
    "age": 4
  }
]

The given CODE is executed in a function bound to the input object (i.e. this is the input object).

Security note: CODE is not executed in a sandbox, so json's globals are available and unguarded. You can shoot yourself in the foot. Do not pass untrusted user-supplied strings here.

Compatibility note: There is also a -e CODE option (lowercase "e") from earlier versions of json to do much the same thing (with slightly different semantics for the CODE). It is still supported for backward compatibility. However it is deprecated because it can cause processing to be 10x or more slower for large inputs. See the COMPATIBILITY section below.

FEATURE: Conditional filtering

Use the -C CODE option to run JavaScript code ending with a statement returning a boolean to filter the input JSON.

$ echo '[{"age":38},{"age":4}]' | json -C 'this.age > 21'
[{"age":38}]

As with -E above, if input is an array, this will automatically process each item separately. This can be overriden with -A.

The given CODE is executed in a function bound to the input object (i.e. this is the input object). A JavaScript function must use return to return a value, so as a convenience if "return" is not in the given CODE it is presumed to be a single statement and it is wrapped:

function () {
    return ( CODE );
}

To use multiple statements in -C CODE you must explicitly use return, e.g.:

$ echo '{"a": 2, "b": 6}' | json -C 'sum = this.a + this.b; return sum > 5'
{
  "a": 2,
  "b": 6
}

Security note: CODE is not executed in a sandbox, so json's globals are available and unguarded. You can shoot yourself in the foot. Do not pass untrusted user-supplied strings here.

Compatibility note: There is also a -c CODE option (lowercase "c") from earlier versions of json to do much the same thing (with slightly different semantics for the CODE. It is still supported for backward compatibility. However it is deprecated because it can cause processing to be 10x or more slower for large inputs. See the COMPATIBILITY section below.

FEATURE: Lookups

Use lookup arguments to extract particular values:

$ echo '{"name":"trent","age":38}' | json name
trent

$ echo '{"name": {"first": "Trent", "last": "Mick"}, "age": 38}' \
    | json name.first age
Trent
38

Use -a for array processing of lookups and tabular output:

$ echo '{"name":"trent","age":38}' | json name
trent
$ echo '[{"name":"trent","age":38},
         {"name":"ewan","age":4}]' | json -a name age
trent 38
ewan 4

Integral values work for array index lookups:

$ echo '["a", "b", "c"]' | json 1
b

Negative array indices are also supported for convenience (a la Python array indexing):

$ echo '["a", "b", "c"]' | json -- -1
c
$ echo '["a", "b", "c"]' | json -- -2
b

If your lookup isn't a number or a JS indentifier you can always use JavaScript array-style lookups like this:

$ echo '{"http://example.com": "my-value"} | json '["http://example.com"]'
my-value

just like you would in JavaScript:

$ node
> var d = '{"http://example.com": "my-value"}
> d["http://example.com"]
'my-value'

FEATURE: Pretty-printing

Output is "jsony" by default: 2-space indented JSON ...

$ echo '{"name": "trent", "age": 38}' | json
{
  "name": "trent",
  "age": 38
}

... with one exception, a bare string value is printed without quotes.

$ echo '{"name": "trent", "age": 38}' | json name
trent

If pure JSON output is wanted, use -o json or the -j shortcut:

$ echo '{"name": "trent", "age": 38}' | json -o json name
"trent"

Indentations other than 2 can be selected via -o json-N

$ echo '{"name": "trent", "age": 38}' | json -o json-0
{"name":"trent","age":38}
$ echo '{"name": "trent", "age": 38}' | json -o json-4
{
    "name": "trent",
    "age": 38
}

The "FORMAT-N" suffix can also be useful on "jsony" when selecting multiple values and wanting tabular output where some cells are objects:

$ cat users.json
[
   {"name": {"first": "Trent", "last": "Mick"}, "age": 38},
   {"name": {"first": "Ewan", "last": "Mick"}, "age": 4}
]

$ json -f users.json -a name age -o jsony-0
{"first":"Trent","last":"Mick"} 38
{"first":"Ewan","last":"Mick"} 4

You can get colored (non-JSON) output using node.js's util.inspect:

$ echo '[{"name": "Trent"},{"name": "Ewan"}]' | json -o inspect
[ { name: 'Trent' },
  { name: 'Ewan' } ]

FEATURE: Listing keys

Sometimes you want the list of keys for an object. Use -k or --keys for that:

$ echo '{"name": "trent", "age": 38}' | json -k
[
  "name",
  "age"
]
$ echo '{"name": "trent", "age": 38}' | json -ka
name
age

FEATURE: In-place editing

You can edit a file in place with -I and -f FILE:

$ cat config.json
{"hostname":"127.0.0.1"}

$ json -I -f config.json                        # format the file
json: updated "config.json" in-place
$ cat config.json
{
  "hostname": "127.0.0.1"
}

$ json -I -f config.json -E 'this.port=8080'    # add port field
json: updated "config.json" in-place
$ cat config.json
{
  "hostname": "127.0.0.1",
  "port": 8080
}

Some limitations. Only one file at a time:

$ json -I -f foo.json -f bar.json
json: error: must specify exactly one file with '-f FILE' to use -I/--in-place

Lookups are not allowed:

$ json -I -f foo.json key.subkey
json: error: lookups cannot be specified with in-place editing (-I/--in-place), too easy to lose content

because that can too easily result in data loss, e.g. with something like:

$ json -I -f *.json    # if there is more than match to the glob
json: error: lookups cannot be specified with in-place editing (-I/--in-place), too easy to lose content

OPTIONS

-h, --help

Print this help info and exit.

--version

Print version of this command and exit.

-q, --quiet

Don't warn if input isn't valid JSON.

By default json will process input from stdin. Alternatively, an input file (or files) can be specified:

-f FILE

Specify an input file (instead of stdin).

By default json output is to stdout. Together with -f FILE, in-place editing can be done:

-I, --in-place

Edit the file given with -f FILE in-place. Lookups are not allowed with in-place editing, because it is too easy to accidentally lose file data.

If your JSON output is a REST API response, it might include the headers (e.g. when calling with curl -i). By default json will pass those headers through (without choking on them). However if you want them stripped you can use:

-H

drop any HTTP header block (as from curl -i ...)

Other pre-JSON input handling:

-g, --group

Group adjacent objects into an array of objects, or concatenate adjacent arrays into a single array.

--merge, --deep-merge

Merge adjacent objects into a single object with merged keys. Values in later objects win. Use --deep-merge to recursively merge keys in objects.

You can process elements of an input array separately and generate tabular output:

-a, --array

Process input as an array of separate inputs and output in tabular form.

-d DELIM

Delimiter character for tabular output (default is ' ').

-A

Process input as a single object, i.e. stop -E and -C automatically processing each item of an input array. Likewise for the deprecated -c and -e options.

You can execute code on (-E) and filter (-C) the input (this is done before LOOKUPS are processed, if any).

-E CODE

Execute the given JavaScript code on the input. If input is an array, then each item of the array is processed separately (use -A to override).

-C CODE

Filter the input with JavaScript CODE. If CODE returns false-y, then the item is filtered out. If input is an array, then each item of the array is processed separately (use -A to override).

Older similar -e and -c options exist for execution and conditional filtering -- now deprecated because they can be 10x or more slower.

-e CODE

Execute the given JavaScript code on the input. Deprecated in v7. Use -E CODE instead.

-c CODE

Filter the input with JavaScript CODE. If CODE returns false-y, then the item is filtered out. Deprecated in v7. Use -C CODE instead.

Finally, if LOOKUP arguments are given, these are extracted from the JSON. By default . is used as a separator for nested object lookup. This can be overridden:

-D DELIM

Delimiter char between LOOKUPS (default is '.'). For example: $ echo '{"a.b": {"b": 1}}' | json -D / a.b/b

An alternative to lookups is to output the keys of the input object:

-k, --keys

Output the input object's keys.

json can be restricting to just validating its input, i.e. processing and output of the input is skipped:

-n, --validate

Just validate the input, no processing or output of the JSON content.

By default json outputs in "jsony" mode. Basically this is JSON output, with the exception that a single string output value is emitted without the quotes. The intention here is to be of most use to the UNIX command-line. Other output formats are supported:

-o MODE, --output MODE

Specify an output mode. One of jsony (the default; JSON, if a single string then quotes are elided), json (JSON output, 2-space indent), json-N (JSON output, N-space indent, e.g. 'json-4'), or inspect (node.js util.inspect output).

-i

Shortcut for -o inspect.

-j

Shortcut for -o json.

EXAMPLES

A typical JSON REST API response:

$ curl -s http://ifconfig.me/all.json
{"connection":"","ip_addr":"216.57.203.67","lang":"","remote_host":...

Nice output by default:

$ curl -s http://ifconfig.me/all.json | json
{
  "connection": "",
  "ip_addr": "201.73.103.12",
  "lang": "",
  "remote_host": "",
  "user_agent": "curl/7.23.1 (i386-sun-solaris2.11) libcurl/7.23.1 OpenSSL/0.9.8w zlib/1.2.3 libidn/1.23 libssh2/1.2.2",
  "charset": "",
  "port": "63713",
  "via": "",
  "forwarded": "",
  "mime": "*/*",
  "keep_alive": "",
  "encoding": ""
}

Say you just want to extract one value:

$ curl -s http://ifconfig.me/all.json | json ip_addr
201.73.103.12

Or, looking at the node.js project using the Github API:

$ curl -s https://api.github.com/repos/joyent/node | json open_issues
517

If you use curl -i to get HTTP headers (because perhaps they contain relevant information), json will skip the HTTP headers automatically:

$ curl -is https://api.github.com/repos/joyent/node | json
HTTP/1.1 200 OK
Server: nginx/1.0.13
Date: Tue, 24 Jul 2012 04:01:08 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Status: 200 OK
ETag: "1a21d980a01768dde42145ce2b58694c"
X-RateLimit-Remaining: 4997
Content-Length: 1513
Cache-Control: public, max-age=60
Vary: Accept
X-RateLimit-Limit: 5000
Last-Modified: Tue, 24 Jul 2012 03:50:11 GMT

{
  "master_branch": "master",
  "has_issues": true,
  "has_downloads": false,
  "homepage": "http://nodejs.org/",
  "html_url": "https://github.com/joyent/node",
...

Or, say you are stuck with the headers in your pipeline, 'json -H' will drop HTTP headers:

$ curl -is https://api.github.com/repos/joyent/node | json -H forks
2158

Here is an example that shows indexing a list. (The given "lookup" argument is basically JavaScript code appended, with '.' if necessary, to the JSON data and eval'd.)

$ curl -s https://api.github.com/legacy/repos/search/nodejs \
    | json 'repositories[2].name'
socket.io

Having the quote to avoid shell interpretation of '[' is annoying, so json allows a special case for an integer lookup:

$ curl -s https://api.github.com/legacy/repos/search/nodejs \
    | json 'repositories.2.name'
socket.io

Array processing with -a

json includes the -a (aka --array) option for processing each element of an input JSON array independently and using tabular output. Let's first get a list of open node.js issues (note that this is a subset because of GH API pagination):

$ curl -s https://api.github.com/repos/joyent/node/issues?state=open\&per_page=100
[
  {
    "number": 3757,
    "html_url": "https://github.com/joyent/node/issues/3757",
    "body": "Fix #3756.\n\nReview, please: @TooTallNate",
    "milestone": null,
    "user": {
      "gravatar_id": "73a2b24daecb976af81e010b7a3ce3c6",
      "login": "isaacs",
      "avatar_url": "https://secure.gravatar.com/avatar/73a2b24dae...
...

We can then print a table with just some fields as follows:

$ curl -s https://api.github.com/repos/joyent/node/issues?state=open\&per_page=100 \
    | json -a comments number title
0 3757 readline: Remove event listeners on close
0 3756 readline: No way to completely unhook interface from input/output
1 3755 node-v0.6.20 hello example segfaults on RaspberryPi (w/Arch + bash)
0 3753 Prohibit same listeners in EventEmitter. Closes #964.
1 3752 Auto-detect hardfloat eabi and armv7 variables on ARM based on compiler
3 3751 persistent REPL history
0 3749 glibc errors on SheevaPlug / Debian Squeeze
...

Ultimately this can be useful for then using other command-line tools. For example, we could get the list of top-five most commented open node issues:

$ curl -s https://api.github.com/repos/joyent/node/issues?state=open\&per_page=100 \
    | json -a comments number title | sort -n  | tail -5
9 3510 Automatically `.toString()` functions in REPL.
11 3668 JSON documentation index listing
12 3624 Add a return value to Buffer.write* methods that returns the ...
12 3655 defer dgram listening event
14 3613 Connections closed by node stay permanently in FIN_WAIT2

Or get a breakdown by ISO language code of the recent tweets mentioning "nodejs":

$ curl -s http://search.twitter.com/search.json?q=nodejs\&rpp=100 \
    | json results | json -a iso_language_code | sort | uniq -c | sort
   1 es
   1 no
   1 th
   4 ru
  12 ja
  23 pt
  58 en

The -d option can be used to specify a delimiter:

$ curl -s https://api.github.com/repos/joyent/node/issues?state=open \
        | json -a created_at number title -d,
2012-07-24T03:45:03Z,3757,readline: Remove event listeners on close
2012-07-24T03:32:10Z,3756,readline: No way to completely unhook inte...
2012-07-23T21:17:50Z,3755,node-v0.6.20 hello example segfaults on Ra...
2012-07-22T16:17:49Z,3753,Prohibit same listeners in EventEmitter. C...
2012-07-22T13:43:40Z,3752,Auto-detect hardfloat eabi and armv7 varia...

COMPATIBILITY

The json tool major version is incremented when there is a backward incompatible change. An overview of those changes is here.

• v7: -E CODE and -C CODE were added in favour of -e CODE and -c CODE because the former provide a 10x or more performance improvement for larger inputs. The latter are still included for backward compatibility. -E/-C use a JavaScript function to execute CODE, which -e/-c use node.js's vm.runInNewContext which is crazy slow. Use of a JavaScript function places slightly different semantics and requirements on the given CODE, so new options were required for compat.
• v6: Grouping (via -g or --group) of adjacent arrays no longer groups arrays separated by no space. I.e. adjacent arrays must be separated by a newline.
• v5: Special case the output for a single lookup AND JSON output (i.e. -j or -o json*) to only output the value instead of the more general array or table that is necessary for multiple lookups.
• v4: Made "auto-arrayification" require an explicit '-g' or '--group' option to prefer that implicit processing never magically fix otherwise invalid JSON. The feature is now called grouping.
• v3: Cleaned up json and "jsony" output formatting to be more consistent, especially for array processing.

See the changelog for full compatibility and change details.

INSTALL, PROJECT, BUGS

json is written in JavaScript and requires node.js (node). You can either install via npm:

npm install -g jsontool

or manually get the script and put it on your PATH somewhere (json is a single file with no external deps other than node itself):

cd ~/bin
curl -L https://github.com/trentm/json/raw/master/lib/jsontool.js > json
chmod 755 json

(Note: NOT npm install json. Unfortunately and confusingly, that will install a different and incompatible json CLI tool.)

This project lives at https://github.com/trentm/json. Please report bugs to https://github.com/trentm/json/issues. See the full changelog at: https://github.com/trentm/json/blob/master/CHANGES.md.

LICENSE

MIT License (see https://github.com/trentm/json/blob/master/LICENSE.txt)

COPYRIGHT

json is Copyright (c) 2013 Trent Mick and Copyright (c) 2013 Joyent Inc. All rights reserved.

2. October 2013
3. json(1)