\n\nWhat is Pelias?\n \nPelias is a search engine for places worldwide, powered by open data. It turns addresses and place names into geographic coordinates, and turns geographic coordinates into places and addresses. With Pelias, you’re able to turn your users’ place searches into actionable geodata and transform your geodata into real places.\n
\nWe think open data, open source, and open strategy win over proprietary solutions at any part of the stack and we want to ensure the services we offer are in line with that vision. We believe that an open geocoder improves over the long-term only if the community can incorporate truly representative local knowledge.\n\n\n# Pelias CSV Importer\n\nThis importer is designed to bring data into Pelias from a properly formatted CSV file.\n\nIt's originally based off of the [OpenAddresses importer](https://github.com/pelias/openaddresses), which also uses a CSV format.\n\n## Overview\n\nThis importer will process any CSV, attempting to create a Pelias document for each row.\n\nIn order to be useful, each row needs to define a source, a latitude, a longitude, and a name. Address components can optionally be specified.\n\nThis importer will accept any column name as uppercase or lowercase. Lowercase has priority if both are present.\n\n### Latitude\nLatitude can come from a column called `lat`. It should be a\n[WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84) value\nbetween `-90.0` and `90.0`.\n\n### Longitude\nLongitude can come from a column called `lon`. It should be a\n[WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84) value\nbetween `-180.0` and `180.0`.\n\n### Address\nA valid address consists of at least a street, and possibly a housenumber and postalcode.\n\nValid column names for street are: `street`\n\nValid column names for housenumber are: `housenumber`, `number`\n\nValid column names for postalcode are: `postalcode`, `postcode`, `zipcode`\n\nValid column names for intersections are: `cross_street` (note: `street` is also required!)\n\n## Name\nA free-form string that represents the name of a record. It might be\nthe name of a venue which also has an address, or the name of a city, mountain, or other interesting feature.\n\nValid column names for name are: `name`.\n\n## Layer\nPelias allows sorting records into different layers, representing different classes of data.\n\nThe most common layers are `address`, `street`, and `venue`. Address and street\nhave special meaning to Pelias: when Pelias looks for an `address`, it can also\nattempt to use its [interpolation engine](http://github.com/pelias/interpolation/)\nto fill in missing addresses. If no addresses (exact or interpolated) are\nfound, Pelias will try to find a `street` record matching the `street` from the\noriginal `address` in the query.\n\nAnother type of layer is \"administrative\" layers such as `city` and `country`.\n\nLayers do not have to fall into these categories. Any layer that doesn't have\nspecial meaning to Pelias can still be use to filter with the `layers`\nparameter to the Pelias API.\n\nValid column names for the layer value are: `layer` and `layer_id`\n\n## Source\nPelias understands that different data records come from different sources, and allows filtering\nbased on source. Common data projects that represent sources in Pelias include OpenStreetMap,\nOpenAddresses, and Who's on First.\n\nCustom data with arbitrary sources are supported by this importer and can be used for user\nfiltering. The source value won't have any other effect on how Pelias treats a record when querying.\n\nValid column names for the source value are : `source`\n\n## ID\n\nAn ID is a unique identifier for each record. Pelias IDs are strings, so they\ncan contain text. Pelias records must have a unique source, layer, and ID.\nAttempting to create multiple records with the same source, layer and ID will\ncause all but the most recent record to be overwritten.\n\nIf an ID is not specified for a row in a CSV, the row number will be used.\n\n## Names in multiple Languages\n\nMultiple names in different languages can be assigned by using the `name_$lang` fields, where $lang is an [ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) language code.\n\nFor example, to create a record for London in English and French, use the following CSV:\n\nid | name | name_fr | source | layer | lat | lon |\n-- | ---- | ------- | ------ | ----- | --- | --- |\n1 | London | Londres | custom | locality | 5 | 6 |\n\n## Multiple alias names\n\nA record can have multiple aliases, or alternative names, specified as an array using the `name_json` field.\n\nThe following CSV will create a record for [John F Kennedy International Airport](https://en.wikipedia.org/wiki/John_F._Kennedy_International_Airport), with common aliases including `JFK` and `JFK airport`.\n\nid | name | name_json | source | layer | lat | lon |\n-- | ---- | ------- | ------ | ----- | --- | --- |\n1 | John F Kennedy International Airport | \"[\"\"JFK\"\", \"\"JFK Airport\"\"]\" | custom | venue | 40.639722 | -73.778889\n\nThe contents of the `name_json` field must be a JSON array. As a reminder, in CSV files, records that contain commas must be quoted using double quotes, and records with a double quote in the value itself must be double-double-quoted, as shown above.\n\nAliases and languages can _both_ be specified. For example, the `name_json_es` field allows setting multiple aliases in Spanish.\n\n## Popularity\n\nPopularity values can be specified to mark records as more important than others. This value should be an integer greater than zero, in the `popularity` column.\n\n## Categories\n\nCategory values can be added to a record. For a single category, use the `category` field. For multiple categories, use `category_json`, with the same formatting as for alias names.\n\n## Parent\n\nParent information for record can be added using the `parent_json` field.\n\nOnly the valid parent field names specified in the `pelias/model` are supported, records with parent containing unsupported fields names will be ignored with a warning in the logs and will not be imported.\n\n[List of valid fields in pelias/model](https://github.com/pelias/model/blob/master/Document.js), which eventually should match the [list of valid fields in pelias/schema](https://github.com/pelias/schema/blob/master/mappings/document.js).\n\nThe contents of the `parent_json` field must be a valid JSON object. An example of the valid contents of `parent_json` field are:\n\n```\n {\n\t\"county\": [{\n\t\t\"id\": \"34\",\n\t\t\"name\": \"Innlandet\",\n\t\t\"abbr\": \"InL\",\n\t\t\"source\": \"OSM\",\n\t}],\n\t\"country\": [{\n\t\t\"id\": \"NOR\",\n\t\t\"name\": \"Norway\"\n\t\t\"abbr\": \"NO\"\n\t}],\n\t\"locality\": [{\n\t\t\"id\": \"3403\",\n\t\t\"name\": \"Hamar\"\n\t\t\"source\": \"SomeSource\"\n\t}]\n }\n```\n\nIn CSV files, records that contain commas must be quoted using double quotes, and records with a double quote in the value itself must be double-double-quoted, as shown below in the example for the `parent_json` field.\n\n```\n\"{\"\"county\"\":[{\"\"id\"\":\"\"34\"\",\"\"name\"\":\"\"Innlandet\"\"}],\"\"country\"\":[{\"\"id\"\":\"\"NOR\"\",\"\"name\"\":\"\"NO\"\"}],\"\"locality\"\":[{\"\"id\"\":\"\"3403\"\",\"\"name\"\":\"\"Hamar\"\"}]}\"\n```\n\nThe valid properties for any parent field are `id`, `name`, `abbr` (abbreviation), `source`, where `id` and `name` are mandatory fields. \nAny other fields will be ignored without any warning. In case the mandatory fields are missing the record will be ignored with the warning in the logs and will not be imported.\n\nIn the case where multiple parent values are provided for the same field name, we store all copies in the elastic index, making them all searchable, but only the first entry is used for displaying the label.\n\n## Custom data\n\nArbitrary custom data that does not fit into the standard Pelias schema can be stored for later retrieval under the `addendum` property.\n\nCurrently, custom data is supported when encoded as any valid JSON object. In the future, support for adding individual values via CSV columns will be supported.\n\nCustom data entires are namespaced, so this importer supports any column starting with `addendum_json_`. The rest of the column name will determine the namespace.\n\nFor example, to store a WikiData and Geonames concordance ID, the following CSV format might be used:\n\nid | name | source | layer | lat | lon | addendum_json_geonames | addendum_json_wikidata\n-- | -- | -- | -- | -- | -- | -- | --\n1 | test | custom | venue | 5 | 6 | \"{ \"\"id\"\": 600 } | { \"\"id\"\": \"\"Q47\"\" }\"\n\nThe Pelias [API](https://github.com/pelias/api) will then return a `GeoJSON` `Feature` like the following:\n\n```\n{\n \"properties\": {\n \"id\": \"1\",\n \"gid\": \"custom:venue:1\",\n \"layer\": \"venue\",\n \"source\": \"custom\",\n \"source_id\": \"1\",\n \"name\": \"test\",\n \"confidence\": 1,\n \"match_type\": \"exact\",\n \"accuracy\": \"centroid\",\n \"label\": \"London, England, United Kingdom\",\n \"addendum\": {\n \"geonames\": {\n \"id\": 600\n },\n \"wikidata\": {\n \"id\": \"Q47\"\n }\n }\n }\n}\n```\n\n## Requirements\n\nNode.js is required.\n\nSee [Pelias software requirements](https://github.com/pelias/documentation/blob/master/requirements.md) for supported versions.\n\n## Installation\n```bash\ngit clone https://github.com/pelias/csv-importer\ncd csv-importer\nnpm install\n```\n\n## Usage\n```bash\n# download files, if desired\n./bin/download\n\n# run an import\n./bin/start\n```\n\n## Downloading CSV files\n\nThis importer includes a downloader that supports downloading any uncompressed CSV files over HTTP/HTTPS.\n\n## Configuration\nThis importer can be configured in [pelias-config](https://github.com/pelias/config), in the `imports.csv`\nhash. A sample configuration file might look like:\n\n```javascript\n{\n \"esclient\": {\n \"hosts\": [\n {\n \"env\": \"development\",\n \"protocol\": \"http\",\n \"host\": \"localhost\",\n \"port\": 9200\n }\n ]\n },\n \"api\": {\n \"targets\": {\n \"yoursource\": [\"yourlayers\"]\n }\n },\n \"logger\": {\n \"level\": \"debug\"\n },\n \"imports\": {\n \"whosonfirst\": {\n \"datapath\": \"/mnt/data/whosonfirst/\",\n \"importPostalcodes\": false,\n \"importVenues\": false\n },\n \"csv\": {\n \"datapath\": \"/path/to/your/csv/files\",\n \"files\": [],\n \"download\": [\n \"https://example.com/csv-to-download.csv\"\n ]\n }\n }\n}\n```\n\n**Important:** You must put any custom source and layers imported by your data in `pelias.json` as explained in the relevant [API configuration documentation](https://github.com/pelias/api#custom-sources-and-layers). For a reasonably common use case for the source `csv` with only records in the `address` layer, the following configuration is a good starting point:\n\n\n```\n{\n \"api\": {\n \"targets\": {\n \"csv\": [\"address\"]\n }\n }\n}\n```\n\nThe following properties are recognized:\n\nThis importer is configured using the [`pelias-config`](https://github.com/pelias/config) module.\nThe following configuration options are supported by this importer.\n\n| key | required | default | description |\n| --- | --- | --- | --- |\n| `datapath` | yes | | The absolute path of the directory containing data files, or where downloaded files will be stored. |\n| `files` | no | `[]` | An array of the names of the files to import. If specified, *only* these files will be imported. If not specified, or empty, *all* `.csv` files in the given directory will be imported. |\n| `download` | no | `[]` | An array of URLs of CSV files that can be downloaded. Files must be plain-text (uncompressed) CSV files |\n","readmeFilename":"README.md"}
![](\"https://raw.githubusercontent.com/pelias/design/master/logo/pelias_github/Github_markdown_hero.png\"){kind=logo}
\n![](\"https://img.shields.io/github/license/pelias/api?style=flat&color=orange\")
![](\"https://img.shields.io/docker/pulls/pelias/api?style=flat&color=informational\")
![](\"https://img.shields.io/gitter/room/pelias/pelias?style=flat&color=yellow\")