\n\nWhat is Pelias?\n \nPelias is a search engine for places worldwide, powered by open data. It turns addresses and place names into geographic coordinates, and turns geographic coordinates into places and addresses. With Pelias, you're able to turn your users' place searches into actionable geodata and transform your geodata into real places.\n
\nWe think open data, open source, and open strategy win over proprietary solutions at any part of the stack and we want to ensure the services we offer are in line with that vision. We believe that an open geocoder improves over the long-term only if the community can incorporate truly representative local knowledge.\n\n\n# Pelias Query\n\nElasticsearch geospatial and linguistic matching queries used by Pelias.\n\n## Installation\n\n```bash\n$ npm install pelias-query\n```\n\n[](https://nodei.co/npm/pelias-query)\n\n## NPM Module\n\nThe `pelias-query` npm module can be found here:\n\n[https://npmjs.org/package/pelias-query](https://npmjs.org/package/pelias-query)\n\n#### About\n\nThis repository contains all the *geospatial* and *linguistic* matching Elasticsearch queries used in the [Pelias geocoder](https://github.com/pelias/pelias).\n\nAn attempt has been made to provide the queries in a more general-purpose fashion. Only a few variables need to be changed in order to use the same queries with any Elasticsearch [schema](https://github.com/pelias/schema).\n\nFeel free to fork the project, Pull Requests are welcome!\n\n#### Motivation\n\nAs the complexity and variability of database queries grows in a large project, their maintenance starts to become more and more difficult.\n\nChanges to the controller layer can have significant impact on the query layer and vice versa, making refactoring a chore.\n\nAdditionally; the controller code used to compose these queries becomes a horrible mix of user input validation and query composition logic.\n\nIn many cases query logic is simply copy->pasted between queries to ensure validity when it could simply be reused.\n\nThis repo aims to solve some of these issues by providing:\n\n- a logical boundary between query composition and input validation.\n- a way to notate query variables which is distinct from the RESTful API.\n- a method of composing complex queries from smaller components.\n- a way of testing/debugging and re-using queries across repos/forks.\n- a language which describes the problem domain rather than an individual implementation.\n\nThe composition workflow should be instantly familiar to anyone who has used an MVC-type framework before.\n\n### Variables\n\nVariables are used as placeholders in order to pre-build queries before we know the final values which will be provided by the user.\n\n**note:** Variables can only be Javascript primitive types: `string` *or* `numeric` *or* `boolean`, plus `array`. No objects allowed.\n\n#### VariableStore API\n\n```javascript\nvar query = require('pelias-query');\n\n// create a new variable store\nvar vs = new query.Vars();\n\n// set a variable\nvs.var('input:name', 'hackney city farm');\n\n// or\nvs.var('input:name').set('hackney city farm');\n\n// get a variable\nvar a = vs.var('input:name');\n\n// get the primitive value of a variable\nvar a = vs.var('input:name');\na.get(); // hackney city farm\na.toString(); // hackney city farm\na.valueOf(); // hackney city farm\na.toJSON(); // hackney city farm\n\n// check if a variable has been set\nvs.isset('input:name'); // true\nvs.isset('foo'); // false\n\n// bulk set many variables\nvs.set({\n 'boundary:rect:top': 1,\n 'boundary:rect:right': 2,\n 'boundary:rect:bottom': 2,\n 'boundary:rect:left': 1\n});\n\n// export variables for debugging\nvar dict = vs.export();\nconsole.log( dict );\n```\n\n#### Default Variables\n\nThis library provides a dictionary of common [default values](https://github.com/pelias/query/blob/master/defaults.json) which can be used when instantiating a new variable store.\n\nThe defaults should be sufficient in the vast majority of cases but you may elect to change these defaults in order to modify how the queries execute for your specific installation.\n\n**note:** You can override any of the defaults at runtime.\n\n```javascript\nvar query = require('pelias-query');\n\n// create a new variable with the defaults\nvar vs = new query.Vars( query.defaults );\n\n// print all set variables\nconsole.log( vs.export() );\n```\n\n#### User Variables\n\nVariables coming from user input should be set on the variable store **whenever they are available**, below is a list of common user variables which can be set/unset to enable/disable query functionality.\n\n**note:** This list is non exhaustive, see the validation section of each view in order to confirm which specific variables it uses (explained below).\n\n```\ninput:name: 'hackney city farm'\n\nfocus:point:lat: 1.1\nfocus:point:lon: 2.2\n\ninput:housenumber: 101\ninput:street: \"hackney road\"\ninput:postcode: \"E81DN\"\n\ninput:country_a: \"GBR\"\ninput:country: \"hackney\"\ninput:region: \"hackney\"\ninput:region_a: \"hackney\"\ninput:county: \"hackney\"\ninput:localadmin: \"hackney\"\ninput:locality: \"hackney\"\ninput:neighbourhood: \"hackney\"\n\ninput:categories: \"food,education\"\n\nboundary:circle:lat: 1\nboundary:circle:lon: 2\nboundary:circle:radius: \"50km\"\n\nboundary:rect:top: 1\nboundary:rect:right: 2\nboundary:rect:bottom: 2\nboundary:rect:left: 1\n\nboundary:country: \"USA\"\n```\n\n### Views\n\nComplex queries can be composed of smaller 'views', these are query blocks which are marked up with placeholder variables and later 'compiled' with the actual user variables in place.\n\nViews are essentially just a function which takes one argument (the variable store `vs`) and returns either `null` (if the required variables are not available) *or* a javascript object which encapsulates the view.\n\n```javascript\n// example of a 'view'\nfunction ( vs ){\n\n // validate required params\n if( !vs.isset('input:name') ||\n !vs.isset('ngram:analyzer') ||\n !vs.isset('ngram:field') ||\n !vs.isset('ngram:boost') ){\n return null;\n }\n\n // base view\n var view = { \"match\": {} };\n \n // match query\n view.match[ vs.var('ngram:field') ] = {\n analyzer: vs.var('ngram:analyzer'),\n boost: vs.var('ngram:boost'),\n query: vs.var('input:name')\n };\n\n return view;\n}\n```\n\nIt's best practice to validate the variable(s) you are going to use at the top of your view so that:\n\n1. it doesn't execute with unmet dependencies *and*\n2. it is clear for other developers which variables are required to execute it\n\n#### View API\n\nAn example of the above view rendered would look like this:\n\n```javascript\nvar query = require('pelias-query'),\n view = query.view.ngrams;\n\nvar vs = new query.Vars({\n 'input:name': 'hackney city farm',\n 'ngram:analyzer': 'standard',\n 'ngram:field': 'name.default',\n 'ngram:boost': 1\n});\n\nvar rendered = view( vs );\n```\n\n```javascript\n{\n \"match\": {\n \"name.default\": {\n \"analyzer\": \"standard\",\n \"boost\": 1,\n \"query\": \"hackney city farm\"\n }\n }\n}\n```\n\n### Layouts\n\nJust as with most MVC frameworks the 'meta' view is called a 'layout', this is the envelope which wraps all other views.\n\nThere is only one view available in this library (at this time), named the `FilteredBooleanQuery`. This is essentially the most versatile type of Elasticsearch query, all other examples you find online are simplified versions of this `layout`.\n\n```javascript\nvar query = require('pelias-query');\n\nvar q = new query.layout.FilteredBooleanQuery();\n```\n\n##### FilteredBooleanQuery API\n\nThe `FilteredBooleanQuery` has two different methods for **assigning conditional views** *and* one method for handling the **sorting of results**.\n\n##### .score()\n\nThe `.score` method is used to assign views which **will affect the scoring** of the results.\n\nIn most cases you can assume that **records which match more of these conditions will appear higher in the results than those which match fewer**.\n\n```javascript\nvar q = new query.layout.FilteredBooleanQuery();\n\n// a 'should' condition, if a record matches, it's score will be increased\nq.score(view);\n\n// this is simply a more explicit equivalent of the above ('should' is the default)\nq.score(view, 'should');\n\n// in this case we mark the view as a 'must' match condition.\n// Matching results will affect the score **but** in this case\n// **non-matching records will be removed from the results completely**\nq.score(view, 'must');\n```\n\n##### .filter()\n\nThe `.filter` method is used to assign views which **do not affect the scoring** of results.\n\n**note:** The more results you remove before sorting; using either this method *or* the `.score` method above (with 'must'), the better your query performance will be.\n\n```javascript\nvar q = new query.layout.FilteredBooleanQuery();\n\n// **non-matching records will be removed from the results completely**\nq.filter(view);\n```\n\n##### .sort()\n\nThe `.sort` method is used to assign views which affect the sorting of results.\n\nThis method is not as useful as it sounds, for the most part you should be using `.score` methods above to affect the sorting of results.\n\nThis function is only really useful in cases where a 'tiebreaker' is needed. For example: searching 'mcdonalds' may result in several records which scored the same value, in this case we can attempt to 'break the tie'.\n\n**warning:** These functions are computed for every document which matches the above conditions. Adding many `.sort` conditions may have a negative affect on query performance.\n\n```javascript\nvar q = new query.layout.FilteredBooleanQuery();\n\n// this view is used to mediate 'tied' scoring situations\nq.sort( view );\n```\n\n### Composing Complex Queries\n\nGreat! So with the building blocks above we can start to build composable, testable and re-usable queries.\n\n#### Reverse Geocode\n\nOne of the simplest queries to build is a reverse geocoder, in this case we have indexed some documents with a `lat/lon` centroid and we would like to find the 1 nearest record to an arbitrary point.\n\n```javascript\nvar query = require('pelias-query'),\n vs = new query.Vars( query.defaults );\n\n// this is our focus point (somewhere in London)\nvar focus = { lat: 51.5, lon: -0.06 };\n\n/**\n build a query with 2 conditions:\n - (optional) geographic bounds\n - sort results by distance\n**/\nvar q = new query.layout.FilteredBooleanQuery()\n .filter( query.view.boundary_circle )\n .sort( query.view.sort_distance );\n\n// we only want 1 result\nvs.var('size', 1);\n\n// set bounding variables\nvs.set({\n 'boundary:circle:lat': focus.lat,\n 'boundary:circle:lon': focus.lon,\n 'boundary:circle:radius': '5km'\n});\n\n// set focus point\nvs.set({\n 'focus:point:lat': focus.lat,\n 'focus:point:lon': focus.lon\n});\n\n// render the query\nvar rendered = q.render( vs );\n```\n\nresults in a query such as:\n\n```javascript\n{\n \"query\": {\n \"bool\": {\n \"filter\": [\n {\n \"geo_distance\": {\n \"distance\": \"5km\",\n \"distance_type\": \"plane\",\n \"center_point\": {\n \"lat\": 51.5,\n \"lon\": -0.06\n }\n }\n }\n ]\n }\n },\n \"size\": 1,\n \"track_scores\": true,\n \"sort\": [\n \"_score\",\n {\n \"_geo_distance\": {\n \"order\": \"asc\",\n \"distance_type\": \"plane\",\n \"center_point\": {\n \"lat\": 51.5,\n \"lon\": -0.06\n }\n }\n }\n ]\n}\n```\n\n\n#### Linguistic Search with Local Bias\n\nThis example is the most commonly requested full-text search query. In this case we match *all* results but we also apply the following scoring:\n\n1. better linguistic matches rank higher in the results\n2. records near the 'focus' point also gain a localized 'boost'\n\nIn effect this means that we still show far away places but we also give more priority to local places.\n\n```javascript\nvar query = require('pelias-query'),\n vs = new query.Vars( query.defaults );\n\n// this is our focus point (somewhere in London)\nvar focus = { lat: 51.5, lon: -0.06 };\n\n/**\n build a query with 2 conditions:\n - the linguistic matching strategy for scoring (phrase)\n - the geographic decay function (focus)\n**/\nvar q = new query.layout.FilteredBooleanQuery()\n .score( query.view.phrase )\n .score( query.view.focus(query.view.phrase) );\n\n/**\n configure implementation-specific settings (or simply use the defaults):\n - phrase settings\n - focus settings\n**/\nvs.set({\n 'phrase:field': 'phrase.default',\n 'phrase:analyzer': 'standard',\n 'focus:function': 'gauss',\n 'focus:offset': '10km',\n 'focus:scale': '100km',\n 'focus:decay': 0.4\n});\n\n/**\n set the user-specific variables:\n - the input text provided by the user\n - the input point to use for localization\n**/\nvs.var('input:name', 'union square');\nvs.var('focus:point:lat', focus.lat);\nvs.var('focus:point:lon', focus.lon);\n\n// render the query\nvar rendered = q.render( vs );\n```\n\nresults in a query such as:\n\n```javascript\n{\n \"query\": {\n \"bool\": {\n \"should\": [\n {\n \"match\": {\n \"phrase.default\": {\n \"analyzer\": \"standard\",\n \"type\": \"phrase\",\n \"boost\": 1,\n \"slop\": 2,\n \"query\": \"union square\"\n }\n }\n },\n {\n \"function_score\": {\n \"query\": {\n \"match\": {\n \"phrase.default\": {\n \"analyzer\": \"standard\",\n \"type\": \"phrase\",\n \"boost\": 1,\n \"slop\": 2,\n \"query\": \"union square\"\n }\n }\n },\n \"functions\": [\n {\n \"weight\": 2,\n \"gauss\": {\n \"center_point\": {\n \"origin\": {\n \"lat\": 51.5,\n \"lon\": -0.06\n },\n \"offset\": \"10km\",\n \"scale\": \"100km\",\n \"decay\": 0.4\n }\n }\n }\n ],\n \"score_mode\": \"avg\",\n \"boost_mode\": \"replace\"\n }\n }\n ]\n }\n },\n \"size\": 10,\n \"track_scores\": true,\n \"sort\": [\n \"_score\"\n ]\n}\n```\n\n#### More Examples\n\nThe above are examples of how you can compose queries which are testable, debuggable and re-usable, they can also be mixed & matched with other queries to build even more complex queries.\n\nRather than trying to document an exhaustive list of geospatial and linguistic queries here; we have added a bunch of examples in the [examples directory](https://github.com/pelias/query/tree/master/examples).\n\nIf you have any further questions please open an issue.\n\n## Contributing\n\nPlease fork and pull request against upstream master on a feature branch.\n\nPretty please; provide unit tests and script fixtures in the `test` directory.\n\n### Running Unit Tests\n\n```bash\n$ npm test\n```\n\n### Continuous Integration\n\nCI tests every release against All supported Node.js versions.\n\n### Versioning\n\nWe rely on semantic-release and Greenkeeper to maintain our module and dependency versions.\n\n","readmeFilename":"README.md"}
![](\"https://raw.githubusercontent.com/pelias/design/master/logo/pelias_github/Github_markdown_hero.png\"){kind=logo}
\n![](\"https://github.com/pelias/query/workflows/Continuous%20Integration/badge.svg\"){kind=icon}
\n![](\"https://img.shields.io/github/license/pelias/query?style=flat&color=orange\")
![](\"https://img.shields.io/gitter/room/pelias/pelias?style=flat&color=yellow\")