{"_id":"esri-gnip","_rev":"40-9b92d19d88d9e93f385b6906b02985f8","name":"esri-gnip","time":{"modified":"2022-06-17T20:53:18.785Z","created":"2014-08-15T16:46:23.797Z","0.1.0":"2014-08-15T16:46:26.443Z","0.1.1":"2014-08-15T16:48:30.946Z","0.1.2":"2014-08-21T16:11:46.153Z","0.2.0":"2014-08-21T17:37:58.452Z","0.3.0":"2014-08-25T22:32:55.479Z","0.3.1":"2014-08-25T22:55:45.996Z","0.3.2":"2014-08-26T00:19:46.107Z","0.3.3":"2014-09-03T23:10:18.992Z","0.3.4":"2014-10-10T19:36:10.335Z"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}],"description":"A module to write GNIP query records to Esri FeatureServices","readme":"esri-gnip\n=========\n\nA simple node package to parse and write [Gnip JSON](http://support.gnip.com/sources/twitter/data_format.html) records into an [ArcGIS Feature Service](https://developers.arcgis.com/en/features/cloud-storage/).\n\n## Features\n* Write raw [Gnip JSON](http://support.gnip.com/sources/twitter/data_format.html) to an [ArcGIS Feature Service](https://developers.arcgis.com/en/features/cloud-storage/).\n* Automatically exclude Gnip records with no geolocation.\n* Optionally exclude Gnip records at 0,0.\n* Optionally parse Gnip JSON to ArcGIS JSON without writing.\n* Supports ArcGIS token-based authentication.\n\nSee [gnip-reader](https://www.npmjs.org/package/gnip-reader) to obtain Gnip JSON records from the [Gnip Search API](http://support.gnip.com/apis/search_api/api_reference.html#SearchRequests).\n\n## Requirements\n1. Gnip data - A set of Gnip records in [Gnip JSON format](http://support.gnip.com/sources/twitter/data_format.html), retrieved from the [Gnip APIs](http://support.gnip.com/apis/). See also [gnip-reader](https://www.npmjs.org/package/gnip-reader). Contact these [guys](http://gnip.com/) if you need to get data.\n2. A pre-configured [ArcGIS Online Feature Service](https://developers.arcgis.com/en/features/cloud-storage/) to store the data. See details below to configure your [target Feature Service](#creating-a-target-feature-service).\n3. [node.js](http://nodejs.org)\n\n## Usage\n### Installing\n    $ npm install esri-gnip\n\n### Writing to an ArcGIS Feature Service\nYou must have the URL to a target Feature Service with editing enabled and a specific set of attributes (see [Creating a target Feature Service](#creating-a-target-feature-service) below).\n\nOnce you have that URL, initialize the esri-gnip writer and write records like this:\n\n``` JavaScript\nvar esriGnip = require('esri-gnip');\n\n// The REST Endpoint URL of the target Feature Service\nvar featureServiceURL = 'http://services.arcgis.com/...../arcgis/rest/services/Gnip/FeatureServer/0';\n\nvar myGnip = new esriGnip.Writer(featureServiceURL, function(err, metadata) {\n  if (err) {\n    console.error(err);\n  } else {\n    this.postGnipRecordsToFeatureService([ /* array of gnip records */ ], function(err, results) {\n      if (err) {\n        console.error(err);\n      } else {\n        console.log(results);\n      }\n    });\n  }\n});\n```\n\nThe callback is called once the writer is initialized and ready to accept Gnip records. Within the callback, `this` will refer to the newly created writer object.\n\nThere is also an `initialized` property on the writer which can be used to determine if the writer is ready. It is recommended you use the callback pattern where possible.\n\n#### Output\nThe `results` callback parameter will look like this:\n\n``` JavaScript\n{\n  arcgisRecords: [],    // arcGISRecords\n  unlocated: [],        // gnipRecords\n  translationErrors: [] // translationErrors\n}\n```\n\nA `translationError` is structured as follows:\n\n``` JavaScript\n{\n  translationError: { \n    message: '<string>', \n    stack: '<string>'\n  },\n  record: { /* a Gnip record */ }\n}\n```\n\n#### Options\nYou can pass in an options object instead of a URL as the first parameter to `new esriGnip.Writer()`. The options object must contail a `url` property:\n``` JavaScript\nvar esriGnip = require('esri-gnip');\n\nvar options = {\n  // The REST Endpoint URL of the target Feature Service\n  url: 'http://services.arcgis.com/...../arcgis/rest/services/Gnip/FeatureServer/0'\n};\n\nvar myGnip = new esriGnip.Writer(options, function(err, metadata) {\n  // See above\n});\n```\n\n**Note**: you should avoid sending too many Gnip records in a single call to `.postGnipRecordsToFeatureService()` to avoid the POST call becoming too large. While the Feature Service will likely handle it, it could cause HTTP timeouts. Informal tests have shown that 50-100 records per post is a reasonable and very safe limit.\n\n### Authentication\nThe options parameter can optionally contain a `token` property for authentication with the target Feature Service, in case that feature service is not public:\n\n``` JavaScript\nvar esriGnip = require('esri-gnip');\n\nvar options = {\n  // The REST Endpoint URL of the target Feature Service\n  url: 'http://services.arcgis.com/...../arcgis/rest/services/Gnip/FeatureServer/0',\n  token: '<some-authentication-token>'\n};\n\nvar myGnip = new esriGnip.Writer(options, function(err, metadata) {\n  // See above\n});\n```\n\n### Excluding records at [0,0]\nAll too often a poorly geolocated record will have a coordinate of [0,0]. It is very rare that this is actually the correct coordinate. You can exclude these records from consideration by setting the `excludeNullIslands` property of the options parameter to `true`:\n\n``` JavaScript\nvar esriGnip = require('esri-gnip');\n\nvar options = {\n  // The REST Endpoint URL of the target Feature Service\n  url: 'http://services.arcgis.com/...../arcgis/rest/services/Gnip/FeatureServer/0',\n  excludeNullIslands: true\n};\n\nvar myGnip = new esriGnip.Writer(options, function(err, metadata) {\n  // See above\n});\n```\n\nIf `excludeNullIslands` is set to `true`, records with coordinates of [0,0] will be included in the `unlocated` output property and will not be added to the target feature service.\n\n### Parsing Gnip JSON records\nTo simply parse an array of Gnip records without posting to an ArcGIS FeatureService, call `parse()`:\n\n``` JavaScript\nvar esriGnip = require('esri-gnip');\nvar output = esriGnip.parse([ /* array of gnip records */ ]);\n```\n\nWhere `output` will look like the `results` callback parameter in `.postGnipRecordsToFeatureService()` described above.\n\nYou can optionally reject Gnip records that have a location of [0,0] by passing `true` as the second parameter to `.parse()`. Such records will be included in the `unlocated` output property.\n\n## Creating a target Feature Service\nA target feature service of the correct schema is required to write Gnip records to. Follow these instructions to create such a feature service from the [pre-created template](http://services.arcgis.com/OfH668nDRN7tbJh0/arcgis/rest/services/Gnip/FeatureServer/0).\n\n1. Log in [here](https://www.arcgis.com/home/signin.html) with your ArcGIS Online account.\n2. Browse to `My Content`, then select `Create Layer`\n3. Choose `an existing feature layer` in the pulldown.\n4. Select the `Enter a URL to a feature layer` radio button.\n5. Paste [http://services.arcgis.com/OfH668nDRN7tbJh0/arcgis/rest/services/Gnip/FeatureServer/0](http://services.arcgis.com/OfH668nDRN7tbJh0/arcgis/rest/services/Gnip/FeatureServer/0) into the `Url` field.\n6. (Optional) Rename the layer.\n7. Click `Next`\n8. Click `Next` to accept the default extent.\n9. Enter a `Title`\n10. Enter at least one `Tag`\n11. (Optional) update the `Summary` and/or `Folder`\n12. Click `Done`\n\nYou will be taken to the new Feature Service's *item page*. To get the REST Endpoint URL, click the layer name in the `Layers` section (or click the caret next to it to display the popup menu and select `Service URL`). The URL of the newly opened page is the REST Endpoint needed by esri-gnip and will look similar to the URL in Step 5 above.\n\nNOTE: Go here to get a free [ArcGIS Developer subscription](https://developers.arcgis.com/en/) or a trial [ArcGIS Online Organization subscription](http://www.arcgis.com/home/). \n\n\n## Resources\n\n* [ArcGIS Feature Service overview](https://developers.arcgis.com/en/features/cloud-storage/)\n* [ArcGIS Feature Service documentation](http://resources.arcgis.com/en/help/arcgis-rest-api/#/Feature_Service/02r3000000z2000000/)\n* [ArcGIS REST Specification](http://resources.arcgis.com/en/help/arcgis-rest-api/)\n* [node.js documentation](http://nodejs.org/api/)\n* [gnip-reader](https://www.npmjs.org/package/gnip-reader)\n* [Terraformer](https://github.com/esri/terraformer)\n* [Geoservices-js](https://github.com/Esri/geoservices-js)\n* [Gnip Documentation](http://support.gnip.com/sources/twitter/data_format.html) (incomplete)\n* [Gnip Test Records](example/example-gnip-data.json)\n\n## Notes\n* Gnip records without location information are not added.\n* Gnip records with coordinates of [0,0] can optionally be considered to have no location (see [Excluding records at [0,0]](#excluding-records-at-00)  and [Parsing Gnip JSON records](#parsing-gnip-json-records) above).\n* Advanced users can use the `Gnip.sd` Service Definition file included in this repo to create a suitable target Feature Service. `Gnip.sd` can be used with ArcGIS Online or ArcGIS Server. It will create a Feature Service named `Gnip`, so ensure there is no pre-existing Feature Service of that name before use.\n\n## Issues\n\nFind a bug or want to request a new feature?  Please let us know by submitting an Issue.\n\n## Contributing\n\nAnyone and everyone is welcome to contribute.\n\n## Licensing\nCopyright 2014 Esri\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n   http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nA copy of the license is available in the repository's [license.txt](license.txt) file.\n[](Esri Tags: NodeJS GeoServices REST Gnip)\n[](Esri Language: JavaScript)\n","versions":{"0.1.1":{"name":"esri-gnip","version":"0.1.1","description":"A service to transfer GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/nixta/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~0.2.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","bugs":{"url":"https://github.com/nixta/esri-gnip/issues"},"_id":"esri-gnip@0.1.1","dist":{"shasum":"e16e151106666b85866c1a85d0acf411ad9c6cad","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.1.1.tgz","integrity":"sha512-UOglTZRbVsV3IPK8EkPjFmc5byWcxVRc/XXk3Lu27DIPpmZ5b7vIry7ASGT7Qc8ycp2lOW9cOPqpThtomI7Drw==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIQDN6BOZgKBUN2JHFhvqugOazsJeMJM3u9PhnBWuvRgN+wIgAyj4zm+t2fXLe9M2Q0ua1uBg6XNOzy+qL6mTBPpx13g="}]},"_from":".","_npmVersion":"1.3.8","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}]},"0.1.2":{"name":"esri-gnip","version":"0.1.2","description":"A service to transfer GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/ArcGIS/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~0.2.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","bugs":{"url":"https://github.com/ArcGIS/esri-gnip/issues"},"_id":"esri-gnip@0.1.2","dist":{"shasum":"56758988d60aeb8af512e1705339888036ce4a90","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.1.2.tgz","integrity":"sha512-CWFwajRqE0x0MR+L99clp0H6da0O5UsfNq+/6kPKkKrUEF4KgsYXsvaBzbOM5qINFkB3+q84KZLjxTz2V3N+zQ==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIQCJpe/T3CDFbO1CRQsfCp7Jf5lPyPu4Q+hPiSjHny1NJwIganOyOLgXdQecbQxDhZaGxrXc+AsK/TwTmdm07mmZI70="}]},"_from":".","_npmVersion":"1.3.8","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}]},"0.2.0":{"name":"esri-gnip","version":"0.2.0","description":"A service to transfer GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/ArcGIS/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~1.0.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","bugs":{"url":"https://github.com/ArcGIS/esri-gnip/issues"},"_id":"esri-gnip@0.2.0","dist":{"shasum":"2730434b21953aaeecaa2ba7738ded3a5aad6e3d","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.2.0.tgz","integrity":"sha512-v+/pIJtFmC35CbsBefs6KAkPE8uFQfC41x5jHElTIUhQVISo5U47PCV01nBUjVwP0CyVyZgXswAWtYn/2sqROQ==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIHBJsJvfZ6zF4eNhyBaLrX/yGVcTxS1ciKBAeMi2/0xPAiEAhWhu3l068L1W2eDNbXbZJ4TsxAQ4m47M633MgOqShe0="}]},"_from":".","_npmVersion":"1.3.8","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}]},"0.3.0":{"name":"esri-gnip","version":"0.3.0","description":"A service to transfer GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/ArcGIS/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~1.0.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","bugs":{"url":"https://github.com/ArcGIS/esri-gnip/issues"},"_id":"esri-gnip@0.3.0","dist":{"shasum":"eaa6478b2b63107e9cb33e2c0c557b38d73b0b64","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.3.0.tgz","integrity":"sha512-dbzCwq39Z0Hv7GmK/KymUNQxBaGCTDn/jWGITg45OYUqQTNbc9x5uJ4tZRcrFIdwA3QGV84m4+e7yWK7c9VIvw==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIFWoiiTYEYWiJrL/eY/ShlmZlC8LfPKD7tWLpvpVm2r8AiEA+0vNRRf6VWAlSGzdQm2r3yEzvD1pR1SlM5l0keWb8S4="}]},"_from":".","_npmVersion":"1.3.8","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}]},"0.3.1":{"name":"esri-gnip","version":"0.3.1","description":"A service to transfer GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/ArcGIS/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~1.0.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","bugs":{"url":"https://github.com/ArcGIS/esri-gnip/issues"},"_id":"esri-gnip@0.3.1","dist":{"shasum":"ca2af64b2139a122999f71ddf4820a16cd56efb0","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.3.1.tgz","integrity":"sha512-V843OrsjGlGm75bPmeAK9J2F51EFN/86Vh4c4GSuRL7ivb+ZqN6ji4dcC7jJC/dORBHOseVuGYS4jsVtefBVow==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEQCIGJm9AZc2IM6xgJu9Ej/Nh+oGaW72iDOSGOowL5YkmmvAiB58pCc+lYvezyb13+f8G1s+J0GHpdNuQt8ffDAeTHCxg=="}]},"_from":".","_npmVersion":"1.3.8","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}]},"0.3.2":{"name":"esri-gnip","version":"0.3.2","description":"A service to transfer GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/ArcGIS/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~1.0.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","bugs":{"url":"https://github.com/ArcGIS/esri-gnip/issues"},"_id":"esri-gnip@0.3.2","dist":{"shasum":"7075955433d0ab1750a02b04e3f45f422c0b63c3","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.3.2.tgz","integrity":"sha512-vbWuRWzIg8IGJ18bInkOvp5TTkArPpPtaFOXgaoJyVHb+XEgx//J4LYeprRQZBN12cJLpjnkXdvpB5dons9KXA==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIQDHkjy0nOUnFr1OQ/kd1lS7mfGKURuEzdAR/BRbnJxxzAIgXs57YUwKlbgX3Hfv5amgvq8MfHEF3OdEEHwIZP74s2Q="}]},"_from":".","_npmVersion":"1.3.8","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}]},"0.3.3":{"name":"esri-gnip","version":"0.3.3","description":"A service to transfer GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/ArcGIS/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~1.0.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","gitHead":"ca9347b0368c53d32c91281a57bed8d842850aa2","bugs":{"url":"https://github.com/ArcGIS/esri-gnip/issues"},"homepage":"https://github.com/ArcGIS/esri-gnip","_id":"esri-gnip@0.3.3","scripts":{},"_shasum":"85eb76c423d01bd39c2b87d9f67bb5d9e4837788","_from":".","_npmVersion":"1.4.23","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}],"dist":{"shasum":"85eb76c423d01bd39c2b87d9f67bb5d9e4837788","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.3.3.tgz","integrity":"sha512-uI15/MAWXCXHqwgHNQmWxzZ5om0MFLk/U0EetYD4AqObznG+5XndA9XWEvAPQrhhKFXmkzQRyG5A4E40Rxt7uA==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIQCmHXvNPHXiIF1wOMKYH9YyDZ06TVBLWQ3tvceEBw8+kQIgB1edlLRV7DRIegyCnXUpYHOPpoH9dXfKEYt2pTPpG8c="}]}},"0.3.4":{"name":"esri-gnip","version":"0.3.4","description":"A module to write GNIP query records to Esri FeatureServices","main":"index.js","repository":{"type":"git","url":"https://github.com/Esri/esri-gnip.git"},"dependencies":{"terraformer":"~1.0.4","terraformer-arcgis-parser":"~1.0.4","geoservices":"~1.0.0","lodash":"~2.4.1"},"author":{"name":"Nick Furness"},"license":"Apache","gitHead":"b3018dfcaab8295554b8f1500b0972dab6590265","bugs":{"url":"https://github.com/Esri/esri-gnip/issues"},"homepage":"https://github.com/Esri/esri-gnip","_id":"esri-gnip@0.3.4","scripts":{},"_shasum":"cbeea4870f6c41640c051d069a27142eeeab8d16","_from":".","_npmVersion":"1.4.28","_npmUser":{"name":"nixta","email":"nfurness@nixta.com"},"maintainers":[{"name":"nixta","email":"nfurness@nixta.com"}],"dist":{"shasum":"cbeea4870f6c41640c051d069a27142eeeab8d16","tarball":"https://registry.npmjs.org/esri-gnip/-/esri-gnip-0.3.4.tgz","integrity":"sha512-TVtKrVRFIRonKiAMPXfyGXkwk8T/a0pkHd/U2qhTEqRzAAwg6DR1qLNkyD8KcmACbLBJAuGTCxC6AyOUoqKVjQ==","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEQCIChjbhzjeyrN5Dec5okhWf0J9GlV16R5AYvZE6szcr+hAiAffgQiameBP/rgFLNfaXPVMiATD582KnNDdBmYLEqQiA=="}]}}},"dist-tags":{"latest":"0.3.4"},"readmeFilename":"README.md","repository":{"type":"git","url":"https://github.com/Esri/esri-gnip.git"},"author":{"name":"Nick Furness"},"bugs":{"url":"https://github.com/Esri/esri-gnip/issues"},"license":"Apache","homepage":"https://github.com/Esri/esri-gnip"}