_page(count, pageRequests, id, itemJson, layerId, options, hash)

Add pages of requests through the throttle Q

parameter	type	description
count	number	the total number of features in the service
pageRequests	string	an array of page urls to be requested
id	string	id of the item
itemJson	object	the item json metadata from the server
layerId	number	the numeric id of the layer being requested
options	object	optional params from req.query (the querystring)
hash	string	the sha1 hash of the params and querystring

_removeExportDirs(dir, callback)

Method to remove all the data in each export dir this logic is being used in 4 places

parameter	type	description
dir	string	The base dir to remove all files
callback	function	The callback.

_throttleQ

A queue used to throttle requests we need this to prevent duplicate paging requests that cause features to be inserted twice

buildGeohash(params, filePath, fileName, options, callback)

Puts the dataset into a state of processes makes async call to create the geohash saves the geohash agg into the file given

parameter	type	description
params	string	request params from the url
filePath	string	the path of the file to be saved
fileName	string	the name of the file to save, determined by request params
options	object	optional params from the url querystring
callback	function	callback to call when the requests are done

csvQueue

A queue used to control the flow of the csv inserts if we get many requests for a new CSV they insert multiple times here we handle removing the data cache before we insert

dropItem(host, itemid, options, callback)

Drops the item from the cache will remove all exported files as well, but will only remove "latest" files if forceDelete is set

parameter	type	description
host	string	the host is needed to tell what dataset to remove
itemid	string	id of the item
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

featureServiceRequest(hostId, id, itemJson, hash, options, callback)

Makes a request to the feature service first checks the count and determines if koop should make one or many requests

parameter	type	description
hostId	string	the host is needed to tell what dataset to remove
id	string	id of the item
itemJson	object	json metadata from the item in the host
hash	string	the sha1 hash of the params and querystring
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

find(id, callback)

Get a service by id if no id given then return all services

parameter	type	description
id	string	the id used to reference this host in the db
callback	function	The callback.

geomTypes

Mapping between Esri geomTypes and GeoJSON types

getCount(key, options, callback)

Get the count of the features in the cache wrapper around the Cache.getCount method

parameter	type	description
key	string	a table name in the db
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getCSV(baseUrl, hostId, id, itemJson, options, callback)

Gets CSV item data from the server and inserts it into the Cache

parameter	type	description
baseUrl	string	id of the item
hostId	string	the host is needed to tell what dataset to remove
id	string	id of the item
itemJson	object	json metadata from the item in the host
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getData(itemJson, host, itemid, hostId, hash, options, callback)

This is really the main entry point to this model here we route to the correct data type method based on the item type

parameter	type	description
itemJson	object	json metadata from the item in the host
host	string	the host is needed to tell what dataset to remove
itemid	string	id of the item
hostId	string	the id of the host in the DB
hash	string	the sha1 hash of the params and querystring
options	option	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getDataFromCache(hostId, id, itemJson, hash, options, callback)

Determines if the FeatureService is already cached or not

parameter	type	description
hostId	string	the host is needed to tell what dataset to remove
id	string	id of the item
itemJson	object	json metadata from the item in the host
hash	string	the sha1 hash of the params and querystring
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getFeatureCollection(baseUrl, hostId, id, itemJson, options, callback)

Get a feature collection from the server and inserts it FeatureCollection are simply JSON features requested in one just (not services)

parameter	type	description
baseUrl	string	id of the item
hostId	string	the host is needed to tell what dataset to remove
id	string	id of the item
itemJson	object	json metadata from the item in the host
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getFeatureService(hostId, id, itemJson, hash, options, callback)

Gets the service metadata first and adds any fields to request for the actual features

parameter	type	description
hostId	string	the host is needed to tell what dataset to remove
id	string	id of the item
itemJson	object	json metadata from the item in the host
hash	string	the sha1 hash of the params and querystring
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getFeatureServiceLayerInfo(url, layer, callback)

Gets the feature service info

parameter	type	description
url	string	the max number of features in the service
layer	number	the id of the service layer
callback	function	called when the service info comes back

getInfo(key, callback)

Wraps Cache.getInfo to make testing possible w/o the cache

parameter	type	description
key	string	a table name in the db
callback	function	the callback for when all is gone

getItem(host, itemid, options, callback)

Get the service then get the item

parameter	type	description
host	string	the host is needed to tell what dataset to remove
itemid	string	id of the item
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getItemData(host, hostId, itemid, hash, options, callback)

Get the actual features and metadata from the DB

parameter	type	description
host	string	the host is needed to tell what dataset to remove
hostId	string	the id of the host in the DB
itemid	string	id of the item
hash	string	the sha1 hash of the params and querystring
options	object	optional params from req.query (the querystring)
callback	function	the callback for when all is gone

getItemMetadata(host, item, json, callback)

Get an item's metadata requests the metadata and attachs it to an object before calling the callback

parameter	type	description
host	string	the agol deploy to get data from
item	string	the agol item id
json	object	an item's json data to attach metadata to
callback	function	the callback for when all is gone

getObjectIDField(info)

Find and return the OID field from the list of fields on metadata

parameter	type	description
info	object	metadata from the service

getQueueCounts(callback)

Returns the count of jobs on each Kue queue

parameter	type	description
callback	function	a function to call when all counts have returns

isExpired(info, layerId, callback)

Checks to see if an item is expired or not

parameter	type	description
info	object	metadata from the service
layerId	number	the number of the layer in the request
callback	function	callback to call when the requests are done

register(id, host, callback)

Adds a service to the Cache needs a host, generates an id

parameter	type	description
id	string	the id used to reference this host in the db
host	string	host to request items from
callback	function	The callback.

remove(id, host, callback)

Adds a service to the Cache needs a host, generates an id

parameter	type	description
id	string	the id used to reference this host in the db
host	string	host to request items from
callback	function	The callback.

req(url, callback)

Centralized request method that forces URI encoding all ajax requests should use this so it can be tested

parameter	type	description
url	string	The url for the request.
callback	function	The callback.

requestQueue(pages, params, options, done)

Make requests for feature pages execute done when we have all features

parameter	type	description
pages	array	an array of pare URLs to request
params	object	service params
options	object	optional params for creating pages
done	function	callback to call when the requests are done

sendToWorkers(pages, params, options, done)

Create a worker job that will handle the requests

parameter	type	description
pages	array	an array of pare URLs to request
params	object	service params
options	object	optional params for creating pages
done	function	callback to call when the requests are done

stripLayerOffUrl(url, length)

Removes the layer from the end of a url

parameter	type	description
url	string	the url to strip the layer from
length	string	length of the layer value to string

_createCacheKey

Creates a unique based on request params and the querystring

Returns

_exportLarge(params)

Exports a large dataset calls the exportLarge method on the agol model

parameter	type	description
params	objects	file export parameters

_exportToFormat(params)

Exports "non-large" data to a file format

parameter	type	description
params	objects	file export parameters

_getItemData(params, options, callback)

Gets the actual json data from the model

parameter	type	description
params	object	required params used to fetch data
options	object	optional query string based params
callback	function

_setHeaders

Set file download headers for data exports adds the content-disposition and content-type bases on the file format

Returns

del(req, res)

handles a DELETE to remove a registered host from the DB

parameter	type	description
req	object	the incoming request object
res	object	the outgoing response object

deleteItemData

Forced a drop of the cache for an item and DELETEs all known files used for responding to DELETE calls.

dropItem(req, res)

Drops the cache for an item

parameter	type	description
req	object	the incoming request object
res	object	the outgoing response object

featureserver

Handles all requests for FeatureServices

find(req, res)

looks up a host based on a given id

parameter	type	description
req	object	the incoming request object
res	object	the outgoing response object

findItem(req, res)

get the item metadata from the host

parameter	type	description
req	object	the incoming request object
res	object	the outgoing response object

findItemData(req, res)

Finds the item's data this method is crux of the controller. It handles the logic for returning data in several forms:

• status processing return 202
• status is processing but a file exists
• a file exists for the data
• a new file needs to be created
• no format given, just returns json

parameter	type	description
req	object	the incoming request object
res	object	the outgoing response object

getGeohash

Get the geohash for an item/layer

getQueueCounts

Request handler for returning the counts of currently queued workers

list(req, res)

parameter	type	description
req	object	the incoming request object
res	object	the outgoing response object

preview

renders the preview map view

register(req, res)

Registers a host with the given id this inserts a record into the db for an ArcGIS instances ie: id -> hostname :: arcgis -> arcgis.com

parameter	type	description
req	object	the incoming request object
res	object	the outgoing response object

thumbnail

Handles request for thubmnails

tiles

Tile request handler, responds to z/x/y tile requests