What is the Esqlate project

Esqlate is an attempt to build a user interface that allows the quick creation of incredibly basic admin panels for people who can either read SQL, but who cannot necessarily write it or for people who wish to create a library of SQL operations / reports for a specific project.

It aims to give a reasonable web interface which gives the user the ability to run queries while specifying the value of specific parameters (while other parameters will be restricted to the systems administrator).

Overall Data Flow

README rest diagram

To install this project

Starting a local, sample database.

First you need a PostgreSQL database, My favourite way to get a locally running PostgreSQL server is to create a simple docker-compose.yml file per project:

echo '
version: "3"
services:
  db:
    image: postgres:11.4
    environment:
    - POSTGRES_USER=postgres
    - POSTGRES_PASSWORD=postgres
    - POSTGRES_DB=postgres
    ports:
    - "127.0.0.1:5432:5432"
    volumes:
    - postgres-data:/var/lib/postgresql/data
volumes:
  postgres-data:' > docker-compose.yaml

Import a sample data set

Then import a sample data set (in this case, Microsoft’s Northwind dataset converted into PostgreSQL).

curl https://raw.githubusercontent.com/pthom/northwind_psql/5f9ba34aa1d980392685042037b0b1112d01bd32/northwind.sql | psql

Create a definition for a query you wish to run

Next, create a definition file. These definition files are like templates for queries.

mkdir -p definition
echo '
    {
        "name": "customer_search",
        "title": "Customer Search",
        "description": "List customers using a substring search",
        "variables": [{ "name": "search_string", "variable_type": "string" }],
        "statement": "SELECT * FROM customers\nWHERE\n  LOWER(company_name) LIKE CONCAT('"'%'"', LOWER($search_string), '"'%'"') OR\n  LOWER(contact_name) LIKE CONCAT('"'%'"', LOWER($search_string), '"'%'"')"
    }' > definition/customer_search.json

Lastly start the service

You are no ready to start the service:

export PGUSER=postgres
export PGPASSWORD=postgres
export PGHOST=127.0.0.1
export PGDATABASE=postgres

export DEFINITION_DIRECTORY="$PWD/definition"
export ADVERTISED_API_ROOT=http://localhost:8803 # This can cause issues with redirects in browsers. Using `/` fixes the problem, but I like full URL locations.
export LISTEN_PORT=8803

npm run-script build
npm start

Testing the API

The most basic thing you can do is get a definition:

curl -v http://localhost:8803/definition/customer_search

Suppose you want to do something useful however, such as actually run the query:

curl -v -H "Content-Type: application/json" \
    -X POST \
    --data '[{ "field_name": "search_string", "field_value": "Simon" }]' \
    http://localhost:8803/request/customer_search

The output of this will look similar to the following

> POST /request/customer_search HTTP/1.1
> Host: localhost:8803
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 59
>
* upload completely sent off: 59 out of 59 bytes
< HTTP/1.1 202 Accepted
< X-Powered-By: Express
< Access-Control-Allow-Origin: *
< Location: /request/customer_search/oDzS9suv
< Content-Type: application/json; charset=utf-8
< Content-Length: 48
< ETag: W/"30-K4lHAC8iwpUSjSDR/g3P1KLFAUU"
< Date: Tue, 24 Sep 2019 09:35:24 GMT
< Connection: keep-alive
<
{"location":"http:/localhost:8803/request/customer_search/Uz9rkntC

Using the above URL will allow you to monitor the request:

curl -v http:/localhost:8803/request/customer_search/Uz9rkntC

It is likely that your request has already completed giving you the result below, but Esqlate is designed as a Queue based system so the system administrator has some degree of control how much load you wish to put on your PostgreSQL server. If it has not yet completed you will get the resonse { "status": "pending" } and will need to re-issue the request.

> GET /request/customer_search/uQEnGH1z HTTP/1.1
> Host: localhost:8803
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< X-Powered-By: Express
< Access-Control-Allow-Origin: *
< Location: http:/localhost:8803/result/customer_search/uQEnGH1zDLaT
< Content-Type: application/json; charset=utf-8
< Content-Length: 91
< ETag: W/"5b-3tdNMonceUSkJklVx8nakJZihfY"
< Date: Tue, 24 Sep 2019 09:39:48 GMT
< Connection: keep-alive
<
{"status":"complete","location":"http:/localhost:8803/result/customer_search/Uz9rkntC9reP"}

Now you know that the request is complete and the location of the final result, you can go ahead and simply get the result:

curl http:/localhost:8803/result/customer_search/Uz9rkntC9reP
{
  "fields": [
    { "field_name": "customer_id", "field_type": "bpchar" },
    { "field_name": "company_name", "field_type": "varchar" },
    { "field_name": "contact_name", "field_type": "varchar" },
    { "field_name": "contact_title", "field_type": "varchar" },
    { "field_name": "address", "field_type": "varchar" },
    { "field_name": "city", "field_type": "varchar" },
    { "field_name": "region", "field_type": "varchar" },
    { "field_name": "postal_code", "field_type": "varchar" },
    { "field_name": "country", "field_type": "varchar" },
    { "field_name": "phone", "field_type": "varchar" },
    { "field_name": "fax", "field_type": "varchar" }
  ],
  "rows": [
    [
      "NORTS",
      "North/South",
      "Simon Crowther",
      "Sales Associate",
      "South House 300 Queensbridge",
      "London",
      null,
      "SW7 1RZ",
      "UK",
      "(171) 555-7733",
      "(171) 555-2530"
    ],
    [
      "SIMOB",
      "Simons bistro",
      "Jytte Petersen",
      "Owner",
      "Vinbæltet 34",
      "Kobenhavn",
      null,
      "1734",
      "Denmark",
      "31 12 34 56",
      "31 13 35 57"
    ]
  ],
  "status": "complete"
}