## 16.2. Amazon Web Services (AWS)

This chapter describes procedures that wrap Amazon Web Services Comprehend Natural Language API.

Amazon Web Services (AWS) Comprehend Natural Language API uses machine learning to find insights and relationships in text. The procedures in this chapter act as a wrapper around calls to this API to extract entities from text stored as node properties.

Each procedure has two modes:

- Stream - returns a map constructed from the JSON returned from the API
- Graph - creates a graph or virtual graph based on the values returned by the API

This section includes the following:

- Section 16.2.1, “Procedure Overview”
- Section 16.2.2, “Install AWS Java SDK For Amazon Comprehend”
- Section 16.2.3, “Setting up API Key and Secret”
- Section 16.2.4, “Entity Extraction”

- Section 16.2.5, “Examples”

  - Section 16.2.5.1, “Entity Extraction”

### 16.2.1. Procedure Overview

The procedures are described below:

type | qualified name | signature | description
--- | --- | --- | ---
procedure | `apoc.nlp.aws.entities.graph` | `apoc.nlp.aws.entities.graph(source :: ANY?, config = {} :: MAP?) :: (graph :: MAP?)` | Creates a (virtual) entity graph for provided text
procedure | `apoc.nlp.aws.entities.stream` | `apoc.nlp.aws.entities.stream(source :: ANY?, config = {} :: MAP?) :: (node :: NODE?, value :: MAP?, error :: MAP?)` | Returns a stream of entities for provided text

### 16.2.2. Install AWS Java SDK For Amazon Comprehend

The AWS NLP procedures are compiled against version 1.11.163 of the AWS Java SDK For Amazon Comprehend library, which is not included in the APOC Library. To use the library we’ll need to download the following jar files and place them in the `plugins` directory:

- aws-java-sdk-comprehend-1.11.683.jar
- aws-java-sdk-core-1.11.683.jar
- httpclient-4.5.9.jar
- httpcore-4.4.11.jar
- joda-time-2.8.1.jar

Once we’ve done this we’ll need to restart the Neo4j server.

### 16.2.3. Setting up API Key and Secret

We can generate an Access Key and Secret by following the instructions at docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html. Once we’ve done that, we can populate and execute the following commands to create parameters that contains these details.

The following define the `apiKey` and `apiSecret` parameters.

    :param apiKey => ("<api-key-here>");
    :param apiSecret => ("<api-secret-here>");

### 16.2.4. Entity Extraction

The entity extraction procedures (`apoc.nlp.aws.entities.*`) are wrappers around the Detect Entities operations of the AWS Comprehend Natural Language API. This API method finds entities in the text, which are defined as a textual reference to the unique name of a real-world object such as people, places, and commercial items, and to precise references to measures such as dates and quantities.

The procedures are described below:

type | qualified name | signature | description
--- | --- | --- | ---
procedure | `apoc.nlp.aws.entities.graph` | `apoc.nlp.aws.entities.graph(source :: ANY?, config = {} :: MAP?) :: (graph :: MAP?)` | Creates a (virtual) entity graph for provided text
procedure | `apoc.nlp.aws.entities.stream` | `apoc.nlp.aws.entities.stream(source :: ANY?, config = {} :: MAP?) :: (node :: NODE?, value :: MAP?, error :: MAP?)` | Returns a stream of entities for provided text

The procedures support the following config parameters:

Table 16.9. Config parameters
name | type | default | description
--- | --- | --- | ---
key | String | null | AWS Access Control Key
secret | String | null | AWS Access Control Secret
nodeProperty | String | text | The property on the provided node that contains the unstructured text to be analyzed

In addition, `apoc.nlp.aws.entities.graph` supports the following config parameters:

Table 16.10. Config parameters
name | type | default | description
--- | --- | --- | ---
write | Boolean | false | persist the graph of entities
relationshipType | String | ENTITY | relationship type for relationships from source node to entity nodes

Streaming mode.

    CALL apoc.nlp.aws.entities.stream(source:Node or List<Node>, {
      key: String,
      secret: String,
      nodeProperty: String
    })
    YIELD value

Graph mode.

    CALL apoc.nlp.aws.entities.graph(source:Node or List<Node>, {
      key: String,
      secret: String,
      nodeProperty: String,
      relationshipType: String,
      write: Boolean
    })
    YIELD graph

### 16.2.5. Examples

The examples in this section are based on the following sample graph:

    CREATE (:Article {
     uri: "https://neo4j.com/blog/pokegraph-gotta-graph-em-all/",
     body: "These days I’m rarely more than a few feet away from my Nintendo Switch and I play board games, card games and role playing games with friends at least once or twice a week. I’ve even organised lunch-time Mario Kart 8 tournaments between the Neo4j European offices!"
    });

    CREATE (:Article {
     uri: "https://en.wikipedia.org/wiki/Nintendo_Switch",
     body: "The Nintendo Switch is a video game console developed by Nintendo, released worldwide in most regions on March 3, 2017. It is a hybrid console that can be used as a home console and portable device. The Nintendo Switch was unveiled on October 20, 2016. Nintendo offers a Joy-Con Wheel, a small steering wheel-like unit that a Joy-Con can slot into, allowing it to be used for racing games such as Mario Kart 8."
    });

#### 16.2.5.1. Entity Extraction

Let’s start by extracting the entities from the Article node. The text that we want to analyze is stored in the `body` property of the node, so we’ll need to specify that via the `nodeProperty` configuration parameter.

The following streams the entities for the Pokemon article.

    MATCH (a:Article {uri: "https://neo4j.com/blog/pokegraph-gotta-graph-em-all/"})
    CALL apoc.nlp.aws.entities.stream(a, {
      key: $apiKey,
      secret: $apiSecret,
      nodeProperty: "body"
    })
    YIELD value
    UNWIND value.entities AS entity
    RETURN entity;

Table 16.11. Results
entity
{score: 0.780032217502594, endOffset: 71, text: "Nintendo Switch", type: "COMMERCIAL_ITEM", beginOffset: 56}
{score: 0.8155304193496704, endOffset: 151, text: "at least", type: "QUANTITY", beginOffset: 143}
{score: 0.7507548332214355, endOffset: 156, text: "once", type: "QUANTITY", beginOffset: 152}
{score: 0.8760746717453003, endOffset: 172, text: "twice a week", type: "QUANTITY", beginOffset: 160}
{score: 0.9944096803665161, endOffset: 217, text: "Mario Kart 8", type: "TITLE", beginOffset: 205}
{score: 0.9946564435958862, endOffset: 247, text: "Neo4j", type: "ORGANIZATION", beginOffset: 242}
{score: 0.6274040937423706, endOffset: 256, text: "European", type: "LOCATION", beginOffset: 248}

We get back 7 different entities. We could then apply a Cypher statement that creates one node per entity and an `ENTITY` relationship from each of those nodes back to the `Article` node.

The following streams the entities for the Pokemon article and then creates nodes for each entity.

    MATCH (a:Article {uri: "https://neo4j.com/blog/pokegraph-gotta-graph-em-all/"})
    CALL apoc.nlp.aws.entities.stream(a, {
      key: $apiKey,
      secret: $apiSecret,
      nodeProperty: "body"
    })
    YIELD value
    UNWIND value.entities AS entity
    MERGE (e:Entity {name: entity.text})
    SET e.type = entity.type
    MERGE (a)-[:ENTITY]->(e)

Alternatively we can use the graph mode to automatically create the entity graph. As well as having the `Entity` label, each entity node will have another label based on the value of the `type` property. By default a virtual graph is returned.

The following returns a virtual graph of entities for the Pokemon article.

    MATCH (a:Article {uri: "https://neo4j.com/blog/pokegraph-gotta-graph-em-all/"})
    CALL apoc.nlp.aws.entities.graph(a, {
      key: $apiKey,
      secret: $apiSecret,
      nodeProperty: "body",
      relationshipType: "ENTITY"
    })
    YIELD graph AS g
    RETURN g;

We can see a Neo4j Browser visualization of the virtual graph in Figure 16.3, “Pokemon entities graph”.

Figure 16.3. Pokemon entities graph

We can compute the entities for multiple nodes by passing a list of nodes to the procedure.

The following returns a virtual graph of entities for the Pokemon and Nintendo Switch articles.

    MATCH (a:Article)
    WITH collect(a) AS articles
    CALL apoc.nlp.aws.entities.graph(articles, {
      key: $apiKey,
      secret: $apiSecret,
      nodeProperty: "body",
      relationshipType: "ENTITY"
    })
    YIELD graph AS g
    RETURN g

We can see a Neo4j Browser visualization of the virtual graph in Figure 16.4, “Pokemon and Nintendo Switch entities graph”.

Figure 16.4. Pokemon and Nintendo Switch entities graph

If we’re happy with this graph and would like to persist it in Neo4j, we can do this by specifying the `write: true` configuration.

The following creates a `HAS_ENTITY` relationship from the article to each entity.

    MATCH (a:Article)
    WITH collect(a) AS articles
    CALL apoc.nlp.aws.entities.graph(articles, {
      key: $apiKey,
      secret: $apiSecret,
      nodeProperty: "body",
      relationshipType: "HAS_ENTITY",
      write: true
    })
    YIELD graph AS g
    RETURN g;

We can then write a query to return the entities that have been created.

The following returns articles and their entities.

    MATCH (article:Article)
    RETURN article.uri AS article,
           [(article)-[:HAS_ENTITY]->(e) | e.text] AS entities;

Table 16.12. Results
article | entities
--- | ---
"https://en.wikipedia.org/wiki/Nintendo_Switch" | ["October 20, 2016", "Mario Kart 8", "March 3, 2017", "Nintendo Switch", "Nintendo"]
"https://neo4j.com/blog/pokegraph-gotta-graph-em-all/" | ["Mario Kart 8", "European", "once", "Neo4j", "Nintendo Switch", "at least", "twice a week"]