# AWS EC2

## Table of Contents 

  - [Specific Adapter Information](#specific-adapter-information) 
    - [Authentication](#authentication) 
    - [Sample Properties](#sample-properties) 
    - [Swagger](#swagger) 
  - [Generic Adapter Information](#generic-adapter-information) 

## Specific Adapter Information
### Authentication

This document will go through the steps for authenticating the AWS EC2 adapter with AWS Signature 4 Authentication. Properly configuring the properties for an adapter in Itential Platform is critical for getting the adapter online. You can read more about adapter authentication <a href="https://docs.itential.com/opensource/docs/authentication" target="_blank">HERE</a>. 

#### AWS Authentication
The AWS EC2 adapter requires AWS Authentication therefore the `auth_method` should be set to `aws_authentication`. The adapter utilizes AWS signature 4 authentication. There are three mechanisms for doing this. There is a brief description in this section and more information in the specific section for each authentication.

The first way is using a "service" account and its AWS keys to authenticate as that account. In this case, you will get the aws_access_key, aws_secret_key, and aws_session_token from AWS and configure them into the adapter service instance as shown below.

The second way is using AWS STS. this still requires a "service" account and its AWS keys to authenticate as that account. In this case, you will get the aws_access_key, aws_secret_key, and aws_session_token from AWS and configure them into the adapter service instance as shown below. In addition, you will provide STS paramaters in the workflow tasks that tell the adapter the role you want used on the particular call.

The third authentication method is to use an IAM role. With this method, you do not need any authentication keys as the adapter will utilize an "internal" AWS call to get the things that it needs for authentication. Since the adapter needs to make the call to this "internal" AWS IP address, the Itential Platform server needs to be where it has access to that address or you will not be able to use this method.

If you change authentication methods, you should change this section accordingly and merge it back into the adapter repository.

#### AWS Signature 4 Service Account Authentication
This can also be referred to as standard credential signing using an Adapter Role or Pod Role.

Provide access key and secret key in adapter service instance configuration (Adapter Role). Those credentials are used to sign all calls. If no access key and/or secret key are provided, it will use AWS Environment variables (which in Saas is the Pod Role) to sign all calls. If this role has no permission to make the call, the call will fail with authentication issues.

STEPS  
1. Ensure you have access to a AWS EC2 server and that it is running
2. Follow the steps in the README.md to import the adapter into Itential Platform if you have not already done so
3. Use the properties below for the ```properties.authentication``` field
```json
"authentication": {
  "auth_method": "aws_authentication",
  "aws_access_key": "aws_access_key",
  "aws_secret_key": "aws_secret_key",
  "aws_session_token": "aws_session_token"
}
```
you can leave all of the other properties in the authentication section, they will not be used for AWS EC2 authentication.
4. Restart the adapter. If your properties were set correctly, the adapter should go online. 

#### AWS Security Token Service
The AWS EC2 adapter also supports AWS Security Token Service (STS) Authentication. This can also be referred to as standard credential but then STS assume role based on a global role or a task role provided in the STS Params on Task.
Adapter Role => Global or Task Role(s) or Pod Role => Task Role(s).

Provide access key and secret key in adapter service instance configuration (Adapter Role). Will use these credentials on the assume role request to assume a different role defined in the Global Role or STSParams on the task (Task Role). The Task Role will then be used to sign the call to AWS. The Adapter Role has to have the right to assume the Task Role or the call will fail. If no access key and/or secret key are provided, it will use AWS Environment variables (which is Saas is the Pod Role) to attempt the assume a different role defined in the STSParams on the task (Task Role). The Task Role will then be used to sign the call to AWS. The Pod Role has to have the right to assume the Task Role or the call will fail.

For using the global role you need to define the global role in the adapter service instance configuration.

```json
"authentication": {
  "aws_iam_role": "arn:aws:iam::1234567:role/my_role"
}
```

For using this authentication dynamically on a task by task basis, you need to use the calls in the Adapter that have the STSRole suffix on them and pass the STS information into the method. You will still need to provide the relevant `aws_secret_key` and `aws_access_key` as described above. Below is an example of the data required in the `sts` tasks:

```json
{
  "RoleArn": "arn:aws:iam::1234567:role/my_role",
  "RoleSessionName": "mySession"
}
```

The AWS STS Authentication goes to the AWS STS Service endpoint in order to validate that the primary "service" account the adapter has authenticated with has the permission to assume the role. This call is made to sts.amazonaws.com or a regional sts sevice (e.g. sts.us-east-1.amazonaws.com). By default traffic to these endpoints will go out through the Internet. In the case where you would prefer these route through your network, it is possible to change the STS config for the adapter. 

The proxy field should point to the AWS loadbalancer or a proxy server that forwards to AWS STS. In Itential Cloud, this can be NAT'd to your network. In addition to this, you may need to set the endpoint in order to have the STS SSL certificate validated successfully. By default the adapter will use sts regional servers. If the loadbalancer and proxy are set up for that you should be fine. If however, they point to the global STS service (sts.amazonaws.com) You will need to set the global as the endpoint or the STS certificate will be rejected due to the hosts not matching. 

Region can be important as it is the region in which the STS assume role request will be processed. Each AWS partition may have one primary region for STS. In our tests, we have found that for the standard partition the STS region should be set to us-east-t but this is configurable should your primary region be different or you are working in a different AWS partition.

If doing cross organizational role assumption it is recommended to have an external id that is agreed upon for further security. THis external id can be provided in the aws_sts properties (global) or defined in the STS Params object on the task (dynamic).

```json
"authentication": {
  "aws_sts": {
    "region": "us-east-1",
    "endpoint": "<sts certificate endpoint>",
    "proxy": "<proxy/loadbalancer ip>",
    "externalId": "<sts external id>"
  }
}
```

#### AWS IAM Role
The AWS EC2 adapter also supports AWS IAM Role Authentication. For using this authentication, you need to use the calls in the Adapter that have the STSRole suffix on them and provide the role's ARN in the RoleName variable. In addition to passing the IAM Role in the task, it is possible to set an IAM Role in the Service Instance Configuration by using the `aws_iam_role` property in the authentication section and providing the role's ARN.

This can also be referred to as role signing scenarios - need to have either a Task Role (roleName) and/or Adapter Role (aws_iam_role) to use ay of these scenarios.

Scenarios:
- IAM to internal AWS Server - either Task Role (roleName) or Adapter Role (aws_iam_role)
- Adapter Role (aws_iam_role) assumes Task Role (STSParams or roleName)
  - IAM to internal AWS Server for Adapter Role
  - AWS STS for assuming Task Role(s) using Adapter Role
- Pod Role assumes Adapter Role (aws_iam_role) assumes Task Role (STSParams, RoleName)
  - AWS STS for assuming Adapter Role using AWS Environment (Pod Role)
  - AWS STS for assuming Task Role(s) using Adapter Role

```json
"authentication": {
  "auth_method": "aws_authentication",
  "aws_iam_role": "role_arn"
}
```

#### AMAZON STEPS FOR IAM ROLE
Increase number of hops if running Itential Platform inside of docker on EC2 instance
```bash
aws sso login --profile aws-bota-1
<export aws keys for CLI access>

aws ec2 modify-instance-metadata-options  --instance-id i-0e150236026b7c45d  --http-put-response-hop-limit 3 --http-endpoint enabled --region us-east-1
```

Create a new role and attach to it policies:
- go to your EC2 instance, select it
- Actions->Security->Modify IAM Role
- Click 'Create New IAM Role'
- Create a role:
```text
Trusted entity type: AWS service
Use Case: EC2
```

Add desired policies to the role. 

Save the role

Go back to your EC2 instance and Actions->Security->Modify IAM Role, associate newly created role with your EC2 instance

#### Troubleshooting
- Make sure you copied over the correct access key, secret key and session token.
- Turn on debug level logs for the adapter in Itential Platform Admin Essentials.
- Turn on auth_logging for the adapter in Itential Platform Admin Essentials (adapter properties).
- Investigate the logs - in particular:
  - The FULL REQUEST log to make sure the proper headers are being sent with the request.
  - The FULL BODY log to make sure the payload is accurate.
  - The CALL RETURN log to see what the other system is telling us.
- Credentials should be ** masked ** by the adapter so make sure you verify the username and password - including that there are erroneous spaces at the front or end.
- Remember when you are done to turn auth_logging off as you do not want to log credentials.
- For IAM, you can run this on the Itential Platform server to verify you are getting to the "internal" AWS Server
```bash
TOKEN=`curl -v -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` && curl -v -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/<rolename>
```
### Sample Properties

Sample Properties can be used to help you configure the adapter in the Itential Platform. You will need to update connectivity information such as the host, port, protocol and credentials.

```json
  "properties": {
    "host": "ec2.us-east-1.amazonaws.com",
    "region": "us-east-1",
    "port": 443,
    "choosepath": "",
    "base_path": "/",
    "version": "",
    "cache_location": "none",
    "encode_pathvars": true,
    "encode_queryvars": true,
    "save_metric": false,
    "stub": true,
    "protocol": "https",
    "service": "ec2",
    "xmlArrayKeys": [
      "item"
    ],
    "authentication": {
      "auth_method": "aws_authentication",
      "username": "username",
      "password": "password",
      "token": "token",
      "token_timeout": 180000,
      "token_cache": "local",
      "invalid_token_error": 401,
      "auth_field": "header.headers.Cookie",
      "auth_field_format": "Token {token}",
      "auth_logging": false,
      "client_id": "",
      "client_secret": "",
      "grant_type": "",
      "auth_request_datatype": "",
      "auth_response_datatype": "",
      "token_response_placement": "",
      "sensitive": [],
      "sso": {
        "protocol": "",
        "host": "",
        "port": 0
      },
      "multiStepAuthCalls": [
        {
          "name": "",
          "requestFields": {},
          "responseFields": {},
          "successfullResponseCode": 200
        }
      ],
      "aws_access_key": "aws_access_key",
      "aws_secret_key": "aws_secret_key",
      "aws_session_token": "aws_session_token",
      "aws_iam_role": "",
      "aws_sts": {
        "region": "us-east-1",
        "sslEnable": true,
        "endpoint": "",
        "proxy": "",
        "proxyagent": "",
        "use_proxy_for_initial_auth": false,
        "externalId": ""
      }
    },
    "healthcheck": {
      "type": "startup",
      "frequency": 60000,
      "query_object": {
        "Action": "DescribeRegions",
        "Version": "2016-11-15"
      },
      "addlHeaders": {}
    },
    "throttle": {
      "throttle_enabled": false,
      "number_pronghorns": 1,
      "sync_async": "sync",
      "max_in_queue": 1000,
      "concurrent_max": 1,
      "expire_timeout": 0,
      "avg_runtime": 200,
      "priorities": [
        {
          "value": 0,
          "percent": 100
        }
      ]
    },
    "request": {
      "number_redirects": 0,
      "number_retries": 3,
      "limit_retry_error": 0,
      "failover_codes": [],
      "attempt_timeout": 5000,
      "global_request": {
        "payload": {},
        "uriOptions": {},
        "addlHeaders": {},
        "authData": {}
      },
      "healthcheck_on_timeout": true,
      "return_raw": false,
      "archiving": false,
      "return_request": false,
      "keep_alive_interval": 0
    },
    "proxy": {
      "enabled": false,
      "host": "",
      "port": 1,
      "protocol": "http",
      "username": "",
      "password": ""
    },
    "ssl": {
      "ecdhCurve": "",
      "enabled": false,
      "accept_invalid_cert": false,
      "ca_file": "",
      "ca_file_content": "",
      "key_file": "",
      "cert_file": "",
      "secure_protocol": "",
      "ciphers": ""
    },
    "mongo": {
      "url": "",
      "host": "",
      "port": 0,
      "database": "",
      "dbAuth": false,
      "username": "",
      "password": "",
      "replSet": "",
      "addSrv": false,
      "db_ssl": {
        "enabled": false,
        "accept_invalid_cert": false,
        "ca_file": "",
        "key_file": "",
        "cert_file": ""
      }
    },
    "devicebroker": {
      "enabled": true,
      "getDevice": [
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpcs",
            "Version": "2016-11-15",
            "VpcId.1": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "DescribeVpcsResponse.vpcSet.item",
          "responseFields": {
            "name": "{vpcId}",
            "ostype": "{vpc}",
            "ostypePrefix": "aws-",
            "ipaddress": "{cidrBlock}",
            "port": "n/a"
          }
        }
      ],
      "getDevicesFiltered": [
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpcs",
            "Version": "2016-11-15"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "DescribeVpcsResponse.vpcSet.item",
          "responseFields": {
            "name": "{vpcId}",
            "ostype": "{vpc}",
            "ostypePrefix": "aws-",
            "ipaddress": "{cidrBlock}",
            "port": "n/a",
            "vpcId": "{vpcId}"
          }
        }
      ],
      "isAlive": [
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpcs",
            "Version": "2016-11-15",
            "VpcId.1": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "DescribeVpcsResponse.vpcSet.item",
          "responseFields": {
            "status": "{state}",
            "statusValue": "available"
          }
        }
      ],
      "getConfig": [
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeInternetGateways",
            "Version": "2016-11-15",
            "Filter.1.Name": "attachment.vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeSubnets",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeRouteTables",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeNetworkAcls",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpcPeeringConnections",
            "Version": "2016-11-15",
            "Filter.1.Name": "requester-vpc-info.vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpcEndpoints",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeNatGateways",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeSecurityGroups",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeInstances",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpnConnections",
            "Version": "2016-11-15",
            "VpnConnectionId.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpnGateways",
            "Version": "2016-11-15",
            "Filter.1.Name": "attachment.vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeNetworkInterfaces",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        },
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpcs",
            "Version": "2016-11-15",
            "Filter.1.Name": "vpc-id",
            "Filter.1.Value": "{vpcId}"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "",
          "responseFields": {}
        }
      ],
      "getCount": [
        {
          "path": "/",
          "method": "GET",
          "query": {
            "Action": "DescribeVpcs",
            "Version": "2016-11-15"
          },
          "body": {},
          "headers": {},
          "handleFailure": "ignore",
          "requestFields": {},
          "responseDatakey": "DescribeVpcsResponse.vpcSet.item",
          "responseFields": {
            "name": "{vpcId}",
            "ostype": "vpc",
            "ostypePrefix": "aws-",
            "ipaddress": "{cidrBlock}",
            "port": "n/a",
            "vpcId": "{vpcId}"
          }
        }
      ]
    },
    "cache": {
      "enabled": false,
      "entities": [
        {
          "entityType": "device",
          "frequency": 3600,
          "flushOnFail": false,
          "limit": 1000,
          "retryAttempts": 5,
          "sort": true,
          "populate": [
            {
              "path": "/",
              "method": "GET",
              "pagination": {
                "offsetVar": "",
                "limitVar": "",
                "incrementBy": "limit",
                "requestLocation": "query"
              },
              "query": {
                "Action": "DescribeVpcs",
                "Version": "2016-11-15"
              },
              "body": {},
              "headers": {},
              "handleFailure": "ignore",
              "requestFields": {},
              "responseDatakey": "result.response.DescribeVpcsResponse.vpcSet.item",
              "responseFields": {
                "cidrBlock": "{cidrBlock}",
                "name": "{vpcId}",
                "ostype": "{vpc}",
                "ostypePrefix": "aws-",
                "ipaddress": "{cidrBlock}",
                "port": "n/a",
                "vpcId": "{vpcId}"
              }
            }
          ],
          "cachedTasks": [
            {
              "name": "",
              "filterField": "",
              "filterLoc": ""
            }
          ]
        }
      ]
    }
  }
```
### [Swagger](https://gitlab.com/itentialopensource/adapters/adapter-aws_ec2/-/blob/master/report/adapter-openapi.json) 

## [Generic Adapter Information](https://gitlab.com/itentialopensource/adapters/adapter-aws_ec2/-/blob/master/README.md) 

