{"_id":"@bluefinenterprises/serverless-aws-batch","_rev":"16-554a31f6fdba2d6c5981a216a6b563c5","name":"@bluefinenterprises/serverless-aws-batch","dist-tags":{"latest":"1.0.0"},"versions":{"1.0.0":{"name":"@bluefinenterprises/serverless-aws-batch","version":"1.0.0","keywords":["serverless","aws","batch","python"],"author":{"name":"Justin Ramsey"},"license":"MIT","_id":"@bluefinenterprises/serverless-aws-batch@1.0.0","maintainers":[{"name":"bluefinenterprises","email":"ronak@bluefinenterprises.com"}],"homepage":"https://github.com/bluefinenterprises/serverless-aws-batch#readme","bugs":{"url":"https://github.com/bluefinenterprises/serverless-aws-batch/issues"},"dist":{"shasum":"747d18d08a96cd91709fffc948c5d4725e467e95","tarball":"https://registry.npmjs.org/@bluefinenterprises/serverless-aws-batch/-/serverless-aws-batch-1.0.0.tgz","fileCount":12,"integrity":"sha512-bZ7YB3GtFHh9tm5XRzlRVIDkRc2qKG5zkxbpxHjJxYa8RDR3vU7TtLVG8dWtoHHAsoshi9GWqy6hHLRWvN+HjQ==","signatures":[{"sig":"MEUCIQDNu7idahvBN8qcoUWljPYwVtHh7vC11kO7NgroAJobCwIgYH/FalNJKtQCTT3NkZ2jbvZ6nwGWwrBimBXIrvRgfVM=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":52548,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v4.10.10\r\nComment: https://openpgpjs.org\r\n\r\nwsFzBAEBCAAGBQJiKLcBACEJED1NWxICdlZqFiEECWMYAoorWMhJKdjhPU1b\r\nEgJ2VmqKEQ/+LIAcLBwz1I4oh8htWFmZMGB1z2qt77bFqbLepGnI5sE+1eQM\r\nIx1zgglBcfUiyEde83QREKIvDHBF4KtnEQUmdnZZupO7Xpv5SzNF6oQ8EK2v\r\nMZouyVPzwlUjHcYLXTKtyBoh321R9HYj4cCB8JdLI+NYY6SxEJFjr23Yuwl8\r\nO0qK7rPCFXmB5hr72vqT1pG2ZuH2St03nGT2UHH5pyCDW2Lv6czDeQaWxDgO\r\nvr+f9S3lxQ9vHtxN0qPC6dL3j0tB9+H9Q+TxtaoZTaNrWC5oXb95RKUD/Ffc\r\nNnn0M7h/pjldDanbUSpCL/jLlcFjtgw0FCvPP6PQZkDdDhIxWGDR2wYnoPbY\r\n4NuexbNuNAmqT7bV4/eALJxD2OXw9ad2DmPBcsg95JX1P8EcZt1FxaArddkB\r\nZ/3mY4eC1UHdfkeCJPb2pUgqvQQH7sRmGKiuNtJ4DTuEV5qemDQP/bmlvArS\r\nTLALEYoLY0dhki3fwg4KBle8ebZkm3RECEorO05lj/444FkazqvqovWb886U\r\n+h3pAdMVLujAoY5XGZEfSOo5tB+lk4/h7ScCvFzck0YybIRZDuiqW+nwfj7k\r\nBEXcZLbxqg0/Rq2UYjOZrobzu3toRVEY0gnjX1Npsq5B9zRPf9tuZZ06IFSm\r\nt9gdPA2qKtvyseewHjxgdK4T5g6muxOL1Gc=\r\n=ilin\r\n-----END PGP SIGNATURE-----\r\n"},"main":"index.js","gitHead":"b8d9db9b5af7ebd6475007ee2f15bbae3c00cc24","scripts":{"test":"echo \"Error: no test specified\" && exit 1"},"_npmUser":{"name":"bluefinenterprises","email":"ronak@bluefinenterprises.com"},"repository":{"url":"git+https://github.com/bluefinenterprises/serverless-aws-batch.git","type":"git"},"_npmVersion":"8.3.1","description":"Serverless Framework Plugin for interacting with AWS Batch","directories":{},"_nodeVersion":"16.14.0","dependencies":{"jszip":"^3.4.0","lodash":"^4.17.13","bluebird":"^3.7.2","fs-extra":"^7.0.1","serverless":"^1.70.0","cross-spawn":"^6.0.5"},"_hasShrinkwrap":false,"devDependencies":{"aws-sdk":"^2.672.0"},"_npmOperationalInternal":{"tmp":"tmp/serverless-aws-batch_1.0.0_1646835457507_0.24340638871046738","host":"s3://npm-registry-packages"}}},"time":{"created":"2022-03-09T14:17:37.429Z","modified":"2026-08-31T17:09:50.542Z","1.0.8":"2022-03-08T22:08:39.221Z","1.0.9":"2022-03-08T23:09:23.620Z","1.0.0":"2022-03-09T14:17:37.666Z"},"bugs":{"url":"https://github.com/bluefinenterprises/serverless-aws-batch/issues"},"author":{"name":"Justin Ramsey"},"license":"MIT","homepage":"https://github.com/bluefinenterprises/serverless-aws-batch#readme","keywords":["serverless","aws","batch","python"],"repository":{"url":"git+https://github.com/bluefinenterprises/serverless-aws-batch.git","type":"git"},"description":"Serverless Framework Plugin for interacting with AWS Batch","maintainers":[{"email":"raymond@bluefinenterprises.com","name":"raymond.bluefin"},{"email":"andrey@bluefinenterprises.com","name":"andreybluefin"}],"readme":"# Serverless Batch\n[![serverless](http://public.serverless.com/badges/v3.svg)](http://www.serverless.com)\n\nA Serverless v1.x plugin that makes creating and running AWS Batch tasks as easy as creating a Serverless Lambda Function.\n\nTested with:\n* Serverless >= v1.43\n* Python 3.7\n* Node.JS 14\n\n**Disclaimer: This project has not yet been well tested and is not yet recommended for a production system**\n\n## Install\nFirst make sure than you have Docker installed and running\n\nThen add the plugin to your serverless project:\n\n```\nnpm install @bluefinenterprises/serverless-aws-batch --save-dev\n```\n\nModify the `serverless.yml` file to include the plugin:\n\n```yaml\nplugins:\n  - '@bluefinenterprises/serverless-aws-batch'\n```\n\nNext we need to define our [AWS Batch Compute Resource](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-batch-computeenvironment-computeresources.html)\n\n```yaml\nprovider:\n  name: aws\n  region: us-east-2\n  runtime: python3.7\n  batch:\n    Type: [EC2 | SPOT] # Required\n    BidPercentage: <Integer> # Optional. Defaults to 100 if Type = SPOT (you always pay lowest market price)\n    SecurityGroupIds: # Required\n      - <Security Group ID>\n    Subnets: # Required\n      - <VPC Subnet ID>\n    InstanceTypes: # Optional\n      - <Batch-Supported-Instance-Type> # Default c5.large (cheapest)\n    MinvCpus: <Integer> # Optional. Default 0\n    MaxvCpus: <Integer> # Optional. Default 2\n    Tags: # Optional\n      <Key>: <Value> # Default \"Name\": \"AWS Batch Instance - <service>\"\n```\n\nAnd then define our [AWS Batch Job Definition](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-batch-jobdefinition.html)\non the function definition\n\n```yaml\nfunctions:\n  hello:\n    handler: <handler definition>\n    batch:\n      ContainerProperties:\n        Memory: <Integer> # Optional. Default 2048\n        Vcpus: <Integer> # Optional. Default 1\n        Command: <Command to run in docker> # Optional. Defaults to \"<handler> Ref::event\"\n        JobRoleArn: <ARN> # Optional. Defaults to package.iamRoleStatements\n        Environment:\n          TEST_ENV_1: Test Value 1\n      RetryStrategy:\n        Attempts: <Integer> # Optional. Defaults to 1\n      Timeout:\n        AttemptDurationSeconds: <Integer> # Optional. Defaults to 300\n```\n\nAnd now you should be able to write your batch function like you would any other serverless lambda function:\n\n```python3\nimport logging\n\n# Setup our logger to work both locally and with both AWS CloudWatch\nif len(logging.getLogger().handlers) > 0:\n    logging.getLogger().setLevel(logging.INFO)\nelse:\n    logging.basicConfig(level=logging.INFO)\nlogger = logging.getLogger()\n\n\ndef hello(event, context):\n    logger.info(f\"Hello World: {event}\")\n```\n\n```yaml\nfunctions:\n  hello:\n    handler: handler.hello\n    events:\n      - http:\n          path: hello\n          method: get\n    batch:\n      ContainerProperties:\n        Memory: 2048\n        Vcpus: 1\n      RetryStrategy:\n        Attempts: 1\n      Timeout:\n        AttemptDurationSeconds: 3600\n```\n\n## Implementation\n\nWhat happens behind the scenes is that the Serverless Framework's \".zip\" artifact gets installed into a \n[lambci/lambda:\\<env>](https://hub.docker.com/r/lambci/lambda/) docker image and uploaded to [ECR](https://aws.amazon.com/ecr/).\n\n* _Note: Currently using copy of the docker images (https://cloud.docker.com/u/justinram11/repository/list) that unsets the \nACCESS_KEY_ID and SECRET_ACCESS_KEY environmental variables so that we can use the role attached to the EC2 instance._\n\nA \"schedule\" lambda function is then created with the same name as the regular Serverless Framework's lambda function\nthat can be called in to pass the input event to the AWS Batch Job. The \"schedule\" lambda function can also be subscribed\nto events the same way a regular Serverless Lambda Function is.\n\n![serverless-batch](https://user-images.githubusercontent.com/47002419/60786825-7e80bb00-a11d-11e9-8d4d-bf4916532ab0.png)\n\n## Example\n\nAfter deploying with `sls deploy`, navigate to the [AWS Lambda Console](https://us-east-2.console.aws.amazon.com/lambda/home?region=us-east-2#/functions/serverless-demo-dev-hello?tab=graph)\nand create a new Test Event with any event value and click the \"Test\" button.\n\nThe lambda function will automatically create a new AWS Batch Job which should be visible on the [AWS Batch Console](https://us-east-2.console.aws.amazon.com/batch/home?region=us-east-2#/dashboard)\n\nLogs are visible in CloudWatch under the [/aws/batch/job](https://us-east-2.console.aws.amazon.com/cloudwatch/home?region=us-east-2#logStream:group=/aws/batch/job;streamFilter=typeLogStreamPrefix) Log Group\n\n`serverless.yml`\n```yaml\nservice: serverless-demo\n\nprovider:\n  name: aws\n  region: us-east-2\n  runtime: python3.7 | nodejs12.x # TODO Select one\n  # Creates a SPOT Compute Environment using the VPC defined in our Resources\n  batch:\n    Type: SPOT\n    BidPercentage: 100\n    SecurityGroupIds:\n      - Ref: AllowAllSecurityGroup\n    Subnets:\n      - Ref: PublicSubnet\n    MinvCpus: 0\n    MaxvCpus: 2\n  # Allows the Batch Job (code written in handler.py) to list all of our S3 Buckets.\n  iamRoleStatements:\n    - Effect: \"Allow\"\n      Action:\n        - \"s3:ListAllMyBuckets\"\n      Resource: \"*\"\n\nplugins:\n  # TODO Required if a python project\n  # - serverless-python-requirements\n  - '@bluefinenterprises/serverless-aws-batch'\n\npackage:\n  include:\n    - handler.py\n  exclude:\n    - .serverless/**\n    - node_modules/**\n\nfunctions:\n  hello:\n    handler: handler.hello\n    events:\n      - http:\n         path: hello\n         method: get\n    # Creates a Batch Job with 2GB of memory and 1 vCPU.\n    batch:\n      ContainerProperties:\n        Memory: 2048\n        Vcpus: 1\n      RetryStrategy:\n        Attempts: 1\n      Timeout:\n        AttemptDurationSeconds: 3600\n\n\n# WARNING: Should not be used in production (allows all traffic)\n# VPC: \"Batch VPC\"\n#  - CIDR: 10.224.0.0/16\n#  - Subnet: \"Batch <region>a Public Subnet\"\n#    - CIDR: 10.224.0.0/20\n#    - Allows all incoming and outgoing traffic\n#  - Security Group: \"Batch Allow All\"\n#    - Allows all incoming and outgoing traffic\nresources:\n  Resources:\n\n    VPC:\n      Type: AWS::EC2::VPC\n      Properties:\n        CidrBlock: 10.224.0.0/16\n        EnableDnsSupport: true\n        EnableDnsHostnames: true\n        Tags:\n          - Key: Name\n            Value: Batch VPC\n\n    InternetGateway:\n      Type: AWS::EC2::InternetGateway\n      Properties:\n        Tags:\n          - Key: Name\n            Value: Batch Internet Gateway\n\n    InternetGatewayAttachment:\n      Type: AWS::EC2::VPCGatewayAttachment\n      Properties:\n        InternetGatewayId: !Ref InternetGateway\n        VpcId: !Ref VPC\n\n    PublicSubnet:\n      Type: AWS::EC2::Subnet\n      Properties:\n        VpcId: !Ref VPC\n        AvailabilityZone: ${self:provider.region}a\n        CidrBlock: 10.224.0.0/20\n        MapPublicIpOnLaunch: true\n        Tags:\n          - Key: Name\n            Value: Batch ${self:provider.region}a Public Subnet\n\n    PublicRouteTable:\n      Type: AWS::EC2::RouteTable\n      Properties:\n        VpcId: !Ref VPC\n        Tags:\n          - Key: Name\n            Value: Batch Public Routes\n\n    DefaultPublicRoute:\n      Type: AWS::EC2::Route\n      DependsOn: InternetGatewayAttachment\n      Properties:\n        RouteTableId: !Ref PublicRouteTable\n        DestinationCidrBlock: 0.0.0.0/0\n        GatewayId: !Ref InternetGateway\n\n    PublicSubnetRouteTableAssociation:\n      Type: AWS::EC2::SubnetRouteTableAssociation\n      Properties:\n        RouteTableId: !Ref PublicRouteTable\n        SubnetId: !Ref PublicSubnet\n\n    PublicNetworkAcl:\n      Type: AWS::EC2::NetworkAcl\n      DependsOn: VPC\n      Properties:\n        VpcId: !Ref VPC\n        Tags:\n          - Key: Name\n            Value: AWS Batch Public ACL\n\n    InboundPublicNetworkAclAllowAll:\n      Type: AWS::EC2::NetworkAclEntry\n      Properties:\n        NetworkAclId: !Ref PublicNetworkAcl\n        RuleNumber: 100\n        Protocol: -1\n        RuleAction: allow\n        Egress: false\n        CidrBlock: 0.0.0.0/0\n        PortRange:\n          From: 0\n          To: 65535\n\n    OutboundPublicNetworkAclAllowAll:\n      Type: AWS::EC2::NetworkAclEntry\n      Properties:\n        NetworkAclId: !Ref PublicNetworkAcl\n        RuleNumber: 100\n        Protocol: -1\n        RuleAction: allow\n        Egress: true\n        CidrBlock: 0.0.0.0/0\n        PortRange:\n          From: 0\n          To: 65535\n\n    PublicSubnetNetworkAclAssociation1:\n      Type: AWS::EC2::SubnetNetworkAclAssociation\n      Properties:\n        SubnetId: !Ref PublicSubnet\n        NetworkAclId: !Ref PublicNetworkAcl\n\n    AllowAllSecurityGroup:\n      Type: AWS::EC2::SecurityGroup\n      Properties:\n        GroupName: Batch Allow All\n        GroupDescription: \"Security group for batch instances that allows all traffic\"\n        VpcId: !Ref VPC\n        SecurityGroupIngress:\n          - IpProtocol: \"-1\"\n            CidrIp: 0.0.0.0/0\n        SecurityGroupEgress:\n          - IpProtocol: \"-1\"\n            CidrIp: 0.0.0.0/0\n```\n\n### Python\n\n`handler.py`\n```python3\nimport json\nimport time\nimport logging\nimport boto3\n\n# Setup our logger to work with both AWS CloudWatch and locally\nif len(logging.getLogger().handlers) > 0:\n    logging.getLogger().setLevel(logging.INFO)\nelse:\n    logging.basicConfig(level=logging.INFO)\nlogger = logging.getLogger()\n\n# Setup our boto3 clients\ns3 = boto3.client('s3')\n\n\ndef hello(event, context):\n    logger.info(f\"Hello world: {event}\")\n\n    response = s3.list_buckets()\n    logger.info(f\"S3 Buckets: {response}\")\n\n```\n\n### Node.JS\n\n`handler.js`\n```nodejs\n'use strict';\nconst process = require(\"process\");\nconst AWS = require(\"aws-sdk\");\nconst s3 = new AWS.S3();\n\nmodule.exports.hello = (event, context, callback) => {\n\n  console.log(`Received event: ${JSON.stringify(event)}`);\n\n  console.log(process.env.AWS_ACCESS_KEY_ID);\n  console.log(process.env.AWS_SECRET_ACCESS_KEY);\n\n  s3.listBuckets({}, function(err, data) {\n    console.log(`List buckets data: ${data} err: ${err}`);\n  });\n};\n```\n\n## Contributors\n\n* [simonobe](https://github.com/simonobe)\n* [gcaliene](https://github.com/gcaliene)\n* [MEGApixel23](https://github.com/MEGApixel23)\n* [claydanford](https://github.com/claydanford)\n","readmeFilename":"README.md"}