Production Ready Serverless Ops Boilerplate

Continuous Delivery, user management and authentication are inevitable part of most modern apps. Yet, based on my experience, implementing them is difficult and can prolong your product launch for even month(s). This project is born to adress the issue and reduce your product launch nightmare.

AWS recommends each team to have its own multi-account DevOps oriented setup; however, looking at available materials on the Internet (including AWS own materials), I couldn't find any straighforward and easy-to-use solution. Also, despite being a useful tool, Amazon Cognito currently lacks a good documentation and samples, especially when it comes to integration with Lambda and Api Gateway as well as using Cognito Hosted UI.

To make your job easier, I have created this boilerplate: all you need to do is to inject few parameters and run the Ansible playbook. Behind the scenes, it creates frontend and backend pipeline for you and deploys your application - in just few minutes. You can easily put your business logic in the app, also you can customize any part of the app according to your needs. Leveraging this boilerplate, you can launch your product as fast as possible.

Backend API is supported by Lambda, and is secured by AWS Cognito. Frontend part is a React app which is hosted in S3, is accesible through Cloudfront and is protected through Cognito. Users can authenticate through Cognito Hosted UI and use the app.

Architecture

This boilerplate uses AWS native tools for continous integration, continous delivery and other required managed services: An AWS account (called DevOps account) is used to orchestrate CI/CD pipelines across dev, test, prod, accounts (we refer to them as environment accounts). This setup creates separate pipelines for frontend ./pipeline/frontend.yaml and backend./pipeline/backend.yaml. We assumes that your backend and frontend code are in separate repositories (but of course you can modify it). This repository includes infrastruture-as-a-code for pipelines ./pipeline as well as the application ./app. And here you can find repository for frontend code which is a React.js app.

Pipelines use Github for source control (but you can use Codecommit, Bitbucket or other Git services too), CodePipeline for automating pipeline, CodeBuild for building and testing application, CodeDeploy for deployment. Once pipeline gets triggered, CodeBuild creates, encrypt and stores artifact in S3 bucket. Then, CodeDeploy retrieves artifact and performs cross account deployment into other accounts.

Backend service ./app is an AWS Serverless application Model (SAM) app: AWS Lambda, Lambda Layer, API Gateway, Cloudfront, Cognito User Pool, Secrets Manager, Key Management Service and S3. Lambda gets secret ApiKey from Secrets Manager and makes request to weather API to get today's weather report. Frontend pipeline gets the latest code from frontend repository, builds and deploys it to S3.

Setup your local machine

Login to AWS console, switch to DevOps account, and set your profile following this instruction (we specify the profile in our commands as "devops"). Do this also for all your environment accounts e.g. dev, test, prod. Remember that this is case sensitive.

Install followings in your machine (I have used those versions, but other versions may work too)

AWS CLI version 2.0
Node.js v13.7.0
Ansible 2.9.6 - I had issues when I installed ansible through pip (it couldn't find cloud modules). When I installed it through brew, it worked fine
Jinja2 version 2.11.1 - pip install jinja2

Inject variables

Replace variables in ./inventory file:

• Env: This is case sensitive and adds prefix to resources. If you choose prod, you need to manually approve the deployment in CodePipeline. In pipeline/backend.yaml and pipeline/frontend.yaml there is a Conditions section that enforces manual approval; you can edit it if needed.

• DevOpsAccountId: AWS account id that holds pipelines and artifacts

• EnvAccountId: environment account id, for example dev account

• Region: deploys all resources within this region.

• AppName : this will be used to name all created resources

• GithubOwner: Github handler that holds the repository. If you use any other git provider (e.g. CodeCommit), you can change Sourceaction settings in ./pipeline/frontend.yaml and ./pipeline/backend.yaml

• OAuthToken: Github personal access token used for authentication. Here is the instruction.

• BackendRepo: repository name that holds backend code, that is ./app

• FrontendRepo: repository name that holds frontend code

• WeatherApiKey: You need this key to get weather data. You need to signup and get it from here

• HeaderValue: This value is used to restrict access to S3 only through Cloudfront. Cloudfront injects this value as Referer header and S3 bucket policy only allows requests that have this header value. You can read more about it here

• DevOpsAccountProfile: AWS cli profile that holds credentials for your devops account

• EnvAccountProfile: AWS cli profile that holds credentials for your environment account. e.g. dev.

For the sake of clarity, I've put environmental variables in the inventory file. But you can also inject them through command line, something like this: ansible-playbook playbook.yml -i ./inventory -e "Env=dev DevOpsAccountId=********"

Run the playbook

You can run the app by ansible-playbook playbook.yml -i ./inventory

Once finished, you can see ansible results in terminal:

You can access your app in RedirectUri. Open it in your browser and you should see this:

To access the app, firstly you need to create a user. Login to your environment account (e.g. dev), go to Cognito, then "Manage User Pools", choose the newly created user pool. In the left hand section choose "Users and Groups", and create a user. Now, you should be able to login with your user credentials. Press the button "Sign in to get the weather. The app will redirect you to Cognito hosted UI:

After successful authentication, you can see weather report; something like this:

Suggestion for further developments:

Fix integration test Restrict CORS origin based on cloudfront Add OpenApi specifications to Lambda functions Replace Amazon Cognito Identity SDK with AWS Amplify

Troubleshooting

You can access Cognito hosted UI directly from with following URL:

https://<your_domain>/login?response_type=code&client_id=<your_app_client_id>&redirect_uri=<your_callback_url>

You can get these details from ansible results (RedirectUri is your callback url.). More troubleshooting tips are in the frontend repo

References

This boilerplate is inspired by and is simplified version of Building a Secure Cross-Account Continuous Delivery Pipeline. Implementing that solution was challenging, that's why I have created this boilerplate.

Also, I have leveraged Weatherapp