LAB 13: Create a pipeline using AWS CodePipeline for Lambda deploy

This Lab has the step by step instructions to create a CI CD pipeline for a Serverless Application Deployment - Lambda using AWS CodeCommit, AWS CodeBuild, AWS Lambda, AWS CodeDeploy & AWS CodePipeline

Ensure to complete LAB 9 through LAB 12 including pre-requisite as stated in AWS Developer Tools CI CD Lab blog.

Login to AWS Console as `codedeploy_user`, go to AWS CodePipeline and create a pipeline with the following configuration

Create a Pipeline

Add Approve & Execute Stage

Once the above pipeline is created, edit to add 2 more stages as ApproveChangeSet and ExecuteChangeSet

Select the ARN of the SNS topic you have created earlier in this Lab series

Run the Pipeline

We have 5 stages as Source, Build, Staging, ApproveChangeSet and ExecuteChangeSet.
After the Build & Staging (after creating the CloudFormation ChangeSet), the pipeline is waiting on the manual approval.

Approval process

Based on our email configured in SNS Topic: Subscription, email notification will be sent as follows and the user can launch the URL provided for Approve or Reject and approve it.

Once the Manual approval is completed, ExecuteChangeSet stage will be executed and the Lambda function is created through CloudFormation template.

LAB 12: AWS CI CD Create Service Role for CloudFormation Deploy & Lambda

In this Lab, you will be creating a Service Role to do the CloudFormation deployment and Lambda Execution 

Ensure to complete LAB 9 through LAB 11 including pre-requisite as stated in AWS CI CD - 3rd Workflow - Serverless Application Deployment Lab blog
Login to AWS Console as `codedeploy_user` & go to AWS IAM dashboard -> Roles -> Create Role, select AWS Service as the type of trusted entity and proceed with the below configuration to create the role named CloudFormationServiceRole

Create an in-line policy CFN_Lambda_CICD_Policy with the policy document from Github.

Create Lambda Service Role

Create an in-line policy policy_LambdaWorker with the policy document from Github.

LAB 11 - AWS CI CD - Setup a Build Project in AWS CodeBuild for Lambda

This Lab has the step by step instructions to create & configure a Build using AWS  CodeBuild, which will compile the latest commit of the Lambda function, generate the artifact and uploaded to the named S3 bucket.

Ensure to complete LAB 9 through LAB 10 including pre-requisite as stated in AWS CI CD - 3rd Workflow - Serverless Application Deployment Lab blog.

Please note that for the purpose of this example, the Serverless Application is using demo-4-codedeploy as the bucket name. Make sure to edit that with your S3 bucket name before you proceed further.

Login to AWS Console as `codedeploy_user` & go to AWS CodeBuild from AWS Console, click Create Project and proceed with the configuration as follows

Build Source

Build Environment

Storying Build Artifacts

Enabling Build Logs

Save the Build & Run. Verify that the artifact is uploaded to S3 Bucket.

buildspecl.yml

Here is the closer look of buildspec.yml used in this example. In addition to this, you can environment variables as stated below

<add buildspec.yml>

Debugging CodeBuild

• AWS CodeBuild is a managed service, which uses pre-defined container images for handling builds. Hence you will not have access to the container images running your build.
• Local Build Support is enabled for AWS CodeBuild since May 2018, which you can use for debugging issues locally.
• Enabling Logging to CloudWatch or S3 is key to monitor the build progress from AWS Management Console and for debugging in case of failures

LAB 10: Setup Repository in AWS CodeCommit for Lambda

This Lab has the step by step instructions to setup a Repository in AWS CodeCommit, add the pre-developed Lambda Application Code and commit the Changeset
Ensure that you have already went through the AWS CI CD using AWS Developer Tools & create IAM User, Group and S3 bucket before we move forward.

1. Login to AWS Console as `codedeploy_user` & go to AWS CodeCommit dashboard
2. Create an empty repository as, codecommit_lambda and you may Skip SNS Topic as of now and view the Connect to your repository screen

2. 

   3. Get the clone URL (say git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/codecommit_lambda), paste run it on your terminal. It will ask for username and password, enter the Git Credentials you have generated in Lab-9. You will get a warning that you appear to have cloned an empty repository and you are good to proceed.
3. 
   
4. Download the Serverless Application source code from here and commit to your Git Repo using the commands as in below command line reference
5. $ git clone <git URL>
   
               $ git init
    
               $ git status
6. $ git add .
                  $ git commit -m "Adding Serverless Application to CodeCommit Repo"
                  $ git push
                  $ git status
7. 
   

Now your source code is in git and may use the below command to commit the changes, if any, to the source code

• git status - To view the changes 
• git add . - to add all changes to the local repo and use git add <folder[/file]> to add selected folder or file to the local repo
• git rm . - to add all changes to the local repo and use git rm <folder[/file]> to add selected folder or file to the local repo
• git commit -m “what changes are getting in”
• git commit --amend - will open the editor for you to modify the commit. Commit can be modified from command line as well using git commit --amend -m “New Commit”
• git push - push the changes to the remote
• git log -2 - gives the last two commit
• git  branch -lvv - gives the local branches with remote mapping
• git checkout <branchname> - to checkout from a remote branch
• git branch -D <branchname> - to delete a local branch
• git reset --hard <branchname> - to do a hard reset to the head of the revision from the remote - This is helpful if your local branch is messed-up and you want to overwrite everything from remote.

LAB 9 - AWS CI CD - IAM - Creating CodeDeploy Group & User

2. In this Lab, we will be creating an IAM User under a new Group from IAM, which will be used hence forth for this demo to access AWS CI CD Services (AWS CodeCommit, AWS CodeBuild, AWS CodeDeploy & AWS  CodePipeline both programmatically & through AWS Console.
3. 
   
4. Login to AWS Console as Root User, go to IAM -> Users and enter the details as follows to create codedeploy_user and set the Custom Password, which you can use later to login via IAM Console
   [Note: You will have to uncheck Required password reset option if you do not want to create a new password at the next sign-in.]
   
   
   
   
   
   
6. 
   
7. Next in permissions screen, Create a Group as codedeploy_group  attach policy by clicking Permissions tab to add the following permissions

AWSCodeCommitFullAccess
IAMSelfManageServiceSpecificCredentials
CloudWatchFullAccess
AWSCodeDeployFullAccess
IAMReadOnlyAccess
AWSCodePipelineFullAccess
AWSCodeBuildAdminAccess

8. And add an in-line custom policy for the following hand picked policies                          {
   

   "Version": "2012-10-17",

       "Statement": [

           {

               "Sid": "Stmt1537596486000",

               "Effect": "Allow",

               "Action": [

                   "iam:CreateRole",

                   "iam:AttachRolePolicy",

                   "iam:CreatePolicy",

                   "iam:CreatePolicyVersion",

                   "iam:DeletePolicy",

                   "iam:DeletePolicyVersion",

                   "iam:PutRolePolicy",

                   "events:PutRule",

                   "events:PutTargets",

                   "cloudformation:DescribeStackEvents",

                   "cloudformation:GetStackPolicy",

                   "cloudformation:DeleteStack ",

                   "apigateway:GET"

               ],

               "Resource": [

                   "*"

               ]

           },

           {

               "Effect": "Allow",

               "Action": [

                   "s3:PutObject",

                   "s3:PutObjectAcl",

                   "s3:GetObject",

                   "s3:GetObjectAcl",

                   "s3:DeleteObject"

               ],

               "Resource": "arn:aws:s3:::demo-4-codedeploy/*"

           }

       ]

   }
10. 
    
11. Please note that the following 3 policies were required for CodeCommit as stated in prerequisite section of Connect to your repository and the remaining policies are required for the respective services.

    AWSCodeCommitFullAccess   

    IAMSelfManageServiceSpecificCredentials  

    IAMReadOnlyAccess 

    Note: This gives access to all repositories in your CodeCommit. If you want to restrict access, you need to write Custom Policies. Same applies to all services where unrestricted access is given. 
12. 
    
13. Now add codedeploy_user to codedeploy_group 
14. 
15. and then make sure to download the user credentials as this is the last time you will be able to access it Credentials, else you will have to regenerate the keys later.
16. 
17. 
    
18. Get back to the user codedeploy_user and create Git credentials for this user Under HTTPS Git Credentials for AWS CodeCommit.
19. Click Generate button and download credentials  

AWS CI CD - 3rd Workflow - Serverless Application Deployments

3rd Workflow in AWS CI CD using AWS Developer Tools Lab Series is about Serverless Application Deployment that can be taken-up independently as the material have been scripted from scratch with step by step instructions.  AWS CodeCommit, AWS CodeBuild, AWS CodeDeploy & AWS CodePipeline

Serverless Application is one or more Lambda functions, which is basically a compute service, and has triggering events like CloudWatch, S3 or DynamoDB etc, associated with a Lambda function. Deploying of Lambda functions along with all of the associated resources, as a bundle is termed as serverless deployments

Serverless Application is one or more Lambda functions, which is basically a compute service, and has triggering events like CloudWatch, S3 or DynamoDB etc, associated with a Lambda function. Deploying of Lambda functions along with all of the associated resources, as a bundle is termed as serverless deployments

Serverless Application is one or more Lambda functions, which is basically a compute service, and has triggering events like CloudWatch, S3 or DynamoDB etc, associated with a Lambda function. Deploying of Lambda functions along with all of the associated resources, as a bundle is termed as serverless deployments

Serverless Application Deployment

• In this workflow, a simple Lambda function using python is sourced from Codecommit repository, configured to ‘Codebuild’ service, for compiling and unit testing phase and then based on the buildpsec.yml(which is input for codebuild) the code repo for Lambda, is packaged and zipped into the relevant S3 bucket location configured
• Based on the application framework chosen, JUnit, Unittest or other tests can be configured to run in CodeBuild stage
• ‘aws cloudformation package’ specified in build phase of buildspec.yml converts the SAM template into CloudFormation for Create or replace stack
• SAM template is basically an extension of Cloudformation template and only difference between these two is the ‘AWS Transform: serverless’ in SAM template, which defines the Lambda specific instructions
• Next, a ‘Manual approval’ stage with the SNS topic is configured. The approver will get notified via email with a link to ‘Approve or Reject’
• Once approved, the Cloudformation ‘Execute change set’ action is triggered from next stage, which actually creates all resources bundled with Lambda

1. LAB 9: Prerequisites Setup
1. Create IAM Group & User
2. Create CloudWatch Log group named codedeploy_logs
3. Create S3 Bucket named demo-4-codedeploy
       Cross-verify the configurations defined in the LAB 9 against the below mind-map.
   
2. LAB 10: Setup Repository in AWS CodeCommit
       Cross-verify the configurations defined in the LAB 10 against the below mind-map.
   
3. LAB 11: Setup a Build in AWS CodeBuild
       Cross-verify the configurations defined in the LAB 11 against the below mind-map.
   
4. LAB 12: Create a pipeline using AWS CodeDeploy
       Cross-verify the configurations defined in the LAB 12 against the below mind-map.
   
5. LAB 13: Create a pipeline using AWS CodePipeline
       Cross-verify the configurations defined in the LAB 13 against the below mind-map.
   
6. View AWS CI CD - Serverless Application Deployments Workflow