Simple CI/CD Pipeline in AWS : Trigger JAR Build on CodeCommit Events

A simple and automated JAR package build using AWS services.

Primer

AWS services used are :

1. AWS CodeCommit : A fully-managed source control service. A GitHub on AWS.
2. AWS Lambda : A serverless compute service that lets us run code without provisioning or managing servers.
3. AWS CodeBuild : A fully-managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy.
4. AWS S3 : An object storage services in the cloud.

Steps

Notes :
1. Make sure you create all resources in the same AWS region.
2. Repository, Lambda function, IAM role name should be globally unique.

1. Setup an Empty Repository in CodeCommit

• Go to CodeCommit console, in the side tab, go to Repositories, click Create repository
• Give it a name ; I am using ngandhi-codecommit-lambda-<programming_language>
• Click Create

2. Create an S3 Bucket to Store the CodeBuild Output

• Go to S3 console, click Create bucket
• Give the bucket a name ; I am using ngandhi-test-pipeline-<programming_language>-output-bucket
• Click Create bucket

3. Setup your local repository

Use the following project structure in your local machine (I am using a simple Java project).

.
├── buildspec.yml
├── pom.xml
└── src
    ├── main
    │   └── java
    │       └── MessageUtil.java
    └── test
        └── java
            └── TestMessageUtil.java

src/main/java/MessageUtil.java

src/test/java/TestMessageUtil.java

buildspec.yml

pom.xml

4. Create A Lambda Function That Accepts Triggers from Code Commit

Create the Function

• Go to Lambda console, click Create function
• Choose “Author from scratch” → Give the function a name ; I am using ngandhi-codebuild-trigger-on-codecommit-events
• Choose the Lambda runtime (I am using Python 3.8)
• Default for everything else (role for Lambda invocation will be created later)
• Click Create function

The Lambda Handler Code

Write the following code in the editor and click Deploy .

Note that Python code in Lambda already contains boto3 library.

lambda_handler.py

Setup an IAM Role and Policy for Lambda

We will need the following permissions :

• Write log data to Amazon CloudWatch Logs
• Invoke the AWS CodeBuild StartBuild API

To do that :

• Go to AWS IAM Console and click Create role
• Pick “AWS Service” as trusted entity type and choose Lambda use case, click "Next"
• Attach the following policies and click “Next” :

AWSCodeBuildDeveloperAccess
AWSLambdaBasicExecutionRole

• Skip “Tags” if you want to.
• In the “Review” section, give the role a name ; I am using ngandhi-codebuild-trigger-lambda

• Click Create role

Attach the newly created role to your Lambda function :

• Go back to your Lambda function dashboard, click Configuration → Permissions
• In the “Execution role” panel, click “Edit”
• In the “Existing role” drop down, choose your previously created role and Click Save

5. Create A CodeBuild Project

• Go to CodeBuild console and click at Create build project
• Give the project a name ; I am using ngandhi-codebuild-<programming_language>-project

The following properties to be used :

• Source Provider : AWS CodeCommit
• Repository : Your CodeCommit repository name (here: ngandhi-codecommit-lambda-<programming_language>)
• Reference type : Branch

For “Environment”:

• Image = Managed Image
• Operating system = Amazon Linux 2
• Runtime(s) = Standard
• Image = aws/codebuild/amazonlinux2-x86_64-standard:3.0
• Image version = Always use the latest image for this runtime version
• Environment type = Linux
• Service role = New service role (will automatically created)

For “Buildspec” : Default (Use a buildspec file)

• Buildspec name = buildspec.yml

For “Artifacts”:

• Type: Amazon S3
• Bucket name: Your output bucket name (here: ngandhi-codebuild-<programming_language>-output)
• Name: folder name. I am using output
• The rest, leave as is
• Click Update artifacts

• Logs : Leave as is

Finally, click Create build project

6. Setup Trigger on CodeCommit Repository

• Go to your Lambda function dashboard and visit Configuration → Triggers
• Click Add trigger
• In the “Trigger configuration”, choose “CodeCommit” from the drop down menu. A new setting form will be opened.
• Fill in your repository name under “Repository name” (here : ngandhi-codecommit-lambda-<programming_language>)
• Fill trigger name ; I am using ngandhi-trigger-codebuild
• Under “Events”, pick an event that you would like to trigger the build. You can pick multiple options. The default is All repository events
• In “Branch names”, pick the branch that you would like to be the primary branch to trigger the build. By default it will be “All branches” (represented as empty values)
• In “Custom data — optional”, fill it with the name of your CodeBuild project (here : ngandhi-codebuild-<programming_language>-project). This field is used to pass the name of the CodeBuild project that we want our Lambda function to invoke. This is represented in our Lambda code under the following struct :

build = {
        'projectName': event['Records'][0]['customData'],
        'sourceVersion': event['Records'][0]['codecommit']['references'][0]['commit']
}

• Click Add to create the trigger

7. Reviewing Trigger Configuration

• After CodeCommit trigger has been configured in the Lambda console, visit your CodeCommit repository
• Click Settings from the sidebar
• Click Triggers from the tab
• You will see your previously configured trigger there

8. Test the Trigger

• This is the fun part ; go back to your local text editor
• If you don’t have it yet, install git-remote-codecommit utility (needs pip or pip3) on your local

pip3 install git-remote-codecommit

• Monitor the installation process until you see a success message similar to the following message

“successfully built git-remote-codecommit”

• Go to your local repositories and initialize git project

git init
git add --all
git commit -m "My first commit"

• Set remote origin url to CodeCommit

# URL syntax
codecommit://[aws_profile_name]@[repository_name]
# example
git remote add origin codecommit://aws_profile@ngandhi-codecommit-lambda-java

• Push your local repo to remote branch

git push -u origin master

• Go to your CodeBuild dashboard and see if a new build has kicked off

• View the AWS CodeBuild logs from the build

9. Check the build output in S3

• Go to your previously created S3 bucket
• JAR will be visible inside your bucket (<bucketName>/<artifactsOutputPath>/<objectKeys>)

Congratulations, you have build a simple CI/CD pipeline in AWS.

References

1. Use AWS CodeCommit & Lambda to Trigger CodeBuild, for Fully-Automated Container Image Builds!
2. tdmalone/codebuild-trigger
3. How to deploy AWS Lambda functions using CodePipeline
4. Step 2: Create the buildspec file