William Surya Jaya
 
 
Profile Picture

Hi there!

I'm William, a full-stack web developer with 3+ years of experience in various industries. I'm specialized in backend development and have proficiency in Laravel and JavaScript. Highly interested in technological advancements and personal development.

I also write articles for my blog on Medium. I usually write about tech, self development, and personal finance. These topics really intrigued me to be better each day.

I also like to connect with new people, so I’ll be waiting for you to hit me up!

Streamlining CI/CD process with AWS and Bitbucket Pipelines: A Step-by-Step Guide

September 19th, 2023

deploymentcicdstreamliningbitbucketaws
CI/CD Integration Flowchart

TL;DR: This tutorial is also available in video format. Click here to watch it.

In today’s fast-paced software world, automation is your best friend. Our guide is your ticket to automating how you deploy your applications and improving your development process.

By combining the powers of AWS CodeDeploy and Bitbucket Pipelines, we’ll show you how to make continuous integration and deployment a piece of cake.

This guide will be your trusty companion as we walk you through setting up a strong deployment system. We’ll help you blend AWS and Bitbucket seamlessly to make your development process smoother and more efficient than ever before.

In this tutorial, I’ll be using ASP.NET Core Project to do CI/CD, please adjust accordingly to your needs. You can get the tutorial repository from here.

Part 1: AWS Setup

1.1 Create IAM Group

  1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
  2. In the navigation pane, click User Groups and then click Create New Group.
  3. In the Group Name box, type the group's name (bit_CodeDeployRole).
  4. In the list of policies, select the check box for Policy:
    * AmazonS3FullAccess
    * AWSCodeDeployFullAccess
  5. Click Create Group.

1.2 Create IAM User

  1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
  2. In the navigation pane, click Users and then click Add User.
  3. Type the user name (CodeDeployUser) for the new user.
  4. Don’t tick “Provide user access to the AWS Management Console”.
  5. On the Set permissions page, select Add user to group for permission options.
  6. Choose the group that you’ve created in the previous step (bit_CodeDeployRole).

1.2.1 Generate Access Keys

  1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
  2. In the navigation pane, click Users.
  3. Click on the username that you want to create access keys for.
  4. Click on the Security Credentials tab.
  5. Under Access Keys, click Create Access keys
  6. Select the use case, fill in the description, and click Create access key
  7. Click Download .csv file to save your access key.
You cannot view the access keys after you leave this page.

1.3 Configure IAM Role

  1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
  2. In the navigation pane, click Roles and then click Create Role.
  3. Choose AWS service for Trusted entity type.
  4. Choose EC2 for the use case.
  5. Click Next.
  6. In the list of policies, select the check box for Policy:
    * AmazonS3FullAccess
    * AWSCodeDeployRole
    * AmazonEC2FullAccess
    * AmazonSSMFullAccess
  7. Enter role name (AWSCodeDeployRole).
  8. Click Create Role.
  9. On the Roles page, click on the role that you just created.
  10. Click the Trust Relationship tab.
  11. Click Edit Trust Policy.
  12. Click Add Principal, choose AWS service, and fill codedeploy.ap-southeast-1.amazonaws.com for ARN (Change region accordingly)
  13. Click Save.

2.1 Create an AWS S3 Bucket

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
  2. Click Create Bucket.
  3. Type a unique name for your new bucket in the Bucket name field. Ex: bitbucket-codedeploy-deployment.
  4. For Region, choose Asia Pacific (Singapore) ap-southeast-1 as the region where you want the bucket to reside.
  5. Choose ACLs Enabled, and Bucket Owner preferred as the Bucket option.
  6. Click Create Bucket.

3.1 Set Up an AWS EC2 Instance

  1. Sign in to the AWS Management Console and open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
  2. Click Launch Instance.
  3. Fill in the name for EC2 Instance. Ex: CodeDeployDirect
  4. Choose the required OS Images. I’m using Microsoft Windows Server 2022 Base as I’m trying this.
  5. Click Create a new key pair if you don’t have any key pair. Remember to save the .pem file, you’ll need to access this instance.
  6. Under Network Settings, tick Allow RDP Traffic from Anywhere, Allow HTTP Traffic from Anywhere, Allow HTTPS Traffic from Anywhere.
  7. Under Advanced Options, choose the role that we’ve created for this (AWSCodeDeployRole) for the IAM Instance Profile.
  8. Click Launch Instance.

3.1.1 Connect to Instance

  1. Sign in to the AWS Management Console and open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
  2. In the navigation pane, choose Instances.
  3. Click on the instance that you just created.
  4. Click Connect.
  5. Click the RDP Client tab. Pick Connect using the RDP client.
    Download the Remote Desktop Client file.
  6. Click Get Password. You’ll get prompted for the key pair .pem file that we’ve done before.
  7. Connect to the instance using the RDP Client file.

3.1.2 Install IIS

  1. Open up Server Manager.
  2. In the navigation bar, click Add Roles or Features.
  3. Tick Role-based or feature-based installation.
  4. For Server selection, just choose the available server.
  5. In the Server Roles, tick the following features:
    * Web Server (IIS)
    * Windows Deployment Services
  6. In the Features, tick .NET 3.5 and .NET 4.8.
  7. Then just click Next, and let it install.
  8. If you encounter errors, most probably you’ll need to install NET
    Core Hosting Bundle     first.

3.1.3 Install System Manager Agent

  1. Sign in to the AWS Management Console and open this https://console.aws.amazon.com/systems-manager/fleet-manager/dhmc.
  2. Enable Default Host Management Configuration.
  3. Choose the AWSSystemsManagerDefaultEC2InstanceManagementRole role (create it if you don’t have one).
  4. This will allow our instances to show up in the Fleet Manager as managed nodes. You should be able to see your instance showed up in Fleet Manager.
  5. Connect to the instance.
  6. Install SSM Agent. Follow this instructions.
  7. Run these commands on Powershell.
[System.Net.ServicePointManager]::SecurityProtocol = 'TLS12'
$progressPreference = 'silentlyContinue'
Invoke-WebRequest `
    https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/windows_amd64/AmazonSSMAgentSetup.exe `
    -OutFile $env:USERPROFILE\Desktop\SSMAgent_latest.exe

Start-Process `
    -FilePath $env:USERPROFILE\Desktop\SSMAgent_latest.exe `
    -ArgumentList "/S"

rm -Force $env:USERPROFILE\Desktop\SSMAgent_latest.exe

Restart-Service AmazonSSMAgent

3.1.4 Install CodeDeploy Agent

  1. Sign in to the AWS Management Console and open this https://console.aws.amazon.com/systems-manager/distributor
  2. Choose AWSCodeDeployAgent and click Install one time.
  3. Use default options, except Target Selection.
    Chooses instances manually, you should be able to choose your instance.
  4. Click Run.
  5. Verify that the service is running on the instance by running this command.
powershell.exe -Command Get-Service -Name codedeployagent

4.1 Create an AWS CodeDeploy Application

  1. Sign in to the AWS Management Console and open the CodeDeploy Application console at https://console.aws.amazon.com/codesuite/codedeploy/applications.
  2. Click Create Application.
  3. Fill in the Application Name (e.g.: directcodedeploy) and choose EC2/On-premises as the Compute Platform.
  4. Under Deployment Groups, click Create Deployment Group.
  5. Fill necessary fields:
    * Deployment Group: DG1 (Anything)
    * Service Role: AWSCodeDeployRole
    * Deployment Type: In-place
    * Environment Configuration:
    - Tick Amazon EC2 instances
    -     Fill in the Name for Key
    - Choose the instance name in the Value field. Ex: CodeDeployDirect
    - Should show 1 matching instance.
    * Deployment Settings: CodeDeployDefault.OneAtATime
    * Load Balancer: Disabled
  6. Click Create Deployment Group.

Part 2: Bitbucket Configuration

1. Enable Pipelines

  1. Sign in to the Bitbucket and navigate to your repository.
  2. On the sidebar, navigate to Repository Settings.
  3. Under the Pipelines section, click Settings.
  4. Enable Pipelines.

2. Create bitbucket-pipelines.yml in the root folder of the project

The example below is for the ASP NET Core 6 Web Application.
Please update accordingly.

image: mcr.microsoft.com/dotnet/sdk:6.0
pipelines:
 default:
  - step:
    name: Build and Test
    caches:
     - dotnetcore
    script:
     - dotnet restore
     - dotnet build
     - dotnet test
  - step:
    name: Publish and Deploy
    deployment: staging
    script:
     - apt-get update && apt-get install zip -y
     - dotnet publish -c Release -o ./publish
     - cp appspec.yml ./publish/appspec.yml
     - cp before-iis-install-stop.bat ./publish/before-iis-install-stop.bat
     - cp after-iis-install-start.bat ./publish/after-iis-install-start.bat
     - cd ./publish
     - zip -r WebApp.zip .
     - pipe: atlassian/aws-code-deploy:0.2.5
      variables:
       AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
       AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
       AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
       APPLICATION_NAME: $APPLICATION_NAME
       S3_BUCKET: $S3_BUCKET
       COMMAND: "upload"
       ZIP_FILE: "WebApp.zip"
       VERSION_LABEL: "web-app-1.0.0"
     - pipe: atlassian/aws-code-deploy:0.2.5
      variables:
       AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
       AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
       AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
       APPLICATION_NAME: $APPLICATION_NAME
       DEPLOYMENT_GROUP: $DEPLOYMENT_GROUP
       S3_BUCKET: $S3_BUCKET
       COMMAND: "deploy"
       WAIT: "true"
       VERSION_LABEL: "web-app-1.0.0"
       IGNORE_APPLICATION_STOP_FAILURES: "true"
       FILE_EXISTS_BEHAVIOR: "OVERWRITE"
       DEBUG: "true"
    artifacts:
     - dist/**

3. Create appspec.yml, before-iis-install-stop.bat, and after-iis-install-start.bat in the root folder of the project

appspec.yml is to tell AWS CodeDeploy what to do in the deployment process.
before-iis-install-stop.bat will stop IIS before overwriting the files to prevent IIS from still accessing the web files.
after-iis-install-start.bat will start IIS after copying the web files.

4. Add Repository Variables

  1. Sign in to the Bitbucket and navigate to your repository.
  2. On the sidebar, navigate to Repository Settings.
  3. Under the Pipelines section, click Repository Variables.
  4. These are required variables in the bitbucket-pipelines.yml:
    * AWS_ACCESS_KEY_ID
    * AWS_SECRET_ACCESS_KEY
    * AWS_DEFAULT_REGION
    * S3_BUCKET
    * APPLICATION_NAME
    * DEPLOYMENT_GROUP

By following these detailed steps, you can establish a powerful and automated deployment pipeline, ensuring the smooth delivery of your applications from Bitbucket to AWS services. Say goodbye to manual deployment headaches and embrace the future of streamlined development workflows.

Happy coding and deploying!