Multiple tools are available to do CI/CD. In the article, I’ll compare CodeBuild, Bitbucket Pipelines and a brief overview of Jenkins.

Notifications

Both offer notification options. On Bitbucket, you can add Slack notifications quickly by going to Chat Notifications > Settings > Add Subscription. CodeBuild integration is not that intuitive but offers a more significant deal of flexibility. You would have to subscribe build-notifications to SNS topic. From there you can add AWS ChatBots to push notifications to your Slack, or email, or Lambda, or HTTPS endpoint or even send SMS.

You could also add custom logic to your notifications. Bitbucket offers webhooks on pipeline events.

CodeBuild triggers existing Bitbucket webhooks too, which makes migration less painful.

Networking

Bitbucket Pipelines doesn’t offer networking options. They run on a random for a given IP range. If you want to limit network access to the on-premises server, you’d need to allow whole Bitbucket Pipeline IP ranges (https://confluence.atlassian.com/bitbucket/what-are-the-bitbucket-cloud-ip-addresses-i-should-use-to-configure-my-corporate-firewall-343343385.html) which might cause security issues.

CodeBuild does the same networking options as Bitbucket Pipelines where AWS exposes all possible public IPs, and you can edit your firewall accordingly. However, CodeBuild also offers advanced networking options to run the pipeline in the VPC.

Once your box is in VPC, you can peer VPCs, add Site-to-site VPN and route traffic your private VPC or on-premises servers. The solution could be useful if you’ve npm registry on-premises and you need to access it securely.

Infrastructure-as-a-code

Both offer infrastructure-as-a-code solutions.

Bitbucket pipelines could be enabled using Terraform (Project https://www.terraform.io/docs/providers/bitbucket/r/repository.html and environment variables https://www.terraform.io/docs/providers/bitbucket/r/repository_variable.html). CodeBuild integration requires more lines of config but if more flexible. Also, you could use both Terraform and CloudFormation.

Price

Bitbucket has different pricing options than AWS.

Bitbucket Pipelines offer product package options than AWS offers pay-as-you-go. For example, Bitbucket Pipelines gives 2500 minutes per month for a $3/user/month price. AWS CodeBuild offers $0.005 per build minute.

Billing

Companies might have multiple AWS accounts, various environments for different customer. It’d share same Bitbucket account, but the billing user couldn’t split invoices per project.

CodeBuild runs in AWS accounts. If you set up different AWS accounts for various environments and customer, the billing is per account which simplifies billing.

Authentication with AWS

Using Bitbucket pipelines doesn’t allow IAM Roles, so you’ve to setup long-lived IAM User credentials which are rarely or never rotated. They might get leaked, which is a serious security concern.

CodeBuild has IAM Role integration which gives temporary AWS credentials.

Caching

CodeBuild has docker layer caching, or storage in S3. Bitbucket Pipelines also offers caching solutions, but only up to 1 GB once compressed.

SLA

No SLA on Bitbucket Pipelines and we’ve noticed several serious outrages during the time we’ve used the service. CodeBuild offers 99.9% availability which is pretty good, especially considering most pipelines doesn’t affect customers.

But wait, what about Jenkins?

Jenkins is a beast. It can do so many things and has 1500+ plugins which make your CI/CD platform super flexible.

One simple functionality is allowing custom input and confirmations in the console. It doesn’t exist on Bitbucket and CodeBuild, but it does on Jenkins.

With default settings, Jenkins runs on a server compared to CodeBuild/Bitbucket Pipelines which runs a new docker image on new builds. Docker gives you immutable deployments. However, Jenkins deployments depend on previous builds where the tool reuses environments. It might give you the advantage of faster builds and cached modules.

Jenkins comes as an unmanaged service. You’d have to ensure that a server is running, has internet access, latest security patches, SSH access is locked down etc. If something goes wrong, you’d have to investigate why and fix it. Meanwhile, the service provider solved operational issues on CodeBuild and Bitbucket Pipeline.

To wrap up

CodeBuild has several advantages in AWS, including better pricing, IAM temporary credentials, networking options and CloudFormation support. Bitbucket Pipelines are alright until you increase the number of projects. Jenkins is a beast, you can configure it as you like, but you spend valuable time maintaining and setting it.

Source