Mar 06, 2021
DjangoAWS ·9 min read
What is a CI/CD pipeline?
CI/CD stands for continuous integration/continuous deployment and refers to creating a code pipeline where code changes are committed, tested, and then deployed without any manual interference. Basically, a developer can add a new feature to their codebase and push updates in an efficient, streamlined process.
Why is CI/CD important?
Say you create a Django web application on your local device that you deploy on an AWS Elastic Beanstalk. If you have previously done this yourself or followed this tutorial, you probably know that it's a pretty straightforward process. Once AWS CLI is setup and your application is finished, you run eb init then eb deploy to launch. If you need to make any changes, you edit the necessary files and run eb deploy again. However, if your application is attracting users throughout the day, a new code change might cause a bug and users could get a 404 or 500 error. To alleviate this issue, developers setup a CI/CD pipelines where any code change is tested before automatically being added to the production environment. This also allows developers at larger startups to submit hundreds of changes a day without having to manually check the same things. A CI/CD pipeline will allow you to ship code faster and more effectively -- a crucial process for scalability.
Who uses CI/CD?
Any software company or startup likely already has a CI/CD pipeline setup in place. In fact, any new software position usually requires a prospective developer to understand how to use a code pipeline. Just check out any "careers" tab for a given software company and see what pipeline tools they recommend to know. You might see one of the tools listed below.
What are some common CI/CD tools?
Despite all of the different tools, each one is fairly similar. They all involve pushing code using git to a repository that is connected to a given CI/CD tool. Then the changes go through tests and deploy without having to do any additional work. Of course there are some unique features with each one, but we're going to stick with AWS Codepipeline since we're going to deploy on an ElasticBeanstalk and we've covered other AWS topics here. . When selecting a CI/CD tool you might consider factors such as pricing, ease of implementation, and even popularity since other developers will likely be familiar with the software.
CI/CD Tutorial - AWS Codepipeline
Don't worry, deploying your own code pipeline should not be that difficult; it can be broken down into the following steps: Create a pipeline, specify where your project's source code is hosted (most likely Github), add a build stage to run tests in an isolated environment, then select where you're going to deploy your code. To hopefully make this a little simpler, we'll go through the steps to deploy to ElasticBeanstalk first. That way you'll know where we're going to deploy from the start.
1. Setup your Django Application and deploy on an AWS ElasticBeanstalk
If you've never deployed a Django web application to ElasticBeanstalk, here's a brief breakdown. For detailed instructions check out Django Quick Start and Deploying a Django Web App to AWS ElasticBeanstalk.
python3 -m venv env
(Mac) py -m venv env
(Windows)source bin/activate
(Mac) Scripts\activate
(Windows)pip install Django
django-admin startproject mysite
python3 manage.py runserver
(Mac) py manage.py runserver
(Windows)You should see the following screen when you visit http://127.0.0.1:8000/.
pip freeze > requirements.txt
mkdir .ebextensions
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: mysite/wsgi.py
git init
next to initialize an empty repository. You can use this guide to install git: https://gist.github.com/derhuerst/1b15ff4652a867391f03. As expected, you'll need a Github account to continue.git remote add origin <remote_url>
<remote_url>
with the url from your newly created remote repository:git add .
git commit -m "first commit"
git push origin
2. Create a Codepipeline
Now let's get started with the actual CI/CD integration. If you're not already, login to your AWS account then visit AWS Codepipeline and click "Create pipeline". Give your pipeline a name. Select "New service role" and click "Next".
3. Add source stage
Currently we should have the source code for our demo project hosted on Github, so let's select Github (Version 2). Version 2 manages authentication in more secure way while Version 1 uses OAuth tokens and is no longer recommended according to the official docs. Select "Connect to Github" and create a connection by naming your connection and then installing a new Github app. You'll be asked to authorize permissions.
After creating the connection, you should see "Ready to connect" in a green box if successful. Select the repository and the branch name you'd like to use. Then select "CodePipeline default" instead of cloning the entire repository and hit "Next".
4. Adding a build stage
Now we'll need to setup a build environment. Basically, we can select AWS CodeBuild or Jenkins to build our project on a separate server to run tests in an isolated environment before deployment. If following along, select AWS CodeBuild then click the "Create project" box. A new window will open. Give your build project a name. For environment image, we'll select "Managed image", for operating system, Amazon Linux 2, for runtime, standard, and for image version, select the newest version. We'll leave the rest of the configuration options as is.
Once finished, you should see a green box noting the success of your build creation. Environment variables should be added, however, we typically use python-decouple to setup env variables within our project and add an .env file to the private repository; so we'll skip this step. Select "single build" and hit "Next".
5. Add deploy stage and Review:
(Note: the image above will look a little different from the first time you enter this step. This is an image after setting up the pipeline and then going back and editing this step.) Since we deployed our application on Elastic Beanstalk in the first step, we'll select this option as our deploy provider. Then choose the right application and environment that you created earlier. Click "Next" and review all of the options we configured. This should give you a pretty good idea of the entire process.
6. Deploy Again With a BuildSpec
Here's the fun part. First add a buildspec.yml to the mysite project folder:
version: 0.2
phases:
install:
runtime-versions:
python: 3.7
commands:
# Install dependencies needed for running tests
- echo Installing dependencies
- pip install -r requirements.txt
build:
commands:
- echo Build started on `date`
- echo Running tests
- python3 manage.py test
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- '**/*'
Next, change your Django project by replacing the debug page with your own homepage. Then add, commit, and push the changes. After pushing your changes to the master branch, your code pipeline will start! You'll see your code going through the different stages; from being retrieved from Github, to being tested in AWS CodeBuild, and ultimately, deployed on your ElasticBeanstalk environment. If it all goes successfully, then you should see a green checkbox next to each stage.
Conclusion
That concludes this CI/CD overview and tutorial. Let me know in the comments if I should expand on certain points or if you're a little confused on a given topic. This tutorial was meant for a simple Django project so I didn't discuss how to handle different databases. For example, if you're using AWS RDS, you might have the following in your settings.py file:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': config('DB_NAME'),
'USER': config('DB_USER'),
'PASSWORD': config('DB_PASSWORD'),
'HOST': config('DB_HOST'),
'PORT': config('DB_PORT'),
}
}
This actually creates a problem at the AWS CodeBuild stage. Our build environment does not have permission to connect to the remote database. However, there's a few solutions.
Also I never touched on what's happening in that buildspec.yml file. We're specifying the commands to run during specific phases of the build stage to setup our dependencies and tests. If you wanted to test something, run py manage.py createapp main
to create an app for your project. Here's some tests to check if a robots.txt file has been correctly added to your site. This is helpful for connecting your Django project to Google Search Console.
from django.test import TestCase
from http import HTTPStatus
# Create your tests here.
class RobotsTxtTests(TestCase):
def test_get(self):
response = self.client.get("/robots.txt")
self.assertEqual(response.status_code, HTTPStatus.OK)
self.assertEqual(response["content-type"], "text/plain")
lines = response.content.decode().splitlines()
self.assertEqual(lines[0], "User-Agent: *")
def test_post_disallowed(self):
response = self.client.post("/robots.txt")
self.assertEqual(response.status_code, HTTPStatus.METHOD_NOT_ALLOWED)
Hope you enjoyed!
Follow us@ordinarycoders