We'll start by setting up a Cloud9 which is an EC2 instance accessible via a browser-based IDE and Terminal. Being backed by a dedicated Linux machine, containers run right on these Cloud9 instances when you use the docker commands.
- Sign into the AWS Console for the provided account
- Choose the Sydney region in the upper right dropdown
- Click the
Create environmentbutton - Name your environment
cloud9and click theNext stepbutton - Change the instance type to
m5.largethen click theNext stepbutton accepting the other defaults - Click the
Create environmentbutton - When it comes up close everything (the two bottom tabs as well as the bottom pane that has two more tabs in it)
- Then open a new terminal by choosing the
Windowmenu then pickingNew Terminal - Run
git clone https://github.com/jasonumiker/docker-ecs-immersion.git - Run
cd docker-ecs-immersionthengit submodule update --init --recursiveto bring in the submodules.
- Run
docker versionto confirm that both the client and server are there and working (in this case on the same EC2 Instance running our Cloud9)
- Run
docker run -d -p 8080:80 --name nginx nginx:latestto run nginx in the background as a daemon as well as map port 8080 on our host to 80 in the container- The -d is to run it as a daemon (in the background), the -p maps the host port 8080 to 80 in the container, --name gives us a name we can run further docker commands against easily and then the image repository and tag we want to run.
- Run
docker psto see our container running - Click on the
Previewmenu in the middle of the top bar then choosePreview running application. This opens a proxied browser tab on the right to show what is running on localhost port 8080 on our Cloud9 EC2 instance. Click thePop Out Into New Windowicon in the upper right-hand corner of that right pane to give it its own tab then close the preview tab. - Run
docker logs nginx --followto tail the logs the container is sending to STDOUT (including its access logs) - Refresh the preview in the separate browser tab a few times and then come back to see the new log line entries
- NOTE For some reason refreshing this within Cloud9 sometimes doesn't leave these log lines - do it in the separate tab within your browser
- Press Ctrl-C to exit the log tailing
- Run
docker exec -it nginx /bin/bashto open a shell within our container (-it means interactive) - Run
cd /usr/share/nginx/htmlthencat index.htmlto see the content the nginx is serving which is part of the container. - Run
echo "Test" > index.htmland then refresh the browser preview tab to see the content change- If we wanted to commit this change to the image as a new layer we could do it with a
docker commit- otherwise it'll stay in this container but be reverted if you go back to the image.
- If we wanted to commit this change to the image as a new layer we could do it with a
- Run
exitto exit our interactive shell
- Run
docker stop nginxto stop our container - Run
docker ps -a(-a means all including the stopped containers) to see that our container is still there but stopped. At this point it could be restarted with adocker start nginxif we wanted. - Run
docker rm nginxto remove the stopped container from our machine then anotherdocker ps -ato confirm it is now gone - Run
docker imagesto see that the nginx:latest image is there there cached - Run
docker rmi nginx:latestto remove the nginx image from our machine's local cache
- Run
cd ~/environment/docker-ecs-immersion/aws-cdk-nyan-cat/nyan-cat - Run
cat Dockerfile- this is start from the upstream nginx:alpine image (alpine is a slimmer base image option offered by nginx and many other base images) and then copy the contents of this path into /usr/share/nginx/html in our container replacing the default page it ships with - Run
docker build -t nyancat .to build an image called nyancat:latest from that Dockerfile - Run
docker history nyancat:latestto see all of the commands and layers that make up the image - see our new layer? - Run
docker run --rm -d -p 8080:80 --name nyancat nyancat:latest(--rm means to delete the container once it is stopped rather than leave it around to be restarted) - Click on the
Previewmenu in the middle of the top bar then choosePreview running application. This opens a proxied browser tab on the right to show what is running on localhost port 8080 on our Cloud9 EC2 instance. Click thePop Out Into New Windowicon in the upper right hand corner of that right pane to give it its own tab then close the preview tab.- See our new content that is built into the image for nginx to serve?
- Run
docker stop nyancatto stop and clean up that container (we said --rm so Docker will automatically clean it up when it stops)
Getting a local development environment with the 'right' versions of things like the JDK and associated tooling can be complicated. With docker we can have the docker build do the build but also do it in another build stage and then only copy the artifacts across we need at runtime to our runtime container image with multi-stage docker builds.
This example is Spring Boot's (a common Enterprise Java Framework) Docker demo/example. But it could apply to any compiled language.
- Run
cd ~/environment/docker-ecs-immersion/top-spring-boot-docker/demo - Run
cat Dockerfileand see our two stages - the first running a Maven install and the second taking only the JAR and putting it in a runtime container image as we don't need all those build artifacts at runtime keeping the runtime image lean. - Run
docker build -t spring .to do the build. This will take awhile for it to pull Spring Boot down from Maven etc. We don't have the JDK or tools installed on our Cloud9 but are compiling a Java app. If different apps needed different version of the JDK or tools you could easily build them all on the same machine this way too. - Once that is complete re-run the
docker build -t spring .command a 2nd time. See how much faster it goes once it has cached everything locally? - Run
docker run --rm -d -p 8080:8080 --name spring springto run our container. - Run
curl http://localhost:8080- it just returns Hello World (and Spring Boot is a very heavy framework to just do that! We wanted to see how you'd do a heavy Enterprise Java app though) - Run
docker stop spring
Now that we containerised our nyancat content together with an nginx to serve it let's deploy that to ECS
We'll do this two ways - first with AWS Copilot and then with the AWS Cloud Development Kit (CDK). These both actually generate CloudFormation for you but Copilot is more simple and opinionated while CDK is a general purpose tool that can do nearly anything but is more complex.
First we'll need to give AWS Administrator Access to our Cloud9 Instance:
- Go to the IAM service in the AWS Console
- Go to
Roleson the left-hand navigation pane - Click the blue
Create rolebutton - Choose
EC2underCommon use casesin the middle of the page then click theNext: Permissionsbutton in the lower right - Tick the box next to
AdministratorAccessthen click theNext: Tagsbutton in the lower right - Click the
Next: Reviewbutton in the lower right - Enter
EC2FullAdminfoe theRole nameand then clickCreate role - Go to the
Instancessection of the EC2 service in the AWS Console - Tick the box to the left of our cloud9 instance
- Click
Actions->Security->Modify IAM Rolethen chooseEC2FullAdminand clickSave - Go to the Cloud9 tab and click on the gear in the upper right hand corner
- In the Preferences tab go to AWS Settings on the left then turn off AWS Managed Temporary credentials
- Close the Preferences tab
Then go back to the Terminal tab in our Cloud9 and:
- Run
aws configure set default.region ap-southeast-2to set our default region to Sydney - Run
curl -Lo copilot https://github.com/aws/copilot-cli/releases/latest/download/copilot-linux && chmod +x copilot && sudo mv copilot /usr/local/bin/copilotto install Copilot - Run
cd ~/environment/docker-ecs-immersion - First we'll create our application by running
copilot app init- Enter
nyancatas the name of our application
- Enter
- Then we'll create our environment by running
copilot env init- Enter 'dev' as the name of the environment
- Use the down arrow to select
[profile default]for the credentials and press Enter. This will use the default credentials of the IAM Role assigned to our EC2 instance. - Press Enter to select
Yes, use default.for the network and press Enter. We could instead customise the CIDRs or choose and existing VPCs/subnets if we wanted here but for our workshop we'll let it create a new network with its default settings.
- Next, we'll create or service by running
copilot svc init- Use the down arrows to select
Load Balanced Web Serviceand press Enter. - Enter
wwwfor the name - Use the down arrow twice to select
Enter custom path for your Dockerfileand press Enter - Enter
/home/ec2-user/environment/docker-ecs-immersion/aws-cdk-nyan-cat/nyan-cat/Dockerfilefor the path - Press Enter to select the default port of 80
- This actually just generated a manifest file that we can use to deploy. Have a look at it with
cat copilot/www/manifest.yml. The schema available to you to customise this service is documented at https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/
- Use the down arrows to select
- Finally, we'll deploy our new service to our new environment by running
copilot svc deploy --name www --env dev. This will:- Build the container locally with a
docker build - Push it to the Elastic Container Registry (ECR)
- Deploy it as a Service (which scales/heals ECS Tasks similar to an Auto Scaling Group on EC2) on ECS Fargate behind an ALB
- Output the URL to the new ALB you can go to in your browser to see our nyancat container running on the Internet!
- Build the container locally with a
This was all actually done via CloudFormation and you can go to the CloudFormation service in the AWS Console and see separate stacks for the application, for the environment and for the service. If you go into those you can see the Templates that copilot generated and deployed for you. If you choose not not use copilot then you can be inspired by these Templates to make your own to manage ECS directly.
Rather than building our container on the machine that we run copilot CLI from (our laptop etc.), copilot can set up a CI/CD pipeline in the cloud based on AWS CodePipeline/CodeBuild for us too.
- Run
copilot pipeline init - Press Enter to accept the
devenvironment default - Press Enter to accept the repository
- Note the
buildspec.ymlandpipeline.ymlfiles that it generated - have a look at them.
The AWS Cloud Development Kit as we discussed also generates CloudFormation for you. But while copilot is focused and opinionated to keep things simple, CDK is a more general tool that can allow you to do literally anything that you can do in AWS however you'd like it done.
However, we have added higher-level constructs that build on the foundational constructs/classes of the CDK to get closer to the simplicity of copilot when it comes to many areas like ECS too.
- Run
cd ~/environment/docker-ecs-immersion/aws-cdk-nyan-cat/cdk - Run
cat lib/cdk-stack.ts - Note that there is one line to create the VPC (and this automatically will create a 'proper' VPC with public and private subnets and NAT gateways etc.), one to create the ECS Cluster and then a a construct called
ApplicationLoadBalancedFargateService. That construct will:- Build the container locally on the machine we're running the CDK on
- Create an ECR Repository and push it to that
- Create an ALB
- Create an ECS Fargate Task Definition
- And create and ECS Service to run/scale/heal that and manage its ALB Target Group for us.
- Run
nvm install --ltsto install the LTS version of node.js needed to run the CDK - Run
sudo npm install --upgrade -g aws-cdkto install the CDK via npm - Run
npm upgradeto update the package.json to the latest available package versions - Run
npm installto install the packages needed from npm - Run
cdk synthto generate the CloudFormation template from our CDK template - Note that this is the CloudFormation that the few lines of CDK we cat-ed above has turned into.
- Run
cdk bootstrapto create an S3 artifact bucket CDK needs - Deploy that with a
cdk deployto build our container locally, generate the CloudFormation template as above (as well upload it to the artifact bucket) and then call the AWS APIs to deploy that CloudFormation template. As part of that the local cdk CLI will also push the local image it (re)builds up to the new ECR it creates as part of that process as well. - Answer
yto the security confirmation and press Enter (it will show you any IAM and Security Group/firewall changes that will happen if you proceed)
(Optional from here)
First we'll show how to build an IIS container to host nyancat on Windows instead of our nginx container we used on Linux as well as the local Docker experience on Windows. We'll spin up a Windows bastion host to test this on - but the Docker experience should be similar to that on your Windows laptop/desktop.
Create and log into a Windows EC2 Instance:
- Go to the EC2 Service in the AWS Console
- Go to
Instanceson the navigation pane on the left side - Click the orange
Launch instancesbutton in the upper right - In the search box type
containerand press Enter then click the blueSelectbutton next toMicrosoft Windows Server 2019 Base with Containers - Choose the
t3.largeinstance type in the list and once that is selected click theReview and Launchbutton - Click the
Launch buttonthenCreate a new key pairfrom the dropdown then enter the nameworkshop-windows-bastionand clickDownload Key Pair - Then click the blue
Launch Instancesbutton - Go back to the EC2 Instances view and tick the new instance there to select it
- Click
Actions->Security->Modify IAM Rolethen chooseEC2FullAdminand clickSave - Click the
Connectbutton then choose theRDP clienttab then click theDownload remote desktop filebutton and then clickGet password - Browse to the certificate file you just downloaded and click
Decrypt Password - Copy the Password that has been revealed to your clipboard and then open the .RDP file you downloaded to log into the Windows Bastion
Use Windows Docker locally on the instance:
- Open PowerShell
- Run
docker versionto see that Docker for Windows is built-in to this AMI and ready to go - Run
docker run --rm -d --name iis -p 8080:80 mcr.microsoft.com/windows/servercore/iisto run stock IIS - Note how huge the image is and how long it takes it to both download and extract - 1/2 of Windows needs to be within the Windows container images and this makes them slow to pull/start/scale/heal compared to Linux
- Open Internet Explorer and Go to
http://localhost:8080. You will see the default IIS site - Run
docker stop iisto stop the container
Now lets launch the nyancat content:
- In PowerShell
Run
Invoke-WebRequest -uri "https://github.com/jasonumiker/docker-ecs-immersion/raw/main/nyancat-windows.zip" -Method "GET" -Outfile nyancat-windows.zip - Unzip the files. Run
Expand-Archive -Path nyancat-windows.zip -DestinationPath C:\nyancat-windows - Run cd
c:\nyancat-windows - Run
cat Dockerfileand see how Dockerfiles on Windows are similar but you use Powershell instead of the unix shell - Run
docker build -t nyancat-windows .to delete the default site and put our nyancat in its place - Note how since we had already download the IIS base layers we are building on they were cached really speeding things up
- Run
docker run --rm -d --name nyancat-windows -p 8080:80 nyancat-windowsto run our nycat on Windows via IIS - Go to
http://localhost:8080in IE and see it running - Run
docker stop nyancat-windowsto stop the container
Push our new nyancat-windows image to ECR:
- Install the AWS CLI by running
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msiin Powershell - Restart Powershell so the AWS CLI is now in the PATH
- Run cd
c:\nyancat-windows - Go to the AWS Console and go to the Elastic Container Registry (Amazon ECR)
- Click the orange
Get Startedbutton - Type
nyancat-windowsfor the repository name then clickCreate repository - Enter the repository then click the
View push commandsbutton in the top right of the screen - Select the Windows Tab
- Copy the first command and paste into PowerShell in your RDP session. This will log in
- Copy and paste the third command to re-tag our image (we already did the build in step 2)
- Copy and paste the fourth command to push the image
Now we'll take our nyancat container and run it on ECS
Go back to your terminal in Cloud9 and:
- Run
nvm install --ltsto switch back to the lts version we installed above (since it is already installed this should be quick) - Run
cd ~/environment/docker-ecs-immersion/windows - Run
pip install -r requirements.txtto install the necessary python packages from pip. - Run
cdk deploy - Go to the ALB address output at the end of the deployment to see our nyancat hosted by IIS on Windows via ECS