How to upload docker logs from EC2 instance to CloudWatch before shutdown by Raj Gupta

When you are using an auto scaling group (ASG) on AWS, and sometimes a docker container running in an EC2 instance exits due to some ambiguous reason and the instance may get removed from the ASG. This makes debugging the failure difficult since the ASG terminates the instance and therefore erasing all the evidence of what went wrong. So, below are the way to write docker logs to CloudWatch before it exits.

1. First you attach a role to ec2 server in which docker are running, So that it has permission to write the log to CloudWatch

2. Now create a Log Group from cloudwatch dashboard

3. Now logging into your EC2 server and enter your credentials in file /etc/init/docker.override

[root@ip-172-31-46-121 ~]# vi /etc/init/docker.override

env AWS_ACCESS_KEY_ID=AKIAIUMPX5TCNGRG5RXA

env AWS_SECRET_ACCESS_KEY=qOjKwGQBxwOmZy/yVY/UcROUsVIcMw8pn1RBJLBB

after the save and close the file.

4. Now run the below command to write your docker log to cloudwatch log Group Raj

[root@ip-172-31-46-121 ~]# docker run -it --log-driver="awslogs" --log-opt awslogs-region="us-east-1" --log-opt awslogs-group="Raj" --log-opt awslogs-stream="log-stream" ubuntu:14.04 bash

root@4adc2e0120e6:/#

5. initially if do not have any activity to docker then log group in cloudwatch have no any data like below

root@4adc2e0120e6:/#

6. Once we start any activity to docker then all the log of docker go to log group in cloudwatch like below let say in our case we want to come out of docker then it log will be go to cloudwatch

root@4adc2e0120e6:/# exit

exit

like that all the activity perform on docker will record to cloudwatch log