How To Send Docker Logs From EC2 To CloudWatch

You can send logs from docker containers to AWS CloudWatch easily.

Docker has a built-in logging driver that can ship to CloudWatch.

Background

I was working on my side project this morning. It’s deployed as a single docker container running on an ec2 instance.

It quickly became annoying to shell onto the host to run docker logs -f to get logs.

I didn’t want to set up a heavy centralized logging solution like ELK or Loki or pay for Datadog, so I looked into forwarding the logs to CloudWatch.

It’s pretty easy to do.

Docker has the concept of log drivers built into it. One of the log drivers is awslogs. You can read the docs here.

All you have to do is set your logdriver, region, log group, and whether or not to create the group when starting your container.

docker run -it --log-driver=awslogs --log-opt awslogs-region=us-east-1 --log-opt awslogs-group=appName/environment/backend --log-opt awslogs-create-group=true ubuntu:latest

If you’re using an ec2 instance like me, you’ll also need to make sure its instance profile role has a policy that looks something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:CreateLogGroup",
                "logs:DescribeLogStreams"
            ],
            "Resource": [
                "arn:aws:logs:us-east-1:<account-id>:log-group:appName/environment/backend:*"
            ]
        }
    ]
}

One last, optional, thing is to set the CloudWatch log retention policy for your log group. There’s no option to set it with the log driver. You’ll have to modify the retention policy after the log group has been created in the AWS Console or via the CLI.

aws logs put-retention-policy --log-group-name "appName/environment/backend" --retention-in-days 30