Deploying event driven ECS Tasks using AWS Evenbridge and Fargate

Related services/technologies on this post:
  • AWS ECS
  • AWS Fargate
  • AWS Eventbridge
  • Terraform

When building your app architecture, keeping components loosely coupled can become a difficult task very fast. Designing how your application components connect to each other can become a challenge on its own, not even taking into consideration the implementation side of this design, that can have its own challenges too. In this post, we will go over how we can use several AWS services such as Amazon Eventbridge and AWS Fargate, to implement loosely coupled, event driven microservices with a straightforward and repeatable solution using AWS ECS.

Amazon Eventbridge (formerly Cloudwatch Events) provides us with several event driven options that allow it to send real time data to an AWS Service or even SaaS target via API, depending on the condition or rule you establish. AWS Eventbridge has many use cases such as ingesting or sending data to and from AWS Services and external services or as we will show in this post, triggering an ECS task on a set schedule.

On the other hand, AWS Fargate service provides us with serverless operation for our containers. Fargate serves as a compute engine for your applications, without the hassle of having to manage your own infrastructure. Although more costly than EC2, the main benefit we get from Fargate is that we can stand up containers without having to rely on EC2 all the time. Using Fargate is a great alternative when we can accurately forecast our resources usage and lifespan of the container we want to bring up and we don’t want to compromise the resources of other tasks or services running on our EC2s, keeping our cost reasonable while not risking our application’s performance.

For example, let’s say you have a new container task that you want to run every day at 8am within your ECS Cluster however, the amount of instances that you have and their available memory seems to be pretty tight, and you don’t want to compromise or risk any placement issues of new tasks when there’s a deployment or more tasks need to be added. You can use Fargate as the launch method for your ECS task so you can still get your container running, without risking any impact to your EC2 running tasks.

Similar to the example mentioned above, we will be showing how to launch an ECS Task with AWS Fargate as the launch method for the task instead of EC2, due to the low resource consumption that we know the container requires and cadence of the task to be ran inside one of our ECS cluster.

It’s important to note that the following steps have some basic Terraform snippets that will allow us to create these resources via code, opposed to doing this manually via the AWS Console (which is totally fine and you can create all these same resources manually if preferred). As an IaC (Infrastructure as Code) tool, Terraform gives us the ability to scale and replicate infrastructure more easily, which is why it’s such a great tool when you start adding a lot of resources to your infrastructure.

1. First, you need to define your ECS task as you would normally do with a regular ECS service. Our Eventbrifge rule will be the one to run this task, so we won’t be needing an ECS Service for our new task definition.

resource "aws_ecs_task_definition" "task_1" {
 family                   = "${var.rule_name}_${var.environment}"
 container_definitions    = var.container_definition
 requires_compatibilities = ["FARGATE"]
 cpu                      = 512
 memory                   = 1024
 task_role_arn            = aws_iam_role.task.arn
 execution_role_arn       = aws_iam_role.execution.arn
 network_mode             = "awsvpc"
}

 

Things to consider in step 1:

  • Ensure you add the required compatibility for Fargate, that we’ll be using as our launch type.
  • Since we will be using Fargate for this Eventbridge rule and task, ensure your port mappings values for host port and container port match.
  • CPU and Memory fields are also required for Fargate launch type. Make sure your values are under the available supported configurations here

  2. Once you have defined your ECS Task definition, you can move into setting up the required Eventbridge resources. First we will set up the rule that will trigger our task:

resource "aws_cloudwatch_event_rule" "rule_1" {
 name                = "${var.rule_name}_${var.environment}"
 description         = var.rule_description
 schedule_expression = rate(1 hour)
}

 

3. Once we have our rule, we can move into setting up the target for our rule, which in this case is the ECS Task we defined in step 1:

resource "aws_cloudwatch_event_target" "target_1" {
 arn      = var.cluster_arn
 rule     = aws_cloudwatch_event_rule.rule_1.name
 role_arn = aws_iam_role.eventbridge_rule.arn

 ecs_target {
   task_count          = 1
   task_definition_arn = aws_ecs_task_definition.task_1.arn
   network_configuration {
     subnets         = var.vpc_subnet_ids
     security_groups = [var.security_group_id]
   }
   launch_type = “FARGATE”
 }
}

 

Things to consider in step 3:

  • You need to ensure that the role ARN you will be giving this rule, has the ability to perform the following actions:
    • iam:PassRole: over the task’s execution and task roles.
    • ecs:RunTask: over the family of your previously defined task definition.
  • Since we will be using awsvpc as the network mode for this task, you will need to ensure you are providing the value for network_configuration on your ecs_target block.

Once you create these resources (and any other dependencies such as a dedicated target group for your task if needed) you should be able to monitor your rule invocations using Cloudwatch metrics. You can access these by going into your rule and checking for any failed invocations metric for your event.

 

If you do have any failed invocations, the easiest way to know the root cause of the failure is to use AWS Cloudtrail, using your task definition or target role as filters to check on the error message during target invocation.

One thing to consider is that AWS Lambda could give you similar benefits, as it is serverless and allows you to decouple to some extent the components in your application however with with Eventbridge you can have much more complexity and allows for reutilization of resources such as already existing containers without having to go and adjust the code or target of the event.

In summary, setting up a small event-driven component for your application is pretty simple once you integrate and make use of services such as AWS Eventbridge. Adding Fargate to the mix is just a great way to use serverless compute as well to decrease your failure points and spread your infrastructure.

Share on facebook
Share on twitter
Share on linkedin

Do You Want To Boost Your Business?

Drop us a line and keep in touch.

How Mismo Works

Jobs

About

Resources