AWS Fargate with EFS

About a month ago, AWS announced that AWS ECS and Fargate support for AWS EFS File Systems is now generally available. This is pretty cool!

I've been a huge fan of Fargate for its simplicity. Now that EFS is available, it becomes easier to set up applications that rely on files for state persistence. And I've used EFS in the past and it's worked well, especially with EFS Infrequent Access.

So in my eyes, the combination of AWS Fargate with EFS could probably be one to the simplest and hassle-free scaleable infrastructure setups for any project that requires file persistence. Off the top of my head, I'm thinking setups like a few CMS's, as well as some database implementations.

Getting Set Up

For the last few projects, I've been using AWS CDK for most of my infrastructure setup. I'm a huge fan of being able to program infrastructure with logic and to add to that, they've done a lot of the hard work for you with their patterns established.

Though while trying it with CDK a few things came in the way.

1. At the time of writing this, EFS volumes can't be added to Fargate services through CloudFormation. There is a ticket for it and it's being actively worked on, so hopefully, this won't be an issue soon.
2. The workaround would be to create a task definition using a custom resource, and then link that to the service, though unfortunately there's currently a user can't create an ECS Service with an existing Task Definition, referencing the definition ARN.

So currently with the CDK, I had to use a custom resource for the task definition, and then a neat "escape hatch" in CDK to link the custom resource to the Fargate Service.

Here's a gist of the Fargate + EFS Volume CDK stack I ended up with:

The main pieces of interest:

Create a Fargate service using the @aws-cdk/ecs-patterns. Two things are different though, first you need to use the platform version 1.4.0, as by default it uses LATEST, which currently is 1.3.0.

Also pass along some configuration which will create a basic task definition. We need to do this as currently there's an issue with referencing a task definition by ARN.

const fargateService = new ApplicationLoadBalancedFargateService(
  this,
  "FargateService",
  {
    // ... other config
    // need platform version 1.4.0 to mount EFS
    platformVersion: FargatePlatformVersion.VERSION1_4,
    // Also send some config to create a temporary task definition.
    taskImageOptions: {
      image: ContainerImage.fromRegistry(containerImage),
    },
  }
);

Creating a file system, as well as a volume configuration that uses this file system and the corresponding mount points.

// Create the file system
const fileSystem = new FileSystem(this, "AppEFS", {
  vpc,
  lifecyclePolicy: LifecyclePolicy.AFTER_14_DAYS,
  performanceMode: PerformanceMode.GENERAL_PURPOSE,
  throughputMode: ThroughputMode.BURSTING,
});

const volumeConfig = {
  name: "efs-volume",
  // this is the main config
  efsVolumeConfiguration: {
    fileSystemId: fileSystem.fileSystemId,
  },
};

const mountPoints = [
  {
    containerPath: "/root",
    sourceVolume: volumeConfig.name,
    readOnly: false,
  },
];

Now create the task definition custom resource.

/*
  This object is the final task definition, which includes the
  EFS volume configurations. This definiton is created as a Custom Resource through
  the aws-sdk. This is a stop-gap measure that will be replaced when this
  capability is fully supported in CloudFormation and CDK.
*/
const customTaskDefinitionJson = {
  containerDefinitions: [
    {
      // ... other properties

      // specify the mount points above
      mountPoints,
    },
  ],
  // For fargate, need to use this network mode
  networkMode: NetworkMode.AWS_VPC,
  requiresCompatibilities: ["FARGATE"],
  // Finally add the volume configuration above.
  volumes: [volumeConfig],
};

Also, be sure to allow permissions between the service and the file system, or else the service will time out trying to mount the EFS volume.

fargateService.service.connections.allowFrom(fileSystem, Port.tcp(2049));
fargateService.service.connections.allowTo(fileSystem, Port.tcp(2049));

Now here's a trick I picked up from Michael Moosa's sample implementation, the ability to addPropertyOverride.

This saves us from having to create a custom resource for the service. With this method, we're able to overwrite the core service task definition with the newly created one.

Doing some reading, there's a few of these "Escape hatches" built into CDK, which you can use to overwrite values where needed.

(fargateService.service.node.tryFindChild(
  "Service"
) as CfnService)?.addPropertyOverride(
  "TaskDefinition",
  customTaskDefinition.getResponseField("taskDefinition.taskDefinitionArn")
);

And that's that. With this, you're able to use AWS CDK to create a Fargate service with and EFS volume.

I'm really looking forward to when this gets added to CloudFormation, but for the time being this is not a bad stop-gap solution.