All apps will break after April 1, 2021 due to ECS migration to the new ARN and resource ID format

[ali]

On August 21, 2020 AWS has announced the timeline for migrating ECS to the new ARN and resource ID format. This migration was previously announced here.

What this means is that after April 1, 2021, although existing resources will keep running, we won’t be able to scale up/down our existing apps nor would we be able to recover from any failing ECS tasks (i.e., attempting to create a new ECS task will result in AWS to use the new format but it will fail to do so because the ECS service which those tasks belong will not be using the new format in the first place.)

Convox doesn’t support recreating an ECS service (i.e., a convox app) anew without downtime.

So far there was only one discussion that I am aware of in the convox community which doesn’t say much related to how the migration should happen in existing convox gen2 racks.

One way to approach this is to employ a blue-green deployment between the existing app and a new app I should create with the new configuration (i.e., an ECS service using the new ARN and resource ID format). Blue-green deployment requires a considerable amount of work and potential for service disruption if not executed correctly. We should all be hoping to avoid this option unless it is a last resort. It’s especially painful for products and services with many apps.

If I am reading this right I wonder why the community isn’t as worried as I am and if there’s a better way to do this in convox that I am missing.

Thank you.

[edward]

Nope, I’m equally as worried.

I might be able to work around a few hours of downtime if necessary - but what I need to be reassured of is that gen1 and gen2 racks will support these new ARN formats at all, and that we’re not backing into a corner.

[ali]

As far as I know, support-wise, there shouldn’t be any concerns, however migrating existing resources from old formats to new ones is the tricky part (i.e., brand new convox apps must be created which, if not done via blue-green deployment, will generate downtime).

Whilst generating new apps one must also be careful of additional resources (e.g., databases) and make sure they’re not deleted and connection from the new apps are maintained.

If you were to not make the migration, I am afraid as soon as you’ll want to scale your apps, or a process will have to be recreated due to a failure, you’ll have issues, that is after April 1st.

I am still hopeful we’ll hear some better solutions on how to deal with this from the community.

[ed_convox]

@ali and @edward

Sorry for the delay but this thread has just been brought to my attention! There really isn’t any need to worry, once you’ve opted in to the new ARN format through AWS, then any update (a param update, a redeploy) to your existing apps would move to the new format without downtime. You don’t need to do any blue/green deploys etc. We’ve supported this for years, and even created the TaskTags app param to propagate tags down to your Service level for once you have opted in (https://docsv2.convox.com/management/service-tags).

You can opt in to the new format through your AWS Console or from the CLI:

aws ecs put-account-setting-default --name serviceLongArnFormat --value enabled
aws ecs put-account-setting-default --name taskLongArnFormat --value enabled
aws ecs put-account-setting-default --name containerInstanceLongArnFormat --value enabled

Thanks,
Ed

[ali]

Reporting back here for the sake of feedback to the community: Indeed after having opted-in the root account, a simple update to TaskTags param was enough to recreate ECS service and tasks with the new longer formats.

However, FWIW, I am not so sure about a redeploy does the same.