Importing existing resources

We have a production RDS database that is not managed by convox. It would be nice if we could incorporate it into our application resources. However, creating a new database through convox and migrating it will require too much downtime. Is there a way we could either:

  • initialize a convox database from a snapshot of another database, or
  • import an existing database as a convox resource?
3 Likes

What are the benefits of managing the database via Convox, when it isnā€™t ephemeral/scalable in nature?
Isnā€™t it better to manage it outside of Convox?

I suppose so. What I appreciate about a Convox-managed database is that it is bound to the proper VPC and the cx proxy command can provide secure access. I know thatā€™s stuff I can set up myself, but the value of Convox is not having to manage that myself and risk mis-configuring it.

Another benefit is that as we regularly create and destroy non-production environments, and those databases are ephemeral. Is there a way to make some distinction between the two in convox.yml?

1 Like

Something that I really appreciate is being able to proxy to the resources - that is very useful.

It turns out that you can initialize a database from a snapshot using the DatabaseSnapshotIdentifier option, at least when creating a gen 1 (convox rack resources) resource. Iā€™m investigating if this works for gen 2 (convox.yml-defined) resources as well.

@julian did you find any solutions w.r.t. gen 2 resources? (inquiring before I investigate myself :laughing:

@brandontruong Right now we are using a hack:

We define a gen-2 resource in our convox.yml which is used for most environments. Then we have our production resource as a gen-1 resource, and during deployment to production, or CI job rewrites our convox.yml file to remove the gen-2 definition.

Itā€™s particularly fragile, because if someone tries to deploy to production manually, it will end up creating a new gen-2 resource and sending all traffic to it. Havenā€™t found a solution to this yet. I asked about adding a option to disable gen-2 resources via env vars, but no response.

Iā€™m still open to other suggestions on how to solve this.

1 Like

Bumping this as weā€™re migrating from gen1 to gen2 right now and now faced with the same problem.

I am surprised that itā€™s not possible to define a snapshot ARN for app-based postgres like you can for rack resources.
Likewise is there no solution to disable resources or services? With gen-1 we used the ā€˜-1ā€™ scale parameter but that now fails.

Our convox.yml got complicated enough per-environment that we started generating it with code for our main app. Itā€™s still not great but thereā€™s less concern of accidentally creating an app-level resource in production.

Since the initial post, Iā€™ve learned some go and I donā€™t think adding support for disabling app-level resources would be very difficult. It seems to me the Convox team has gen-2 in maintenance mode but will accept PRs.

I am surprised that itā€™s not possible to define a snapshot ARN for app-based postgres like you can for rack resources.

There is DMS which might work for you instead of initializing from a snapshot. IIRC it didnā€™t work for us for a very particular issue related to escaping Postgres JSONB columns but you may not run into the issue (and they may have fixed it by now anyway).

With gen-1 we used the ā€˜-1ā€™ scale parameter but that now fails.

IIRC with gen-2 services you can just set scale=0 instead of scale=-1. In gen-1, ā€œ-1ā€ disabled the load balancer also but since gen-2 uses ALB instead of ELB thereā€™s only one load balancer so thereā€™s no more need to it off to save money.

That doesnā€™t apply to resources though.

1 Like