# Feedback on v3 Racks? **URL:** \ **Category:** Rack (Version 3)\ **Created:** [March 31, 2020, 11:38am UTC](https://community.convox.com/t/feedback-on-v3-racks/728 "2020-03-31T11:38:53Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ed\_convox](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.convox.com/ed_convox/32/101_2.png) [@ed\_convox](https://community.convox.com/u/ed_convox)\ **Post date:** [March 31, 2020, 11:38am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/1 "2020-03-31T11:38:53Z")
With the Kubernetes-based Racks being released for a few weeks now, we’re wondering who has tried them out? If you have, how have you found them? If you haven’t, what’s stopped you from doing so? Free credits are available from many of the cloud providers - $200 from Azure, $300 from GCP, AWS free tier, and Digital Ocean just being extremely competitive on price (look out for an upcoming blog post on this topic!) - this means that it’s extremely frictionless to get a Rack going for testing. Many users are now running their production apps on v3 Racks, and experiencing the benefits of doing so. Deploys, rollbacks and scaling can be an order of magnitude faster. Running a staging Rack on a cheaper cloud provider makes a lot of sense. Support for custom load balancing allows for more service configuration options. Is there anything else you’d like to see in the new Racks? Thanks 😊 ---
**Author:** ![sam](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@sam](https://community.convox.com/u/sam)\ **Post date:** [April 1, 2020, 7:31am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/2 "2020-04-01T07:31:27Z")
- Support for RDS (Database resources) / Elasticache (Redis resources) / Other Cloud Provider equivalents? - Support for AWS Certificate Manager based SSL certificates (or other cloud provider equivalents), while Lets Encrypt is a great tool, its not a perfect fit for all use cases and sometimes you need something else. - Support for using spot instances (AWS) and equivalents on other clouds out of the box - The documentation needs more work. Thats all I can think of at the moment, but I’ll probably think of more once I’ve started using it more heavily in the next few weeks. Edit 1: - Support for managing Kubernetes pod resource limits. [Resource Management for Pods and Containers | Kubernetes](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) ---
**Author:** ![ed\_convox](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.convox.com/ed_convox/32/101_2.png) [@ed\_convox](https://community.convox.com/u/ed_convox)\ **Post date:** [April 1, 2020, 11:34am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/3 "2020-04-01T11:34:16Z")
Thanks @sam, that’s great feedback thanks! I’ll definitely look at the docs for you and we’ll discuss the other bits 😊 ---
**Author:** ![sam](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@sam](https://community.convox.com/u/sam)\ **Post date:** [April 8, 2020, 3:30am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/4 "2020-04-08T03:30:23Z")
More doc feedback for you @ed_convox - The documentation only covers install using the command line, it should also mention the web UI (and likely include a mention of what happens to the terraform state in this case) - The documentation doesn’t include any explanation how to add a v3 rack that was created with the CLI into the web console. (I worked it out with an educated guess, but it really should be in the documentation so people don’t have to guess about basic setup tasks) Edit (more stuff) - The documentation for deployment could do with some more of the “obvious headings” that were in the v2 docs. Like the “Creating an application” section in the v2 docs. The info is in there, but this is another instance of the v3 docs being in need of work to improve their usability. - After importing a v3 rack created via the CLI there appears to be no usage statistics in the dashboard. ( I have not created a v3 rack using the dashboard to see if this is different) - If this is normal, it should be documented somewhere. ---
**Author:** ![ed\_convox](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.convox.com/ed_convox/32/101_2.png) [@ed\_convox](https://community.convox.com/u/ed_convox)\ **Post date:** [April 8, 2020, 2:13pm UTC](https://community.convox.com/t/feedback-on-v3-racks/728/5 "2020-04-08T14:13:24Z")
Hey @sam, thanks for the further feedback! On those points: - we explicitly tried to keep the Console stuff out of the initial docs to keep things cleaner and (hopefully) less confusing for new people to the platform. Initially, v3 installation wasn’t available in the Console either, but is now! Console docs are available from within the Console itself but they could also do with beefing up. - `rack mv` reference docs are now in there: [https://docs.convox.com/reference/cli/rack#rack-mv](https://docs.convox.com/reference/cli/rack#rack-mv) but I agree a more prosaic explanation would be good! - What other ‘obvious headings’ would you be looking for? We do have the Local Development page ([https://docs.convox.com/tutorials/local-development](https://docs.convox.com/tutorials/local-development)) which covers getting you up and running with an app, and I’m bringing in the Example Application documentation (and extending it) shortly. - stats for v3 Racks are coming shortly 😊 ---
**Author:** ![scott1](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.convox.com/scott1/32/131_2.png) [@scott1](https://community.convox.com/u/scott1)\ **Post date:** [April 8, 2020, 2:15pm UTC](https://community.convox.com/t/feedback-on-v3-racks/728/6 "2020-04-08T14:15:05Z")
For us, it was the following: - The cost. The blog post seems to miss the AWS NAT cost which we found to be about $100/month. Perhaps we configured something wrong or calculated it wrong, but in talking with David, it seems 3 NATs are the default and they’re not cheap. - The lack of metrics. Since v3 racks aren’t using the ALB, we lose the ALB routing metrics. On top of that, convox metrics don’t work. We have app metrics, but with AWS, we need to know metrics from the point of ingestion – e.g. we need to answer the question is AWS sending back 502 before the requests ever reach our apps? If we don’t have that, we’re flying blind. To add metrics, we estimated the cost of using data dog to $200/month for us. For a rack with a base cost of $244 (by our estimation), adding the analytics was both an extra cost and maintenance for us. Also, the convox stats are very helpful, but for us, the critical part is knowing if the load balancer/ingress point is responding with 50x before the request hits our app server. The current v2 stats don’t show that (though they are useful in other regards). Other than those two things, we really loved the v3 racks. I would echo the question about manually configuring k8s – we had to do some custom configuration to make datadog work for us. We knew that if we updated the rack version, we’d have to reapply those changes and we didn’t love that part. But, we kinda assumed that mucking around sometimes would be inevitable. ---
**Author:** ![sam](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@sam](https://community.convox.com/u/sam)\ **Post date:** [April 14, 2020, 4:26am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/7 "2020-04-14T04:26:33Z")
Speaking of stats, metrics. and so on. The current V3 documentation for DataDog is broken. Not because the documentation is wrong, but because v3 cannot deploy DataDog correctly. The current code for deploying an agent, while it makes a DaemonSet and does things mostly correct, it tries to setup some (at least in the case of DataDog) unnecessary service related stuff, which results in scheduling 3 pods, but only one running pod, as the other two fail to schedule producing this error from the cluster scheduler. > `Warning FailedScheduling 57s (x5661 over 5d21h) default-scheduler 0/3 nodes are available: 1 node(s) didn't have free ports for the requested pod ports, 2 node(s) didn't match node selector, 3 Insufficient pods.` In addition, if it was working, as a normal Convox based “agent service” it can’t support the Kubernetes Cluster Agent features, and also cant enable the datadog Network performance monitoring features due to lacking the ability to deploy multi-container pods (which the network performance monitoring uses) As a workaround to these things, I’m using the official DataDog Helm charts, but I’m also looking at the future DataDog operator. So better support / integration with Helm and Operators is definitely rising in importance the more I use v3 racks. ---
**Author:** ![sam](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@sam](https://community.convox.com/u/sam)\ **Post date:** [April 14, 2020, 8:19am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/8 "2020-04-14T08:19:08Z")
More bugs/feedback for you @ed_convox , in addition to the DataDog agent bug, the v3 convox CLI seems to be unable to update rack parameters. It updates the local config files and terraform but attempts to execute them (using terraform under the hood) in such a way that `convox rack param fails due to terraform discovering the existing resources instead of applying it as a modification. I could be wrong since I’m relatively new to terraform and I’m trying to reverse engineer these things from the go source code and the templates in the convox repo, but… From the look of it, this issue might be fixed in the terraform AWS EKS module, [https://github.com/terraform-aws-modules/terraform-aws-eks/pull/625](https://github.com/terraform-aws-modules/terraform-aws-eks/pull/625) but this requires the AWS provider `>= 2.52.0` (the convox terraform templates/sources specify `~> 2.49`) and I think, to specifically designate that you want to use the EKS module from the registry… (per the instructions [https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/11.0.0](https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/11.0.0) on ) which I don’t think the current terraform templates do. Regardless of that, its definitely some kind of underlying terraform issue, since manually executing terraform apply in the root of the local terraform state folder causes the same error. So it looks pretty clear that its something to do with the terraform state that convox is trying to apply. ---
**Author:** ![keith.chau](https://avatars.discourse-cdn.com/v4/letter/k/d6d6ee/32.png) [@keith.chau](https://community.convox.com/u/keith.chau)\ **Post date:** [September 5, 2022, 4:15am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/9 "2022-09-05T04:15:06Z")
> [@sam](#): > > - How well does Convox play with any manual Kubernetes management? Can I use kubectl to directly schedule work into the cluster without Convox having any issues? For instance if I wanted to deploy a Kubernetes operator from a vendor like [MongoDB](http://docs.mongodb.com/kubernetes-operator/master/) or [Elastic](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html) … or start using a tool like [KubeDB](https://kubedb.com) to have more database/resource options in my Kubernetes cluster ([Their supported tools list](https://kubedb.com/docs/v0.13.0-rc.0/concepts/)) Is there any additional info around this? I am looking into deploying a MongoDB Kubernetes operator with our setup. ---
**Author:** ![nathan.f77](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.convox.com/nathan.f77/32/256_2.png) [@nathan.f77](https://community.convox.com/u/nathan.f77)\ **Post date:** [November 8, 2022, 3:42am UTC](https://community.convox.com/t/feedback-on-v3-racks/728/10 "2022-11-08T03:42:10Z")
> Is there anything else you’d like to see in the new Racks? I have some feature requests for v3 racks. Here are the main ones: - Support for setting `proxy-buffer-size` in ingress nginx config. I manually set this annotation on the web ingress: `nginx.ingress.kubernetes.io/proxy-buffer-size: 32k`. It would be very nice if I could set this as a rack param. (I was seeing a lot of 502 errors with the default 4k buffers.) - Support for DNS01 challenges in cert-manager for LetsEncrypt - This is really handy when migrating to a new Convox rack, so that I can issue SSL certs before switching the DNS records. Would make it much easier for me to set up blue/green deployments for safe Convox release updates.