Can't create new app in version 20190509115116

I have a pre-existing rack. It is several months old and working fine.

Recently, I updated the rack to version 20190509115116. Redeployed my old apps to update the Lambda version to 8. Also updated the Console Webhook to node 8. Worked fine, and all the apps are working fine.

Now I create a new a new app -

convox apps create APPNAME -g 2 -w

Works fine. I can see the app using convox apps command.

Now I set environment variables -

convox env set KEY=VALUE -a APPNAME

Works fine. I can see the env vars using convox env -a APPNAME command.

Now I deploy my code using -

convox deploy -a APPNAME -w

I get the mail to verify the new domain name. Works fine, but then the deployment gets stuck. I investigate by running -

convox logs -a APPNAME

Here is the output I see -

2019-05-20T11:39:23Z service/web:RYDLFRAIBRS/7076fc633c50 ERROR: could not fetch convox env: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
2019-05-20T11:39:23Z service/web:RYDLFRAIBRS/7076fc633c50 status code: 403, request id: D9696C323E629CEC, host id: LAid23/xB85ljCoR0SKkV/YfpHiF0Sn2FVVcWuFxrHWbPeZqFYMvcGNbru944QsCFT1CJ064Hbk=

On investigating the error message, I believe it originates from this line.

How to resolve this issue?

Issue resolved. The fault was on our end.

We had hardcoded an AWS Access Key ID and AWS Secret Access Key into the Dockerfile for development purpose and had forgotten to remove it, even though we had deleted the Key in AWS IAM.

Fixing the Dockerfile, fixed the issue.

In case anyone else runs across the same issue, the problem for us was that we were defining a bogus AWS_ACCESS_KEY_ID just so we could run bundle exec rails assets:precompile. We were running into this issue when we had ENV AWS_ACCESS_KEY_ID="bogus" in the Dockerfile and fixed it by changing the precompile step to just pass in the bogus env variable directly. ie: AWS_ACCESS_KEY_ID=bogus bundle exec rails assets:precompile.