I’m running into an issue on a new v3 rack where logging does not appear to be working (from my Rails app.) It looks like the logs are accessible via kubectl, but they are not getting forwarded to CloudWatch.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
resource-database-7b9cb5ddb5-vznrb 1/1 Running 0 98m
resource-redis-5fc7577b9d-h52tv 1/1 Running 0 98m
web-85c9d99477-4s2jl 1/1 Running 0 12m
web-85c9d99477-6tq4c 1/1 Running 0 12m
web-85c9d99477-hjxv7 1/1 Running 0 12m
worker-5894f7db9-62hbn 1/1 Running 0 12m
worker-5894f7db9-996zt 1/1 Running 0 12m
worker-5894f7db9-bssld 1/1 Running 0 12m
worker-5894f7db9-vtvxz 1/1 Running 0 12m
worker-5894f7db9-w7vvh 1/1 Running 0 12m
Logs are showing up for web-85c9d99477-hjxv7 if I use kubectl logs:
$ kubectl logs web-85c9d99477-hjxv7
Skipping iptables configuration. (Requires --cap-add=NET_ADMIN or --privileged)
02:37:35 web.1 | started with pid 18
02:37:37 web.1 | [18] Puma starting in cluster mode...
02:37:37 web.1 | [18] * Puma version: 5.3.1 (ruby 2.7.1-p83) ("Sweetnighter")
02:37:37 web.1 | [18] * Min threads: 5
02:37:37 web.1 | [18] * Max threads: 5
02:37:37 web.1 | [18] * Environment: production
02:37:37 web.1 | [18] * Master PID: 18
02:37:37 web.1 | [18] * Workers: 3
02:37:37 web.1 | [18] * Restarts: (✔) hot (✖) phased
02:37:37 web.1 | [18] * Preloading application
...
02:45:01 web.1 | I, [2021-09-17T02:45:01.643308 #44] INFO -- : [495e749b82326e28f358a47606941bb4] method=GET path=/users format=html controller=UsersController action=index status=200 duration=141.77 view=61.71 db=34.57 request_id=495e749b82326e28f358a47606941bb4 host=testapp.docspring.co source_ip=10.1.196.72 user_id=1 account_id=1 params={} basic_auth=false
02:46:18 web.1 | I, [2021-09-17T02:46:18.050010 #48] INFO -- : [79e44d3c623e815f7ab806097cc59b13] method=GET path=/templates format=html controller=TemplatesController action=index status=200 duration=512.33 view=256.04 db=68.76 request_id=79e44d3c623e815f7ab806097cc59b13 host=testapp.docspring.co source_ip=10.1.196.72 user_id=1 account_id=1 params={} basic_auth=false
However, the logs are not showing up in the CloudWatch log stream:
How could I debug this further and fix the issue? Thanks!