Dockerized PostgreSQL log to both /log & `docker logs`? - database

Question

My PostgreSQL 11.6 is running inside a Docker container based on an existing image. Running docker logs my-postgres shows the log messages produced by the dockerized PostgreSQL instance.
Problem: I am trying to set up the system such that PostgreSQL logs the messages to a log file in /var/lib/postgresql/data/log and still be able to show the log messages when you run docker logs my-postgres.
Logging to a file works when /var/lib/postgresql/data/postgresql.conf was modified to have the following:
log_collector = on
log_directory = 'log'
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
log_rotation_age = 1d
log_rotation_size = 100MB
However, running docker logs my-postgres now shows
2020-02-23 18:01:49.388 UTC [1] LOG: redirecting log output to logging collector process
2020-02-23 18:01:49.388 UTC [1] HINT: Future log output will appear in directory "log".
and new log messages no longer appear here.
Is it possible to log to the log file and also show the same log messages in docker logs my-postgres?
docker-compose.yml
version: '3.3'
services:
my-postgres:
container_name: my-postgres
image: timescale/timescaledb:latest-pg11
ports:
- 5432:5432

Answer 1

By default docker uses json-file logging driver. This driver saves everything from container's stdout and stderr into /var/lib/docker/containers/<container-id>/<container-id>-json.log on your docker host. The docker logs command just reads from that file. By default postgresql logs into stderr #log_destination = 'stderr'
You enabled the logging collector which catches the logs sent to stderr and saves them into filesystem instead. This is the reason why you don't see them anymore in docker logs output. I don't see anywhere in postgresql documentation how to send logs both to file and stderr. I'm no expert on psql though.
https://www.postgresql.org/docs/9.5/runtime-config-logging.html
Containers should log into stderr and stdout. Configure the cont. process to log into file inside container's filesystem is considered bad practice. You will lose the filesystem when container dies unless you attach the folder as volume.
If you insist your only chance is to change config line log_filename into something static like postgresql.log and create symbolic link (and rewrite the postgresql.log file created by psql) pointing it to it to stderr ln -fs /dev/stderr /var/lib/postgresql/data/log/postgresql.log.
I haven't tested the solution and I have certain doubts about loggin collector and its log_rotate capabilities. I have no idea what happens with postgresql.log when log file is rotated. Maybe it's copied and original file deleted / recreated by postgres and you'll lose your link. You can try to disable log_truncate_on_rotation boolean to overcome this but I'm not sure if this would help tbh.