Gitlab failed upgrade postgresql 12
I run my gitlab with podman. This has been working good. I recently ran in to problem with upgrade to 13.8. With this version postresql default version is 12.x.
I analyzed why podman image did not start, and found out that it failed on upgrade of postgresql from 11 to 12. Message was kind of cryptic:
Creating temporary data directory:Error creating new directory: /var/opt/gitlab/postgresql/data.12
First I was thinking it was permission, that was wrong. I tried to premake the directory, same error. This did not make any sense. After some internet search I found a work around to skip upgrade. This can be done by starting container with:
With this I was able to start the container, but database did not get upgraded. I attached to the container with:
podman exec -ti gitlab /bin/bash
And tried to upgrade database manual with:
gitlab-ctl pg-upgrade -w
I got same error as before, but this time I looked closer. And see following:
STDERR: su: System error
Looks like the container do not accept to change user. When i tried to change user with sudo I got following message:
sudo: unable to send audit message: Operation not permitted
So actually it is auditing of change user that fail. This is probably something that need to be fixed in the gitlab-ce image. For now you can get around it by allow auditing by run container with:
I added this to my podman run command and removed
-e GITLAB_SKIP_PG_UPGRADE=true and now when I run the container it upgrades the database how it is supposed to and all is good again.