Recently, I noticed that whenever my server, banan, was rebooted, containers running under the git user did not automatically start.
❯ podman --version
podman version 5.6.0
❯ cat /etc/os-release
NAME="AlmaLinux"
VERSION="9.7 (Moss Jungle Cat)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.7"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.7 (Moss Jungle Cat)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"
ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
ALMALINUX_MANTISBT_PROJECT_VERSION="9.7"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.7"
SUPPORT_END=2032-06-01
Let’s investigate and fix this issue.
After switching to the git user, I encountered the following error when trying to list the running containers:
[git@banan ~]$ podman ps
Error: current system boot ID differs from cached boot ID; an unhandled reboot has occurred. Please delete directories "/tmp/storage-
run-1006/containers" and "/tmp/storage-run-1006/libpod/tmp" and re-run Podman
Using the hints provided in the error message, I removed the stale directories and checked if the podman ps command would work:
[git@banan ~]$ ls /tmp/
containers-user-1006 systemd-private-53b233e792524efe948da9290a3612e3-chronyd.service-hOGp0j
...
[git@banan ~]$ rm -rf /tmp/storage-run-1006/containers
[git@banan ~]$ rm -rf /tmp/storage-run-1006/libpod/tmp
[git@banan ~]$ podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
The command worked. However, the problem persisted: after starting the containers and rebooting the server, they would fail to start automatically. To find a permanent solution, I turned to Google and found a discussion [1] indicating this is a common issue for systems where the /tmp directory resides on an XFS filesystem.
[git@banan ~]$ podman system info | grep -i runroot
runRoot: /tmp/storage-run-1006/containers
The suggested solution from [1] is to ensure the temporary directories created by the containers are removed when the server starts. To do this, I copied the default podman.conf and added a configuration to remove the stale directories on boot.
[root@banan ~]# cp /usr/lib/tmpfiles.d/podman.conf /etc/tmpfiles.d/podman.conf
Then, I edited /etc/tmpfiles.d/podman.conf to add the following lines:
# [2026-01-17:SS] fix issue with cached boot ID after OS reboot
# https://github.com/containers/podman/discussions/23193
R! /tmp/storage-run-*/containers/
R! /tmp/storage-run-*/libpod/tmp/
After applying this change, I was no longer able to reproduce the issue.
References: