[Summary] USER_UID/uid related problems (snap/docker) #10478

@bd4 commented on GitHub (Apr 27, 2023):

I did not test carefully last time I upgraded, but when I rolled back to gitea/gitea:1.18-rootless, the problem goes away. I have a user/group gitea on host system, with uid/gid 1016/1017, and my docker-compose looks like this:

version: "3"

networks:
  gitea:
    external: false

services:
  server:
    image: gitea/gitea:latest-rootless
    container_name: gitea
    user: "1016:1017"
    restart: unless-stopped
    networks:
      - gitea
    volumes:
      - ./data:/var/lib/gitea
      - ./config:/etc/gitea
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3001:3000"
      - "2222:2222"

Maybe it is enough to just have user: set in docker-compose and upgrade to 1.19? In config/app.ini, I have:

APP_NAME = Gitea: Git with a cup of tea
RUN_USER = git
RUN_MODE = prod

Not sure if RUN_USER has anything to do with it.

@bd4 commented on GitHub (Apr 27, 2023): I did not test carefully last time I upgraded, but when I rolled back to `gitea/gitea:1.18-rootless`, the problem goes away. I have a user/group `gitea` on host system, with uid/gid 1016/1017, and my docker-compose looks like this: ``` version: "3" networks: gitea: external: false services: server: image: gitea/gitea:latest-rootless container_name: gitea user: "1016:1017" restart: unless-stopped networks: - gitea volumes: - ./data:/var/lib/gitea - ./config:/etc/gitea - /etc/timezone:/etc/timezone:ro - /etc/localtime:/etc/localtime:ro ports: - "3001:3000" - "2222:2222" ``` Maybe it is enough to just have user: set in docker-compose and upgrade to 1.19? In `config/app.ini`, I have: ``` APP_NAME = Gitea: Git with a cup of tea RUN_USER = git RUN_MODE = prod ``` Not sure if RUN_USER has anything to do with it.

GiteaMirror commented

@bd4 commented on GitHub (Apr 27, 2023):

To be more clear, doing docker-compose pull followed by docker-compose up -d to upgrade to 1.19 from 1.18 reproduces, and rolling back to 1.18 by changing the image from gitea/gitea:latest-rootless to gitea/gitea:1.18-rootless fixes it.

@bd4 commented on GitHub (Apr 27, 2023): To be more clear, doing `docker-compose pull` followed by `docker-compose up -d` to upgrade to 1.19 from 1.18 reproduces, and rolling back to 1.18 by changing the image from `gitea/gitea:latest-rootless` to `gitea/gitea:1.18-rootless` fixes it.

GiteaMirror commented

@bd4 commented on GitHub (Apr 27, 2023):

It is running on Ubuntu 22.04, but no snaps - just have docker and docker-compose installed and manage some docker instances manually.

@bd4 commented on GitHub (Apr 27, 2023): It is running on Ubuntu 22.04, but no snaps - just have docker and docker-compose installed and manage some docker instances manually.

GiteaMirror commented

@bota87 commented on GitHub (Apr 28, 2023):

I have probably the same issue as reported in #24385
I just tried with the latest release 1.19.2-rootless and the issue persist, if I remove user: "1016:1017" it works

@bota87 commented on GitHub (Apr 28, 2023): I have probably the same issue as reported in #24385 I just tried with the latest release `1.19.2-rootless` and the issue persist, if I remove `user: "1016:1017"` it works

GiteaMirror commented

@wxiaoguang commented on GitHub (Apr 28, 2023):

I have a question, why did you set user: "1016:1017" ?

@wxiaoguang commented on GitHub (Apr 28, 2023): I have a question, why did you set `user: "1016:1017"` ?

GiteaMirror commented

@bota87 commented on GitHub (Apr 28, 2023):

because user 1000 it's already used for another service and I want to keep permissions separated

@bota87 commented on GitHub (Apr 28, 2023): because user 1000 it's already used for another service and I want to keep permissions separated

GiteaMirror commented

ad03c6e0a3/Dockerfile.rootless (L40-L49)

@wxiaoguang commented on GitHub (Apr 28, 2023):

I'm not sure whether it's related (haven't looked into the problem at the moment):

Gitea's docker-rootless hard-coded 1000:

ad03c6e0a3/Dockerfile.rootless (L62)

@wxiaoguang commented on GitHub (Apr 28, 2023): I'm not sure whether it's related (haven't looked into the problem at the moment): Gitea's docker-rootless hard-coded 1000: https://github.com/go-gitea/gitea/blob/ad03c6e0a36033c6f59262d8cfd6416ae3cc93d6/Dockerfile.rootless#L40-L49 https://github.com/go-gitea/gitea/blob/ad03c6e0a36033c6f59262d8cfd6416ae3cc93d6/Dockerfile.rootless#L62

GiteaMirror commented

@bd4 commented on GitHub (Apr 28, 2023):

My understanding is that the USER from the dockerfile is the default, but it's it can be overridden by the user config in docker compose or appropriate docker run args: https://docs.docker.com/compose/compose-file/05-services/#user

@bd4 commented on GitHub (Apr 28, 2023): My understanding is that the USER from the dockerfile is the default, but it's it can be overridden by the user config in docker compose or appropriate docker run args: https://docs.docker.com/compose/compose-file/05-services/#user

GiteaMirror commented

@wxiaoguang commented on GitHub (Apr 28, 2023):

Yup, but Gitea can't find the user's name for UID=10xx (non-1000), then bug occurs ..... actually it's also related another problem (#24389), the IsRunUserMatchCurrentUser logic is fragile.

@wxiaoguang commented on GitHub (Apr 28, 2023): Yup, but Gitea can't find the user's name for UID=10xx (non-1000), then bug occurs ..... actually it's also related another problem (#24389), the IsRunUserMatchCurrentUser logic is fragile.

GiteaMirror commented

@bd4 commented on GitHub (Apr 28, 2023):

Ok so the issue happens when using a host UID that does not exist in the docker image? And it stopped working in 1.19 because the code is trying to determine the user name, and before it wasn't?

@bd4 commented on GitHub (Apr 28, 2023): Ok so the issue happens when using a host UID that does not exist in the docker image? And it stopped working in 1.19 because the code is trying to determine the user name, and before it wasn't?

GiteaMirror commented

@wxiaoguang commented on GitHub (Apr 28, 2023):

Yes (I guess so)

@wxiaoguang commented on GitHub (Apr 28, 2023): Yes (I guess so)

GiteaMirror commented

@wxiaoguang commented on GitHub (Apr 29, 2023):

Sorry for bothering, one more question, do you have "DISABLE_SSH=false" and "START_SSH_SERVER=true" in your app.ini , aka, have you enabled Gitea's builtin SSH server?

@wxiaoguang commented on GitHub (Apr 29, 2023): Sorry for bothering, one more question, do you have "DISABLE_SSH=false" and "START_SSH_SERVER=true" in your app.ini , aka, have you enabled Gitea's builtin SSH server?

GiteaMirror commented

@bd4 commented on GitHub (Apr 29, 2023):

Sorry for bothering, one more question, do you have "DISABLE_SSH=false" and "START_SSH_SERVER=true" in your app.ini , aka, have you enabled Gitea's builtin SSH server?

Yes I have both those set in the config, so is using the builtin server.

@bd4 commented on GitHub (Apr 29, 2023): > Sorry for bothering, one more question, do you have "DISABLE_SSH=false" and "START_SSH_SERVER=true" in your app.ini , aka, have you enabled Gitea's builtin SSH server? Yes I have both those set in the config, so is using the builtin server.

GiteaMirror commented

@wxiaoguang commented on GitHub (Apr 30, 2023):

If you could build your own docker image, welcome to try this fix https://github.com/go-gitea/gitea/pull/24435. Or I think it could be merged in one or two days soon then you can try the 1.19 nightly image.

@wxiaoguang commented on GitHub (Apr 30, 2023): If you could build your own docker image, welcome to try this fix https://github.com/go-gitea/gitea/pull/24435. Or I think it could be merged in one or two days soon then you can try the 1.19 nightly image.

GiteaMirror commented

@wxiaoguang commented on GitHub (Apr 30, 2023):

https://hub.docker.com/r/gitea/gitea/tags?page=1&name=1.19-dev these images contain https://github.com/go-gitea/gitea/pull/24435

@wxiaoguang commented on GitHub (Apr 30, 2023): https://hub.docker.com/r/gitea/gitea/tags?page=1&name=1.19-dev these images contain https://github.com/go-gitea/gitea/pull/24435

GiteaMirror commented

@bota87 commented on GitHub (May 2, 2023):

1.19-dev-rootless isn't updated and didn't worked for me, 1.19-dev-linux-amd64-rootless works

@bota87 commented on GitHub (May 2, 2023): 1.19-dev-rootless isn't updated and didn't worked for me, 1.19-dev-linux-amd64-rootless works

GiteaMirror commented

2025-11-02 09:08:35 -06:00

@wxiaoguang commented on GitHub (May 2, 2023):

1.19-dev-rootless isn't updated and didn't worked for me, 1.19-dev-linux-amd64-rootless works

@techknowlogick

@wxiaoguang commented on GitHub (May 2, 2023): > 1.19-dev-rootless isn't updated and didn't worked for me, 1.19-dev-linux-amd64-rootless works @techknowlogick

GiteaMirror commented

@lonix1 commented on GitHub (Jun 23, 2023):

@wxiaoguang as discussed on discord, here is my analysis of the problem.

Assumptions:

I followed the "SSHing Shim (with authorized_keys)" docs. Is it possible I made a mistake? Yes, but I spent a lot of time on this so I doubt it.
gith is user on host and gitc is user in container

Problem 1: uid:gid for gith and gitc differ

gith will have some random values when created
gitc is hardcoded to 1000:1000, and that's impossible to change via environment: in the docker-compose.yml (as suggested by the docs, incorrectly)
So the uid:gid of gith and gitc are diffferent, and that's the core problem
Note that it's possible to specify uid:gid for gith upon creation, but on debian/ubuntu distros (maybe others too?) the primary user already has 1000:1000 by default, so there's no way to match these values

Problem 2: ssh rules

From the man page for sshd:

~/.ssh/authorized_keys
...The content of the file is not highly sensitive, but the recommended permissions are read/write for the user, and not accessible by others. If this file, the ~/.ssh directory, or the user's home directory are writable by other users, then the file could be modified or replaced by unauthorized users. In this case, sshd will not allow it to be used...

~/.ssh/authorized_keys
...similar

~/.ssh/known_hosts
...similar

So these paths must have proper permissions on the host:

path	owner:group	permissions
`/home/gith/`	gith:gith	755
`/home/gith/.ssh/`	gith:gith	700
`/home/gith/.ssh/authorized_keys`	gith:gith	600

Results

On the host:

if those paths are correct, then it will be possible to ssh to the host (but the tunnel to the container will fail)
this can be tested from your local machine: $ ssh -T gith@server; that will connect but the tunnel will fail, which means ssh on the host is correct

In the container:

if those paths are correct on the host, then they will be incorrect in the container - because the uid:gid of gith differ to those of gitc
that is why when performing any git clone/whatever ... action, it always fails with

Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
if one changes them to the gitc user's 1000:1000 to fix that problem, then of course you won't be able to ssh to the host (because that is incorrect for the host's gith user).

Recommendations

I spent a lot of time on this because it was documented and I assumed if there's a problem it's just me; but I have my doubts that this works as documented. I also have a suspicion that the instructions did work, but not for the latest version (was reported in some other issue). I think a simpler alternative should be proposed that does not involve ssh tunnelling.
Alternative 1: in container, change gitc user's hardcoded uid:gid from 1000:1000 to something else (e.g. 1010:1010). That also isn't perfect but will eliminate 99% of problems out here, because as explained above, 1000:1000 is already used for the default user on the host. If you use 1010:1010, then when creating gith we could specify 1010:1010 for that user and thus match the values, and so eliminate the problem.
Alternative 2: create a custom Dockerfile and set uid:gid manually for gitc user.
Alternative 3: do not use ssh tunnelling, rather have a separate port on the host, e.g. port 22 for host's normal sshd process, and port 2222 for container's sshd process (and compose file would have ports: 2222:22).
Alternative 4: add another IP address to host, and bind container's port 22 to that, so both sshd processes work on port 22
Alternative 5: I have a strong suspicion that all the "ssh passthrough" stuff can be simplified using a regular ssh tunnel (ssh -L ...) but I don't have time to test that. It could be MUCH simpler than we think, but someone else must investigate that. There are many examples on StackOverflow, so if someone has time, start there.

Whatever you guys decide, one thing that's important to do right way, is put a "warning" box in that section on the docs stating that this is a very advanced scenario and that most users would probably not be able to get it to work. (I think it's impossible, but I may be wrong.) Don't let people waste time on this.

@lonix1 commented on GitHub (Jun 23, 2023): @wxiaoguang as discussed on discord, here is my analysis of the problem. ## Assumptions: - I followed the "SSHing Shim (with authorized_keys)" [docs](https://docs.gitea.com/installation/install-with-docker#sshing-shim-with-authorized_keys). Is it possible I made a mistake? Yes, but I spent a lot of time on this so I doubt it. - `gith` is user on host and `gitc` is user in container ## Problem 1: uid:gid for gith and gitc differ - `gith` will have some random values when created - `gitc` is hardcoded to `1000:1000`, and that's impossible to change via `environment:` in the `docker-compose.yml` (as suggested by the docs, incorrectly) - So the uid:gid of `gith` and `gitc` are diffferent, and that's the core problem - Note that it's possible to specify uid:gid for `gith` upon creation, but on debian/ubuntu distros (maybe others too?) the primary user already has `1000:1000` by default, so there's no way to match these values ## Problem 2: ssh rules From the [man page](https://man7.org/linux/man-pages/man8/sshd.8.html#FILES) for `sshd`: > **~/.ssh/authorized_keys** >...The content of the file is not highly sensitive, but the recommended permissions are read/write for the user, and not accessible by others. If this file, the ~/.ssh directory, or the user's home directory are writable by other users, then the file could be modified or replaced by unauthorized users. In this case, sshd will not allow it to be used... > **~/.ssh/authorized_keys** > ...similar > **~/.ssh/known_hosts** > ...similar So these paths must have proper permissions on the host: |path|owner:group|permissions| |-|-|-| |`/home/gith/`|gith:gith|755| |`/home/gith/.ssh/`|gith:gith|700| |`/home/gith/.ssh/authorized_keys`|gith:gith|600| ## Results On the host: - if those paths are correct, then it will be possible to ssh to the host (but the tunnel to the container will fail) - this can be tested from your local machine: `$ ssh -T gith@server`; that will connect but the tunnel will fail, which means ssh on the host is correct In the container: - if those paths are correct on the host, then they will be incorrect in the container - because the uid:gid of `gith` differ to those of `gitc` - that is why when performing any `git clone/whatever ...` action, it always fails with > Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. - if one changes them to the `gitc` user's 1000:1000 to fix that problem, then of course you won't be able to ssh to the host (because that is incorrect for the host's `gith` user). ## Recommendations - I spent a lot of time on this because it was documented and I assumed if there's a problem it's just me; but I have my doubts that this works as documented. I also have a suspicion that the instructions did work, but not for the latest version (was reported in some other issue). I think a simpler alternative should be proposed that does not involve ssh tunnelling. - Alternative 1: in container, change `gitc` user's [hardcoded uid:gid](https://github.com/go-gitea/gitea/blob/c0fc53e22602fc4b3c8d890d235d0911bef074fe/Dockerfile#L45-L51) from 1000:1000 to something else (e.g. 1010:1010). That also isn't perfect but will eliminate 99% of problems out here, because as explained above, 1000:1000 is already used for the default user on the host. If you use 1010:1010, then when creating `gith` we could specify 1010:1010 for that user and thus match the values, and so eliminate the problem. - Alternative 2: create a custom `Dockerfile` and set uid:gid manually for `gitc` user. - Alternative 3: do not use ssh tunnelling, rather have a separate port on the host, e.g. port 22 for host's normal sshd process, and port 2222 for container's sshd process (and compose file would have `ports: 2222:22`). - Alternative 4: add another IP address to host, and bind container's port 22 to that, so both sshd processes work on port 22 - Alternative 5: I have a strong suspicion that all the "ssh passthrough" stuff can be simplified using a regular ssh tunnel (`ssh -L ...`) but I don't have time to test that. It could be MUCH simpler than we think, but someone else must investigate that. There are **many** examples on StackOverflow, so if someone has time, start there. Whatever you guys decide, one thing that's important to do right way, is put a "warning" box in that section on the docs stating that this is a very advanced scenario and that most users would probably not be able to get it to work. (I think it's impossible, but I may be wrong.) Don't let people waste time on this.

GiteaMirror commented

2025-11-02 09:08:35 -06:00

@lonix1 commented on GitHub (Jun 23, 2023):

Another thing, I set ENABLE_SSH_LOG = true to be able to diagnose ssh problems in the container's sshd, but no sshd errors were written to the log file (which is working for other cases). I wonder if that setting is working?

@lonix1 commented on GitHub (Jun 23, 2023): Another thing, I set `ENABLE_SSH_LOG = true` to be able to diagnose ssh problems in the container's sshd, but no sshd errors were written to the log file (which is working for other cases). I wonder if that setting is working?

GiteaMirror commented

2025-11-02 09:08:36 -06:00

@wxiaoguang commented on GitHub (Jun 23, 2023):

Another thing, I set ENABLE_SSH_LOG = true to be able to diagnose ssh problems in the container's sshd, but no sshd errors were written to the log file (which is working for other cases). I wonder if that setting is working?

That's another longstanding problem: difficult to debug SSH part with logs. "ENABLE_SSH_LOG" only sends a few (usually the last error log) to the web server, then the error log will be shown in Gitea server's log with prefix "ssh: ". It's not useful in most cases.

@wxiaoguang commented on GitHub (Jun 23, 2023): > Another thing, I set `ENABLE_SSH_LOG = true` to be able to diagnose ssh problems in the container's sshd, but no sshd errors were written to the log file (which is working for other cases). I wonder if that setting is working? That's another longstanding problem: difficult to debug SSH part with logs. "ENABLE_SSH_LOG" only sends a few (usually **the last error log**) to the web server, then the error log will be shown in Gitea server's log with prefix "ssh: ". It's not useful in most cases.

GiteaMirror commented

2025-11-02 09:08:36 -06:00

@lonix1 commented on GitHub (Jun 23, 2023):

TL;DR

The problem is here - the UID and GID are hardcoded in the `Dockerfile`.

They should be based on environment variables; so user can specify them in Dockerfile (ENV), docker-compose.yml (environment:) or via docker cli (docker run -e).
There should be default values, e.g. 1001 (not to clash with default 1000 uid/gid on most distros)

Most of this issue (and MANY others) are based on this one problem. Please consider it as a major issue - because of this, we cannot use passthrough.

@lonix1 commented on GitHub (Jun 23, 2023): TL;DR ### The problem [is here](https://github.com/go-gitea/gitea/blob/c0fc53e22602fc4b3c8d890d235d0911bef074fe/Dockerfile#L45-L51) - the UID and GID are hardcoded in the `Dockerfile`. - They should be based on environment variables; so user can specify them in `Dockerfile` (`ENV`), `docker-compose.yml` (`environment:`) or via docker cli (`docker run -e`). - There should be default values, e.g. 1001 (not to clash with default 1000 uid/gid on most distros) Most of this issue (and MANY others) are based on this one problem. Please consider it as a major issue - because of this, we cannot use passthrough.

GiteaMirror commented

@krokosik commented on GitHub (Sep 21, 2023):

@lonix1 great work, I also spent lots of time with the documentation, thinking I'm plain stupid. We are currently using Alternative 3 from your recommendation, but would love a simpler solution

@krokosik commented on GitHub (Sep 21, 2023): @lonix1 great work, I also spent lots of time with the documentation, thinking I'm plain stupid. We are currently using Alternative 3 from your recommendation, but would love a simpler solution

GiteaMirror commented

@tomholford commented on GitHub (Oct 3, 2023):

@lonix1 thanks for sharing your findings. I also spent many hours trying to debug this issue. A configurable UID / GID via docker env vars would be really useful

@tomholford commented on GitHub (Oct 3, 2023): @lonix1 thanks for sharing your findings. I also spent many hours trying to debug this issue. A configurable UID / GID via docker env vars would be really useful

GiteaMirror commented

@tomholford commented on GitHub (Oct 3, 2023):

@wxiaoguang @lonix1 @wkrasnicki I started on a potential solution in #27405. Could use your feedback to see if this direction makes sense, and what other changes need to be made to resolve this issue.

@tomholford commented on GitHub (Oct 3, 2023): @wxiaoguang @lonix1 @wkrasnicki I started on a potential solution in #27405. Could use your feedback to see if this direction makes sense, and what other changes need to be made to resolve this issue.

GiteaMirror commented

@wxiaoguang commented on GitHub (Oct 3, 2023):

Thank you for inviting me to review. At the moment I don't have full understanding for this problem. So feel free to propose a complete solution (with explanation and documents), maybe other users&maintainers could help and suggest.

@wxiaoguang commented on GitHub (Oct 3, 2023): Thank you for inviting me to review. At the moment I don't have full understanding for this problem. So feel free to propose a complete solution (with explanation and documents), maybe other users&maintainers could help and suggest.

GiteaMirror commented