nginx: [emerg] mkdir() "/var/tmp/nginx/client_body" failed (13: Permission denied)

[polarathene]

Just updated this image which was installed some time ago(2017-2018?), there seems to be an issue with nginx failing to run because it cannot create the client_body dir, /var/tmp/nginx does exist as an empty dir though.

I came across this issue which seems to be the same problem encountered in 2018. The linked issue comment points out that it's likely due to the kernel. The server is running 4.4 kernel(which has been updated earlier this year).

A comment later on in the issue then suggests pointing nginx tmp paths to a different location instead. Though I'm not sure where a list of all those are. It appears there is only the 5(all grouped together on that page, see --http-client-body-temp-path=path).

I also came across this response on the nginx forums which state it could be due to permissions issue from a parent directory. /var/tmp/nginx is nobody:nobody but parent /var/tmp is root:root, perhaps that's somehow the cause? I see that the switch to a nobody user/group was about 7 months ago.

[TrafeX]

Hi Brennan,
Strange issue, I also thought it could be related to running nginx under the nobody user. But I can't reproduce it.

If I use the latest image, POST a file that's bigger than 16K (and smaller than 1MB) to the example page it shows this in the log:

2019/08/12 12:13:57 [warn] 9#9: *1 a client request body is buffered to a temporary file /var/tmp/nginx/client_body/0000000001, client: 192.168.60.1, server: _, request: "POST / HTTP/1.1", host: "localhost"
192.168.60.1 - - [12/Aug/2019:12:13:57 +0000] "POST / HTTP/1.1" 200 218231 "-" "curl/7.64.0" "-" 0.009 0.012 . -

But the request is successful. That's expected behavior.

Can you paste the result of this command; docker exec -ti <name of container> ls -la /var/tmp/nginx. That shows me the permissions for that folder.

[polarathene]

My older version running PHP 7.1.9(Oct 2017)

docker run -it --rm trafex/alpine-nginx-php7 ls -la /var/tmp/nginx

total 8
drwx------    2 nginx    nginx         4096 Oct 13  2017 .
drwxr-xr-x    1 root     root          4096 Oct 13  2017 ..

With the 1.2.0 tag:

docker run -it --rm trafex/alpine-nginx-php7:1.2.0 ls -la /var/tmp/nginx
total 8
drwx------    1 nobody   nobody        4096 Aug  3 12:44 .
drwxrwxrwt    1 root     root          4096 Aug  3 12:44 ..

This is enough to reproduce the error message(no error on prior version):

docker run -it --rm trafex/alpine-nginx-php7:1.2.0 nginx -g 'daemon off;'
nginx: [emerg] mkdir() "/var/tmp/nginx/client_body" failed (13: Permission denied)

If that doesn't reproduce it for you, it might actually be the kernel version as pointed out. My server is running the LTS 4.4.104 kernel on openSUSE Leap 42.2. The linked issue stated they resolved it with kernel 4.13 upgrade, not sure if that's the min required if this issue is due to kernel somehow, The latest openSUSE 15.1 only provides the 4.12 kernel however.

[TrafeX]

Thanks for the input, that really helps!

I indeed can't reproduce it with;

docker run -it --rm trafex/alpine-nginx-php7:1.2.0 nginx -g 'daemon off;'                                                                 
...
Status: Downloaded newer image for trafex/alpine-nginx-php7:1.2.0
2019/08/14 14:57:48 [error] 6#6: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /fpm-ping HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "127.0.0.1:8080"

It just works. I'm on kernel 5.0.0-21-generic (Ubuntu 19.04).

I hope you're able to update to a newer kernel to see if that solves it. Otherwise they mention a workaround by moving all the temp folders nginx needs to /tmp linuxserver-archive/docker-letsencrypt#67 (comment)
I could add that to this image.

[polarathene]

I hope you're able to update to a newer kernel to see if that solves it.

The server is running production stuff and I'm not too confident with doing an upgrade to a new major fixed release atm, it's volunteer work for a community.

What I can do is try it out locally by installing an older kernel on my distro to see if it really is the kernel. I'll let you know how that goes once I get around to it.

Otherwise they mention a workaround by moving all the temp folders nginx needs to /tmp

Yeah, this might work. To keep it flexible perhaps keep those paths in a separate conf file and include it from the server conf file?

That would be useful with the other issue I raised on this project here. If you like, when I do the local kernel test, I can try this out and if it works for both issues well, then submit a PR of the modularized config?

[TrafeX]

I placed a comment on #17 explaining my view on it. You're always welcome to submit a PR of course!
Let me know what comes out of your tests! 👍

[TrafeX]

I'll close this issue. If you have any updates on the kernel tests, let me know!

[polarathene]

@TrafeX Hey there, sorry for disappearing!

I got around to testing with a 4.4 kernel on my local system and the error does occur there. So definitely seems like an issue with the older kernel that was fixed in newer ones.

To fix it requires setting all these paths to use /tmp, instead of whatever their defaults are:

client_body_temp_path /tmp/nginx 1 2;
proxy_temp_path /tmp/nginx-proxy;
fastcgi_temp_path /tmp/nginx-fastcgi;
uwsgi_temp_path /tmp/nginx-uwsgi;
scgi_temp_path /tmp/nginx-scgi;

I'll continue with how to progress with a PR via #17 .

[TrafeX]

Hi @polarathene,

Thanks for coming back to this, as mentioned in #17 (comment) I've changed the temp paths to /tmp.
This is also recommended by nginx in https://hub.docker.com/_/nginx