Skip to content

Troubleshooting error 502 "upstream sent too big header while reading response header from upstream" #676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions extra/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,58 @@ jms_serializer:
```

The JMS Serializer service is available as `jms_serializer`.

## "upstream sent too big header while reading response header from upstream" 502 Error

Some of your API calls fail with a 502 error and the logs for the api container shows the following error message `upstream sent too big header while reading response header from upstream`.

This can be due to the cache invalidation headers that are too big for NGINX. When you query the API, API Platform adds the ids of all returned entities and their dependencies in the headers like so : `Cache-Tags: /entity/1,/dependent_entity/1,/entity/2`. This can overflow the default header size (4k) when your API gets larger and more complex.

You can modify the `api/docker/nginx/conf.d/default.conf` file and set values to `fastcgi_buffer_size` and `fastcgi_buffers` that suit your needs, like so:

```
server {
root /srv/api/public;

location / {
# try to serve file directly, fallback to index.php
try_files $uri /index.php$is_args$args;
}

location ~ ^/index\.php(/|$) {
# Comment the next line and uncomment the next to enable dynamic resolution (incompatible with Kubernetes)
fastcgi_pass php:9000;
#resolver 127.0.0.11;
#set $upstream_host php;
#fastcgi_pass $upstream_host:9000;

# Bigger buffer size to handle cache invalidation headers expansion
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value used here for fastcgi_buffers seems very atypical, and is probably a complete overkill for most cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you suggest to use a lower value by default? The size of header can grow very fastly when the invalidation mechanism is enabled...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 32k is enough in most of case, 256k seems very very high.

Copy link
Contributor

@teohhanhui teohhanhui Dec 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like:

fastcgi_buffer_size 32k;
fastcgi_buffers 8 16k;

(Copied from somewhere around the Internet lol)

@dunglas:

The size of header can grow very fastly when the invalidation mechanism is enabled...

We often forget how big a kilobyte actually is when it comes to text. 😄

Copy link
Contributor

@teohhanhui teohhanhui Dec 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the response header grows past 32 KB because of the cache tags, I think it's a feature for it to not work, because you have a very serious problem. 😆

Assuming each cache tag is a pretty long IRI of 100 characters each, that'd take more than 300 IRIs... You're likely to run into api-platform/api-platform#721 🙈

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for this strange buffer values, it doesn't mean a lot to me, so I though the bigger the better :)

Should this troubleshooting put an accent on the shortName annotation to have shorter urls as a way to reduce the size of sent headers ?


fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
# When you are using symlinks to link the document root to the
# current version of your application, you should pass the real
# application path instead of the path to the symlink to PHP
# FPM.
# Otherwise, PHP's OPcache may not properly detect changes to
# your PHP files (see https://github.com/zendtech/ZendOptimizerPlus/issues/126
# for more information).
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
# Prevents URIs that include the front controller. This will 404:
# http://domain.tld/index.php/some-path
# Remove the internal directive to allow URIs like this
internal;
}

# return 404 for all other php files not matching the front controller
# this prevents access to other php files you don't want to be accessible.
location ~ \.php$ {
return 404;
}
}
```

You then need to rebuild your containers by running `docker-compose build`.