Cannot access request.data in log filter

[amikrop]

I want to pass some custom data to my log records, and I am doing so like this: https://stackoverflow.com/questions/10292082/django-logging-django-request-logger-and-extra-context/
However, when I construct a rest_framework.request.Request out of the given record.request object (Django HttpRequest passed as extra to the django.request logger), I can't access its data attribute if request.data has been explicitly accessed by the view that I am logging. If it hasn't been accessed, then no error occurs. When I say "can't" I mean I get a RawPostDataException: You cannot access body after reading from request's data stream error. Here is the code:

import logging

from rest_framework.request import Request
from rest_framework.settings import api_settings


class RequestInfoFilter(logging.Filter):

    def filter(self, record):
        parsers = [parser() for parser in api_settings.DEFAULT_PARSER_CLASSES]
        authenticators = [
            auth() for auth in api_settings.DEFAULT_AUTHENTICATION_CLASSES]
        request = Request(
            record.request, parsers=parsers, authenticators=authenticators)

        record.method = request.method
        record.query_params = request.query_params
        record.data = request.data  # RawPostDataException: You cannot access body after reading from request's data stream

        return True

Again, if the view that is being logged doesn't access request.data, no error occurs.
I have also asked a relevant question at SO: https://stackoverflow.com/questions/69437529/cannot-access-request-data-in-log-filter
Any help would be much appreciated.

[amikrop]

It looks like the original HttpRequest object somehow "loses" its _body attr, when going through django.core.handlers.base.get_response and/or django.utils.log.log_response, while still "remembers" that its data stream has been read/exhausted (and _read_started is True). Storing .body during custom middleware (on the HttpResponse object as some different attr) and restoring it during logging (record.request._body = record.request.original_body), makes .data work, but I am not sure this the best solution, plus it still leaves this as a mystery for me.

[tomchristie]

Correct - you can't do that.
It's a constraint around how the request body is handled.

[amikrop]

Correct - you can't do that. It's a constraint around how the request body is handled.

I see. Any suggested solutions then? I guess the middleware thing would be one, or doing the logging myself (in a mixin for all my views that does the logging in .finalize_response or something)? Anything better / more elegant?

[tomchristie]

I guess accessing request.body in middleware might resolve this, but the trade-off is that you'll be ingesting all the request body into memory on every request. Given that request bodies could be very large (eg. file uploads) I'd tend to prefer to avoid that. But if you're set on logging the request body on every incoming request, then you'll need to do something like that.

[tomchristie]

Normally REST framework will stream the request body when accessing it, but accessing request.body will preload it into memory, so you ought to be able to repeatedly access it after that.

[amikrop]

I see, thanks.