Skip to content

feat: Use most compact JSON encoding #746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 29, 2020
Merged

Conversation

rhcarvalho
Copy link
Contributor

This shrinks event sizes a bit, even when gzip'ed.
The compact representation is documented in the json module.

Alternatively, we can also look into using a custom encoder (that could
also handle datetime objects, instead of the current manual
serialization of those).

In the absence of proper benchmark data, consider a random transaction
event t:

>>> len(json.dumps(t)), len(json.dumps(t, separators=(',', ':')))
(82174, 78516)

That is 95.5% of the original size.

With gzip compression:

>>> len(gzips(json.dumps(t))), len(gzips(json.dumps(t, separators=(',', ':'))))
(13093, 12988)

That is 99.2% of the original size.

@rhcarvalho rhcarvalho force-pushed the rhcarvalho/compact-json branch 2 times, most recently from 5dacf67 to a6cfc35 Compare June 29, 2020 16:17
This shrinks event sizes a bit, even when gzip'ed.
The compact representation is documented in the json module.

Alternatively, we can also look into using a custom encoder (that could
also handle datetime objects, instead of the current manual
serialization of those).

In the absence of proper benchmark data, consider a random transaction
event t:

>>> len(json.dumps(t)), len(json.dumps(t, separators=(',', ':')))
(82174, 78516)

That is 95.5% of the original size.

With gzip compression:

>>> len(gzips(json.dumps(t))), len(gzips(json.dumps(t, separators=(',', ':'))))
(13093, 12988)

That is 99.2% of the original size.
@rhcarvalho rhcarvalho force-pushed the rhcarvalho/compact-json branch from a6cfc35 to cb2a8b6 Compare June 29, 2020 17:18
@rhcarvalho rhcarvalho marked this pull request as ready for review June 29, 2020 18:05
@rhcarvalho rhcarvalho requested a review from untitaker June 29, 2020 18:05
@rhcarvalho rhcarvalho merged commit 7d482b5 into master Jun 29, 2020
@rhcarvalho rhcarvalho deleted the rhcarvalho/compact-json branch June 29, 2020 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants