Skip to content

Result rows are dynamically shaped; causing massive performance degradations when used #3042

Closed
@koenfaro90

Description

@koenfaro90

I have been chasing performance within our applications this week; one of the things I encountered yesterday was that cloning a quite huge amount of data (~500mb), which are actually simple objects returned from node-postgres, blocked the thread for seconds; in itself logical because JSON.parse(JSON.stringify(row)) was being used. I migrated to using the spread operator instead and that should've improved things by a factor of 10-20; it however only worsened the situation.

Upon further examination and reproduction I found the root-cause of the problem; result rows created by node-postgres are dynamically created; one starts with an empty object; and for every result; every field is dynamically created on that object. Turns out, that Javascript engines don't like this because these objects have a non-simple shape. For more info: https://dev.to/fromaline/why-does-dynamically-adding-properties-is-slow-in-javascript-4hm8 / https://youtu.be/5nmpokoRaZI?t=1180

In order to resolve this I have created a patch that, if one passes "usePrebuiltEmptyResultObjects" to the query constructor; will create the row dynamically once; and then use clones of that to create new rows. This result in a 'clean' shape, and massive performance gains when using the rows returned from node-postgres down the line.

I will create a pull-request, including a test-class shortly, which demonstrates that this increases cloning these objects is at least 50x faster; I actually measure more in the range of 150x; but lets leave some room for wiggle.

I would actually prefer to opt to not have this be an option; but standard behaviour; but I would say that is up to the maintainer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions