Skip to content

[DRAFT] Make async queries possible #188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

fabianfett
Copy link
Collaborator

@fabianfett fabianfett commented Oct 19, 2021

This is a very early draft enabling async queries on a Postgres connection. It allows async/await in Postgres:

// swift-tools-version:5.5
import PackageDescription

let package = Package(
    name: "postgres-perf",
    platforms: [
        .macOS(.v12)
    ],
    dependencies: [
        .package(url: "https://github.com/fabianfett/postgres-nio.git", .branch("ff-async2")),
    ],
    targets: [
        .executableTarget(name: "PostgresExample", dependencies: [
            .product(name: "PostgresNIO", package: "postgres-nio"),
        ]),
    ]
)

And then use it like this:

import NIO
import Logging
import PostgresNIO
import struct Foundation.Date

@main
enum AsyncExample {
    static func main() async throws {
        let eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)
        let eventLoop = eventLoopGroup.next()
        defer { try! eventLoopGroup.syncShutdownGracefully() }
        
        var logger = Logger(label: "psql")
        logger.logLevel = .info
        
        let configuration = PSQLConnection.Configuration(
            host: "postgres", port: 5432,
            username: "postgres",
            database: "postgres",
            password: "postgres",
            tlsConfiguration: nil,
            coders: .foundation
        )
        let connection = try await PSQLConnection.connect(configuration: configuration, logger: logger, on: eventLoop)
        
        let start = Date()
        var i = 0
        var result = 0
        
        let stream = try await connection.query("SELECT * FROM large_test LIMIT 2000000", logger: logger)
        for try await (column1, column2, column3) in stream.decode(Int.self, Double.self, Double.self) {
            result &+= column1
            i &+= 1
        }
        
        try await connection.close()
    }
}

@fabianfett fabianfett force-pushed the ff-async2 branch 4 times, most recently from f159e0c to 0fa1c69 Compare October 22, 2021 12:11
@Jerry-Carter
Copy link
Contributor

Gonna start kicking the tires on your ff-async2 branch. Anything I should know?

@fabianfett fabianfett force-pushed the ff-async2 branch 2 times, most recently from 30336b6 to 320581b Compare November 19, 2021 08:56
@codecov-commenter
Copy link

codecov-commenter commented Nov 19, 2021

Codecov Report

Merging #188 (320581b) into main (f876692) will decrease coverage by 5.76%.
The diff coverage is 54.32%.

❗ Current head 320581b differs from pull request most recent head 02bb7ca. Consider uploading reports for the commit 02bb7ca to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main     #188      +/-   ##
==========================================
- Coverage   43.81%   38.05%   -5.77%     
==========================================
  Files         169      117      -52     
  Lines       12748     8449    -4299     
==========================================
- Hits         5586     3215    -2371     
+ Misses       7162     5234    -1928     
Flag Coverage Δ
unittests 38.05% <54.32%> (-5.77%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...esNIO/Connection/PostgresConnection+Database.swift 0.00% <0.00%> (ø)
Sources/PostgresNIO/Data/PostgresData+Array.swift 0.00% <0.00%> (ø)
...es/PostgresNIO/Message/PostgresMessage+Error.swift 0.00% <0.00%> (ø)
...Message/PostgresMessage+ParameterDescription.swift 0.00% <0.00%> (ø)
...esNIO/Message/PostgresMessage+RowDescription.swift 0.00% <0.00%> (ø)
...rces/PostgresNIO/New/Extensions/Logging+PSQL.swift 18.18% <ø> (ø)
...rces/PostgresNIO/New/Messages/Authentication.swift 63.36% <ø> (ø)
Sources/PostgresNIO/New/PSQLBackendMessage.swift 92.89% <ø> (ø)
Sources/PostgresNIO/New/PSQLChannelHandler.swift 47.96% <ø> (ø)
Sources/PostgresNIO/New/PSQLTask.swift 35.13% <ø> (ø)
... and 93 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 549b17f...02bb7ca. Read the comment docs.

better caching

performance improvements.

More stuff

Compiling tests

Compile errors for swift<5.5

compiles!

compile tests

Decoding optionals works better!

Clean up
fabianfett added a commit that referenced this pull request Nov 22, 2021
… level (#198)

This is a cherry pick of #188.

### Modifications

- `DataRow` and `RowDescription` have been moved out of the `PSQLBackendMessage` namespace. This allows us to mark them as `@inlinable` or `@usableFromInline` at a later point, without marking everything in `PSQLBackendMessage` as `@inlinable`
- `DataRow` does not use an internal array for its columns anymore. Instead all read operations are directly done on its ByteBuffer slice.
- `DataRow` implements the `Collection` protocol now.

### Result

One allocation fewer per queried row.
@fabianfett fabianfett closed this Feb 6, 2022
@fabianfett fabianfett deleted the ff-async2 branch August 8, 2023 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants