bpo-46874: Speed up sqlite3 user-defined aggregate 'step' method

[erlend-aasland]

Also improve error message if the step method is missing.

https://bugs.python.org/issue46874

[corona10]

lgtm

[erlend-aasland]

Thanks, Dong-hee!

[rinsuki]

I just discovered a new bug today, and this bug seems already fixed by this pull-request.

should I still report my bug to issue? or no need to report because already fixed on main/3.11 branch?

Summary about my discovered bug

when passing incomplete aggregate class, and first call (on one query) is with empty data, sqlite3 library raises wrong error.

Repro Code

import sqlite3
import traceback

db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE users (id INTEGER NOT NULL)")
db.execute("INSERT INTO users VALUES (0)")
db.execute("INSERT INTO users VALUES (1)")
db.execute("CREATE TABLE scores (player_id INTEGER NOT NULL, score INTEGER NOT NULL)")
db.execute("INSERT INTO scores (player_id, score) VALUES (1, 10)")

class Median:
    def __init__(self):
        self.data = []

    # ohh i just typo
    def steeep(self, x):
        self.data.append(x)

    def finalize(self):
        data = sorted(self.data)
        ld = len(data)
        if ld == 0:
            return int(0)
        elif ld % 2 == 0:
            return int((data[ld // 2 - 1] + data[ld // 2]) / 2)
        else:
            return int(data[ld // 2])

db.create_aggregate("median", 1, Median)
try:
    cur = db.execute("SELECT player_id, median(score) FROM scores GROUP BY player_id")
    print(cur.fetchall())
except:
    traceback.print_exc()

try:
    cur = db.execute("SELECT users.id, median(score) FROM users LEFT JOIN scores ON users.id = scores.player_id GROUP BY player_id ORDER BY users.id DESC")
    print(cur.fetchall())
except:
    traceback.print_exc()

this repro code prints two error, and they are should be same.

but in 3.11.0a5 or older, python prints different error (wrong behaviour).

Traceback (most recent call last):
  File "/app/sqlite-aggregate-test.py", line 31, in <module>
    cur = db.execute("SELECT player_id, median(score) FROM scores GROUP BY player_id")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Median' object has no attribute 'step'
Traceback (most recent call last):
  File "/app/sqlite-aggregate-test.py", line 37, in <module>
    cur = db.execute("SELECT users.id, median(score) FROM users LEFT JOIN scores ON users.id = scores.player_id GROUP BY player_id ORDER BY users.id DESC")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: user-defined aggregate's '__init__' method raised error

but 3.11.0a6 or newer, python prints same error (correct behaviour).

Traceback (most recent call last):
  File "/app/sqlite-aggregate-test.py", line 31, in <module>
    cur = db.execute("SELECT player_id, median(score) FROM scores GROUP BY player_id")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: user-defined aggregate's 'step' method not defined
Traceback (most recent call last):
  File "/app/sqlite-aggregate-test.py", line 37, in <module>
    cur = db.execute("SELECT users.id, median(score) FROM users LEFT JOIN scores ON users.id = scores.player_id GROUP BY player_id ORDER BY users.id DESC")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: user-defined aggregate's 'step' method not defined

[erlend-aasland]

Hi, @rinsuki! Please open a new ticket for this, and I'll see we can manage to backport this.

I also notice that the error message could be improved by including the object's name instead of just the non-descript "user-defined aggregate".

[rinsuki]

I opened it! #95462