Skip to content

BREAKING: New ILP/HTTP and Sender.from_conf API #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 89 commits into from
Mar 19, 2024
Merged

Conversation

amunra
Copy link
Collaborator

@amunra amunra commented Jan 3, 2024

Overview

  • Implement Sender.from_conf and Sender.from_env APIs which will simplify construction of the sender.
  • Make at= argument in .row() and .dataframe() methods mandatory.
  • Support the latest version of Pandas.
  • Numerous documentation improvements.

Tasks

  • wire up new c-questdb-client code.
  • get HTTP basics working.
  • Match up new config string logic: https://docs.google.com/document/d/1EQZjYrJO_Ll8RrSIZpbr15Ul3N-xAtTxvrunI-TpOR8/edit
  • Rework params to match conf args.
  • implement row-count and interval based auto-flushing.
  • make at=.. KW parameter required in row and dataframe API, new "ServerNow" for old at=None behaviour.
  • implement Sender::from_conf('...', **kwargs).
  • Implement with sender.transaction() to temporarily disable auto-flushing.
  • Implement sender.flush(buffer, transactional=True).
  • test auto_flush_interval
  • test auto_flush_rows
  • test auto_flush='off' and with sender: block: Final flush should still happen.
  • test sender.transaction().
  • set user agent in sender opts.
  • test and implement init_buf_size and max_buf_size params.
  • test every single param as python type.
  • test every single param as str equivalent (e.g. 'on'/'off' for bool)
  • Disable _FLUSH_FMT for HTTP.
  • review all ctor params have correct defaults.
  • update existing examples
  • write a new from_conf example and register it with the manifest file
  • test with dataframes and http, including transactions
  • update API docs
  • write HTTP intro docs
  • fix auto-flushing comments in examples
  • change auto-flush default for TCP to 600 rows.
  • test calling txn.commit() multiple times.
  • test calling sender.flush() from within a txn block.

Closes #51.
Closes #62.

@amunra
Copy link
Collaborator Author

amunra commented Jan 3, 2024

For the related Rust/C client changes, see questdb/c-questdb-client#50 and questdb/c-questdb-client#54

Copy link
Contributor

@nwoolmer nwoolmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM

Copy link
Contributor

@goodroot goodroot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented in a few areas.

Please @goodroot anywhere you'd like me to pickup next week.

Docs look OK to me on surface, prefer to unblock the API changes.

@bluestreak01 bluestreak01 merged commit d26da90 into main Mar 19, 2024
@bluestreak01 bluestreak01 deleted the ilp_http branch March 19, 2024 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[has workaround] Segfault for pandas versions >= 2.2.0 Refresh pandas logic with new Pandas 2.0
5 participants