Skip to content

Move field_name from runtime to schema generation time #715

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 3, 2023

Conversation

adriangb
Copy link
Member

@adriangb adriangb commented Jun 29, 2023

Pydantic should have this information when it is building the schema so this removes a bit of branching that happens at runtime, including a runtime error if the schema is misconfigured and a field from the poor catch-all Extra struct.

If we take this one step further we could replace field_name with field_info which could be an arbitrary generic type, giving us more flexibility on the Pydantic side to e.g. make it a FieldInfo instead of a string which you then need to look up in the model class. This change in particular would allow us to pass FieldInfo into Annotated validators if we wanted to.

Selected Reviewer: @dmontagu

@codspeed-hq
Copy link

codspeed-hq bot commented Jun 29, 2023

CodSpeed Performance Report

Merging #715 will improve performances by 13.51%

Comparing test-field-info (13d2a7a) with main (d285975)

Summary

🔥 3 improvements
✅ 123 untouched benchmarks

Benchmarks breakdown

Benchmark main test-field-info Change
🔥 test_list_of_ints_core_json 6.3 ms 5.7 ms +10.12%
🔥 test_set_of_ints_core_json_duplicates 5.4 ms 4.8 ms +11.66%
🔥 test_field_function_validator 939.4 µs 827.6 µs +13.51%

@adriangb
Copy link
Member Author

Just an idea I got from discussion in #714 . We could move the whole FieldInfo here or at least the field_name. I don't see any reason why we can't supply that to the schema in pydantic, and it would save us the runtime work of copying field_name around in `Extra, some branching, etc.

@adriangb adriangb changed the title Test impact of moving field_info onto schema Move field_name from runtime to schema generation time Jul 1, 2023
@adriangb adriangb marked this pull request as ready for review July 1, 2023 18:31
@codecov
Copy link

codecov bot commented Jul 1, 2023

Codecov Report

Merging #715 (13d2a7a) into main (d285975) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #715      +/-   ##
==========================================
- Coverage   93.63%   93.63%   -0.01%     
==========================================
  Files          99       99              
  Lines       13913    13895      -18     
  Branches       25       25              
==========================================
- Hits        13028    13010      -18     
  Misses        879      879              
  Partials        6        6              
Impacted Files Coverage Δ
src/validators/dataclass.rs 98.31% <ø> (-0.02%) ⬇️
src/validators/generator.rs 91.39% <ø> (-0.07%) ⬇️
src/validators/mod.rs 97.77% <ø> (-0.03%) ⬇️
src/validators/model_fields.rs 98.30% <ø> (-0.01%) ⬇️
src/validators/typed_dict.rs 97.39% <ø> (ø)
python/pydantic_core/core_schema.py 96.64% <100.00%> (+<0.01%) ⬆️
src/validators/function.rs 91.36% <100.00%> (-0.05%) ⬇️
src/validators/model.rs 98.37% <100.00%> (-0.04%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d285975...13d2a7a. Read the comment docs.

@adriangb
Copy link
Member Author

adriangb commented Jul 1, 2023

please review

@adriangb
Copy link
Member Author

adriangb commented Jul 1, 2023

I measured #726 locally and am getting 22.09±0.5us on this branch vs. 26.17±0.362 on main.

@dmontagu dmontagu assigned samuelcolvin and unassigned dmontagu Jul 3, 2023
@adriangb
Copy link
Member Author

adriangb commented Jul 3, 2023

Benchmarks are now reflecting a 13.5% speedup :)

@dmontagu
Copy link
Collaborator

dmontagu commented Jul 3, 2023

@samuelcolvin Adrian and I discussed, and I think Adrian was going to try to implement the pydantic side before merging this to make sure there were no surprises, but we also want to make sure you don't have objections before merging.

(As far as I'm concerned this is good, at least assuming it can be made to work with pydantic, but figure there's a chance I'm not considering something important.)

@adriangb
Copy link
Member Author

adriangb commented Jul 3, 2023

I created pydantic/pydantic#6392, it looks pretty good I think. If that looks good to you David lets merge this and I can update that PR once we have a new core release

@adriangb adriangb merged commit 024efa2 into main Jul 3, 2023
@adriangb adriangb deleted the test-field-info branch July 3, 2023 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants