Add UUID validator #584

martinabeleda · 2023-05-05T07:55:39Z

Closes #554

codspeed-hq · 2023-05-05T08:00:24Z

CodSpeed Performance Report

Merging #584 martinabeleda:uuid-validator (3562f0c) will not alter performances.

Summary

🔥 0 improvements
❌ 0 regressions
✅ 114 untouched benchmarks

🆕 3 new benchmarks
⁉️ 0 dropped benchmarks

Benchmarks breakdown

	Benchmark	`main`	`martinabeleda:uuid-validator`	Change
🆕	`test_core_str`	N/A	19.7 µs	N/A
🆕	`test_core_root_model`	N/A	113.6 µs	N/A
🆕	`test_model_core_json`	N/A	48.5 µs	N/A

samuelcolvin · 2023-05-05T10:33:22Z

Before we get too far on this, it would be good to see the performance difference between this and the python implementation we currently have in pydantic.

codecov-commenter · 2023-05-05T23:41:07Z

Codecov Report

Merging #584 (3562f0c) into main (4767abf) will decrease coverage by 0.42%.
The diff coverage is 61.97%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #584      +/-   ##
==========================================
- Coverage   94.12%   93.71%   -0.42%     
==========================================
  Files          96       99       +3     
  Lines       13021    13292     +271     
  Branches       24       24              
==========================================
+ Hits        12256    12456     +200     
- Misses        759      830      +71     
  Partials        6        6

Impacted Files	Coverage Δ
pydantic_core/__init__.py	`81.81% <ø> (ø)`
src/input/input_abstract.rs	`85.29% <0.00%> (-2.59%)`	⬇️
src/serializers/shared.rs	`89.65% <ø> (ø)`
src/serializers/type_serializers/uuid.rs	`15.55% <15.55%> (ø)`
src/validators/uuid.rs	`57.14% <57.14%> (ø)`
src/errors/types.rs	`84.29% <84.61%> (+0.01%)`	⬆️
src/uuid.rs	`94.54% <94.54%> (ø)`
pydantic_core/core_schema.py	`96.99% <100.00%> (+0.03%)`	⬆️
src/input/input_python.rs	`98.01% <100.00%> (-0.73%)`	⬇️
src/lib.rs	`100.00% <100.00%> (ø)`
... and 1 more

... and 5 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4767abf...3562f0c. Read the comment docs.

martinabeleda · 2023-05-06T00:56:13Z

I implemented some benchmarks in dba435a. Here's the results:

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃  Group                           ┃  Test Name                    ┃             Best (ns/iter)  ┃         Relative  ┃              Stddev (ns/iter)  ┃          Iterations  ┃  Note      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│  uuid model - JSON               │  model_core_json              │                      490.0  │                   │                          4.98  │           1,000,000  │            │
│                                  │  model_pyd_json               │                    2_676.3  │            x5.46  │                         62.75  │           1,000,000  │            │
├──────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼───────────────────┼────────────────────────────────┼──────────────────────┼────────────┤
│  uuid str                        │  core_str                     │                      131.0  │                   │                          2.65  │           1,000,000  │            │
└──────────────────────────────────┴───────────────────────────────┴─────────────────────────────┴───────────────────┴────────────────────────────────┴──────────────────────┴────────────┘

~~I also assume we'd want to do something similar to datetime where we can parse strings and python objects?~~

edit: In saying that, I can't see a type for uuid.UUID in pyo3.types

samuelcolvin

in principle looks good.

But we need a significant change to use the standard lib UUID type, not a new rust type.

samuelcolvin · 2023-05-17T14:04:35Z

pydantic_core/_pydantic_core.pyi

+    def variant(self) -> str: ...
+    @property
+    def version(self) -> str: ...
+    def __init__(self, uuid: str) -> None: ...


__init__ should come first.

samuelcolvin · 2023-05-17T14:05:56Z

pydantic_core/core_schema.py

@@ -3503,6 +3503,38 @@ def multi_host_url_schema(
    )


+class UuidSchema(TypedDict, total=False):
+    type: Required[Literal['uuid']]
+    version: int


I'm guessing this should be something like Literal[3, 4, 5] - not sure exactly what's allowed, I guess that's kind of the point of using a Literal.

I think we should at least have Literal[1, 3, 4, 5] | None so that it's possible to not enforce checking of the version. There are cases, e.g. the Nil UUID or Max UUID, where it may be desirable to not enforce a version.

Note that Python doesn't support generating UUID v2 and nor does the rust uuid crate, but perhaps that should still be included as a valid instance could still be instantiated for a v2 UUID. In addition there is an update in the works to introduce v6, v7 and v8. Not sure whether it's worth adding a comment that this might need looking into in the future? Interestingly the uuid crate does already have preliminary support for generating them, even if Python does not.

samuelcolvin · 2023-05-17T14:06:16Z

pydantic_core/core_schema.py

+    ```
+
+    Args:
+        version: The UUID version number as per RFC 4122


say what it defaults to if ommitted.

samuelcolvin · 2023-05-17T14:06:16Z

pydantic_core/core_schema.py

@@ -3503,6 +3503,38 @@ def multi_host_url_schema(
    )


+class UuidSchema(TypedDict, total=False):
+    type: Required[Literal['uuid']]
+    version: int


I'm guessing this should be something like Literal[3, 4, 5] - not sure exactly what's allowed, I guess that's kind of the point of using a Literal.

samuelcolvin · 2023-05-17T14:07:48Z

src/errors/types.rs

@@ -533,6 +547,9 @@ impl ErrorType {
            Self::UrlSyntaxViolation {..} => "Input violated strict URL syntax rules, {error}",
            Self::UrlTooLong {..} => "URL should have at most {max_length} characters",
            Self::UrlScheme {..} => "URL scheme should be {expected_schemes}",
+            Self::UuidType => "Input should be a string",


Suggested change

Self::UuidType => "Input should be a string",

Self::UuidType => "UUID Input should be a string or UUID object",

or something, you'll also want a custom error message for JSON, see the function below.

samuelcolvin · 2023-05-17T14:09:31Z

src/uuid.rs

+
+static SCHEMA_DEFINITION_UUID: GILOnceCell<SchemaValidator> = GILOnceCell::new();
+
+#[pyclass(name = "Uuid", module = "pydantic_core._pydantic_core")]


I don't think we want a custom type here - the standard library has an existing UUID implementation that we should leverage.

samuelcolvin · 2023-05-17T14:11:15Z

src/serializers/type_serializers/uuid.rs

+    }
+}
+
+impl TypeSerializer for UuidSerializer {


I'm not sure you need this, the existing ToStringSerializer should probably surfice.

samuelcolvin · 2023-05-17T14:11:58Z

src/serializers/type_serializers/uuid.rs

+    }
+}
+
+impl TypeSerializer for UuidSerializer {


I'm not sure you need this, the existing ToStringSerializer should probably surfice.

samuelcolvin · 2023-05-17T14:11:59Z

src/serializers/type_serializers/uuid.rs

+    }
+}
+
+impl TypeSerializer for UuidSerializer {


I'm not sure you need this, the existing ToStringSerializer should probably suffice.

samuelcolvin · 2023-06-21T16:24:31Z

Closing as no response.

samuelcolvin · 2023-07-13T17:44:29Z

To get fast UUID validation, we should do the following:

perform the UUID validation in Rust
then construct the UUID instance directly by setting the attributes, like here

davidhewitt · 2023-07-17T14:37:48Z

I think this has now been superseded by #772.

martinabeleda added 3 commits May 4, 2023 22:03

working on uuid type

d8b9b3e

add uuid validator

93103c1

clean up version check

e8b7ce1

martinabeleda added 2 commits May 5, 2023 08:51

fix up parsing before work

fedac0f

fix version, variant and urn

0b06fb7

add uuid benchmarks

dba435a

add python tests

9f7e356

martinabeleda marked this pull request as ready for review May 6, 2023 18:21

add uuid serializer

3562f0c

samuelcolvin requested changes May 17, 2023

View reviewed changes

pydantic-hooky bot added the awaiting author revision label May 17, 2023

pydantic-hooky bot assigned martinabeleda May 17, 2023

samuelcolvin mentioned this pull request Jul 13, 2023

remove calls to __instancecheck__ on ModelValidator #497

Open

davidhewitt closed this Jul 17, 2023

	Self::UuidType => "Input should be a string",
	Self::UuidType => "UUID Input should be a string or UUID object",


		static SCHEMA_DEFINITION_UUID: GILOnceCell<SchemaValidator> = GILOnceCell::new();

		#[pyclass(name = "Uuid", module = "pydantic_core._pydantic_core")]

Add UUID validator #584

Add UUID validator #584

Uh oh!

Conversation

martinabeleda commented May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq bot commented May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Benchmarks breakdown

Uh oh!

samuelcolvin commented May 5, 2023

Uh oh!

codecov-commenter commented May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

martinabeleda commented May 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samuelcolvin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

samuelcolvin commented Jun 21, 2023

Uh oh!

samuelcolvin commented Jul 13, 2023

Uh oh!

davidhewitt commented Jul 17, 2023

Uh oh!

Uh oh!

martinabeleda commented May 5, 2023 •

edited

Loading

codspeed-hq bot commented May 5, 2023 •

edited

Loading

codecov-commenter commented May 5, 2023 •

edited

Loading

martinabeleda commented May 6, 2023 •

edited

Loading