Description
Summary
The _serialized_on_wire attribute is 99% of the functionality I require.
Because it is a dunder attribute I can't rely on the attribute in future updates.
What is the feature request for?
The core library
The Problem
A method for determining if a betterproto field was set from parsed data in the protobuf bytes vs default value.
Fields in protobuf messages are optional by default.
It is important to detect if a field was set vs the default value.
For example, in Pydantic, optional values are None when unset.
This is done for the user by checking _serialized_on_wire.
I would like a supported function or non-dunder attribute exposing the _serialized_on_wire value.
Additionally, _serialized_on_wire is not enough when dealing with primitive types.
Currently, primitives types are check against the default value to determine their "_serialized_on_wire" status.
This causes primitive types that HAVE been set but match the default value to be indistinguishable from default set primitives.
Below is my testing code and function that works for my use case.
Other use cases will run into the primitive set issue.
# example.proto
syntax = "proto3";
option optimize_for = LITE_RUNTIME;
package example;
message NoPrimitives {
PrimitivesOne one = 1;
PrimitivesTwo two = 2;
}
message PrimitivesOne {
uint64 value1 = 1;
uint64 value2 = 2;
}
message PrimitivesTwo {
string value1 = 1;
string value2 = 2;
}# test.py
import dataclasses
import betterproto
from proto_example import NoPrimitives, PrimitivesOne
data = b"foo"
no_primitives = NoPrimitives().parse(data)
# appends unparsed data at the end
# Is there a supported way to remove unpased bytes?
# I would have expected unparsable bytes to be dropped when converting back.
print(f"original: {data}")
print(f"reconstructed: {bytes(no_primitives)}")
# will return true because bytes exist, even if no bytes are "valid/parsed" in the protobuf structure
print(f"serialized: {no_primitives._serialized_on_wire}")
# a workable solution in my use case, but will not handle other possible cases
def serialized_on_wire(better_proto: betterproto.Message):
"""return true if proto message was serialized on wire"""
for field in dataclasses.fields(better_proto):
meta = betterproto.FieldMetadata.get(field)
v = getattr(better_proto, field.name)
if isinstance(v, betterproto.Message):
if v._serialized_on_wire:
return True
elif v != better_proto._get_field_default(field, meta):
return True
return False
# How to detect serialized_on_wire for messages with primitive types only? (int, string, ect...)
# Assuming primitive types are changed from defaults, desired behavoir.
data = b"foo"
no_primitives = NoPrimitives().parse(data)
no_primitives.one = 1
print(f"serialized original: {no_primitives._serialized_on_wire}")
print(f"serialized new: {serialized_on_wire(no_primitives)}")
# If primitive types correlate with default values, undesired behavoir.
primitives_one = PrimitivesOne().parse(b"")
print(f"serialized before manual set: {primitives_one._serialized_on_wire}")
# setting values that correlate with default values
# will set serialized on wire to true
primitives_one.value1 = 0
primitives_one.value2 = 0
# can detect items have been set but not which ones, to_dict will still return empty
print(f"serialized after manual set: {primitives_one._serialized_on_wire}")
print(f"serialized new: {serialized_on_wire(primitives_one)}")
# Valid values have been set but because they correlate with default values nothing is retured.
# I would like a way to detect if a value if the default value AND parsed over serial or set through python.
# Even if bytes have been given to parse but invalid/unparsed
print(f"empty dictionary: {primitives_one.to_dict() == {}}") # returns trueThe Ideal Solution
Optional fields can be None when unset. (pydantics solution)
If default value returns are desired, there is already a flag for that.
For betterproto.Messages:
A method to return False if all primitive/non-primitive fields are unset.
For primitives:
Some other method for determining unset values beyond checking if the value is equal to the default value.
The Current Solution
99% solution for determining if an attribute if unset:
Check against the dunder attribute _serialized_on_wire directly and checking if primitives are equal to their default values.
Unable to determine if primitives are set but still equal to default value.