Proto3 field presence #281
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kalzoo
force-pushed
the
proto3-field-presence
branch
from
642e745 to
04e1133
Compare
kalzoo
force-pushed
the
proto3-field-presence
branch
from
d5e8aaa to
4455175
Compare
Closed
|
@kalzoo What's the status on this? My team would benefit alot from having this in place. |
Contributor
|
I am using this PR and have found some issues with it. I have written a PR to fix those here: #292 |
= Description
The serialization of a oneof message that contains a message with fields
with explicit presence was buggy.
For example:
```
message A {
oneof kind {
B b = 1;
C c = 2;
}
}
message B {}
message C {
optional bool z = 1;
}
```
Serializing `A(b=B())` would lead to this payload:
```
0A # tag1, length delimited
00 # length: 0
12 # tag2, length delimited
00 # length: 0
```
Which when deserialized, leads to the message `A(c=C())`.
= Explanation
The issue lies in the post_init method. All fields are introspected, and
if different from PLACEHOLDER, the message is marked as having been
"serialized_on_wire".
Then, when serializing `A(b=B())`, we go through each field of the
oneof:
- field 'b': this is the selected field from the group, so it is
serialized
- field 'c': marked as 'serialized_on_wire', so it is added as well.
= Fix
The issue is that support for explicit presence changed the default
value from PLACEHOLDER to None. This breaks the post_init method in that
case, which is relatively easy to fix: if a field is optional, and set
to None, this is considered as the default value (which it is).
This fix however has a side-effect: the group_current for this field (the
oneof trick for explicit presence) is no longer set. This changes the
behavior when serializing the message in JSON: as the value is the
default one (None), and the group is not set (which would force the
serialization of the field), so None fields are no longer serialized in
JSON. This break one test, and will be fixed in the next commit.
* fix: do not serialize None fields in JSON format
This is linked to the fix from the previous commit: after it, scalar
None fields were not included in the JSON format, but some were still
included.
This is all cleaned up: None fields are not added in JSON by default,
as they indicate the default value of fields with explicit presence.
However, if `include_default_values is set, they are included.
Collaborator
Author
|
@alfredgunnar status is that the tests were caught up in an issue with |
|
Hey @kalzoo are the tests only failing on the pipeline? I ran them successfully locally. |
Collaborator
Author
|
This is fixed - CI failure was caused by a mistake in merge from Since I made the final changes, I'd like to ask @Gobot1234 (and others) for review. |
Collaborator
|
Everything here looks good to me! |
Gobot1234
approved these changes
Dec 29, 2021
Gobot1234
reviewed
Dec 29, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This finishes the work done on #268 by @roblabla . See that PR for more information