Update buf plugin with latest v2 beta

[zbstof]

Summary

Buf uses 1.2.5 of betterproto package, which cannot process optional keywords

What is the feature request for?

The core library

The Problem

In our company we have a standard to use buf to generate bindings to proto files.
And we wanted to use betterproto plugin to generate idiomatic python code.
Unfortunately, the only version of the plugin (v1.2.5) doesn't support some "newer" protobuf features, like "optional" keyword.
This was fixed in v.2.0.0b4, but this release is not available in buf.

The Ideal Solution

Can I ask to release latest beta (v.2.0.0b7 as the time of writing) as another version of community plugin?
https://github.com/danielgtaylor/python-betterproto/milestone/4 does not seem to be forthcoming.
Thanks

The Current Solution

Rely on official buf python plugin
https://buf.build/protocolbuffers/python?version=v28.0

[JitPackJoyride]

Hey, to get around this, I did an extra step on top of buf.gen.yaml with a poetry dependency that looked like this:

betterproto = {extras = ["compiler"], version = "^2.0.0b7", allow-prereleases = true}

You're likely also going to need grpcio and grpcio-tools installed.

Then, in my custom generate script, I did this:

  # Find all .proto files
  proto_files=$(find /schemas/proto -name "*.proto")

  # Special case for Python to use betterproto so that we can use Pydantic Dataclasses
  python3 -m grpc_tools.protoc \
      -I /schemas/proto \
      --python_betterproto_out=/schemas/output/python/proto/ \
      --python_betterproto_opt=pydantic_dataclasses,typing.310 \
      $proto_files

Hope that helps!

[zbstof]

Thank you @JitPackJoyride.
I do have similar workaround at hand. But if we are willing to call raw protoc, I don't see why we need buf at all

[alikleit]

any take on this? Will Buf include pre-released commits? or make any exception on libraries like betterproto for beta releases?

[jbouricius]

👍 here. I would also like to use betterproto with buf but the 4 year old version available doesn't work for our protos.

[mohd-saaduddin-ansari]

same issue here - the current version on buf isn't working anymore - any eta on when we will be releasing the latest version in buf plugins ?

[vegarab]

Same issue here. Would love to use the 2.0 pre-release with buf

[mezuzza]

You can work around this by vendoring betterproto yourself to your path. Buf is still handling almost everything outside of supplying the plugin.

# pyproject.toml
dependencies = [
    "betterproto[compiler]>=2.0.0b6",
]

# buf.gen.yaml
version: v2
clean: true
plugins:
  - local: protoc-gen-python_betterproto
    out: src/proto
inputs:
  - directory: ../protos

All you have to do is

1. Create a venv
2. Activate the venv
3. Install dependencies
4. Run buf generate
5. Profit

[kad-stoke]

If you are using the uv toolchain, you can use uvx to run the latest version without needing to alter other files. These updates in buf.gen.yaml work well for me:

version: v2
clean: true
plugins:
  - local:
      - "uvx"
      - "--from"
      - "betterproto[compiler]>=2.0.0b7"
      - "protoc-gen-python_betterproto"
    out: "src/protos"
    opt: "typing.310"

inputs:
  - directory: "protos"