@@ -32,23 +32,29 @@ class ImplicitRenderer(torch.nn.Module):
32
32
33
33
VOLUMETRIC_FUNCTION
34
34
35
- The `forward` function of the renderer accepts as input the rendering cameras as well
36
- as the `volumetric_function` `Callable`, which defines a field of opacity
35
+ The `forward` function of the renderer accepts as input the rendering cameras
36
+ as well as the `volumetric_function` `Callable`, which defines a field of opacity
37
37
and feature vectors over the 3D domain of the scene.
38
38
39
39
A standard `volumetric_function` has the following signature:
40
40
```
41
- def volumetric_function(ray_bundle: RayBundle) -> Tuple[torch.Tensor, torch.Tensor]
41
+ def volumetric_function(
42
+ ray_bundle: RayBundle,
43
+ **kwargs,
44
+ ) -> Tuple[torch.Tensor, torch.Tensor]
42
45
```
43
46
With the following arguments:
44
47
`ray_bundle`: A RayBundle object containing the following variables:
45
- `rays_origins `: A tensor of shape `(minibatch, ..., 3)` denoting
48
+ `origins `: A tensor of shape `(minibatch, ..., 3)` denoting
46
49
the origins of the rendering rays.
47
- `rays_directions `: A tensor of shape `(minibatch, ..., 3)`
50
+ `directions `: A tensor of shape `(minibatch, ..., 3)`
48
51
containing the direction vectors of rendering rays.
49
- `rays_lengths `: A tensor of shape
52
+ `lengths `: A tensor of shape
50
53
`(minibatch, ..., num_points_per_ray)`containing the
51
54
lengths at which the ray points are sampled.
55
+ `xys`: A tensor of shape
56
+ `(minibatch, ..., 2)` containing the
57
+ xy locations of each ray's pixel in the screen space.
52
58
Calling `volumetric_function` then returns the following:
53
59
`rays_densities`: A tensor of shape
54
60
`(minibatch, ..., num_points_per_ray, opacity_dim)` containing
@@ -57,12 +63,20 @@ def volumetric_function(ray_bundle: RayBundle) -> Tuple[torch.Tensor, torch.Tens
57
63
`(minibatch, ..., num_points_per_ray, feature_dim)` containing
58
64
the an feature vector for each ray point.
59
65
66
+ Note that, in order to increase flexibility of the API, we allow multiple
67
+ other arguments to enter the volumentric function via additional
68
+ (optional) keyword arguments `**kwargs`.
69
+ A typical use-case is passing a `CamerasBase` object as an additional
70
+ keyword argument, which can allow the volumetric function to adjust its
71
+ outputs based on the directions of the projection rays.
72
+
60
73
Example:
61
74
A simple volumetric function of a 0-centered
62
75
RGB sphere with a unit diameter is defined as follows:
63
76
```
64
77
def volumetric_function(
65
78
ray_bundle: RayBundle,
79
+ **kwargs,
66
80
) -> Tuple[torch.Tensor, torch.Tensor]:
67
81
68
82
# first convert the ray origins, directions and lengths
0 commit comments