Skip to content

Commit ec9a58b

Browse files
authored
Merge pull request #5 from gadomski/change-include-semantics
Change include/exclude recommendations
2 parents e96b06c + f0c062b commit ec9a58b

File tree

2 files changed

+166
-37
lines changed

2 files changed

+166
-37
lines changed

CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,16 @@
11
# Changelog
2+
23
All notable changes to this project will be documented in this file.
34

45
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
56
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
67

8+
## [Unreleased]
9+
10+
### Changed
11+
12+
- Include/exclude semantics are more precisely defined ([#5](https://github.com/stac-api-extensions/fields/pull/5))
13+
714
## [v1.0.0-rc.2] - 2022-11-01
815

916
### Fixed

README.md

Lines changed: 159 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -35,32 +35,32 @@ an object value to the core JSON search request body. The `fields` object contai
3535
values, `include` and `exclude`.
3636

3737
When used with GET, the semantics are the same, except the syntax is a single parameter `fields` with
38-
a comma-separated list of attribute names, where `exclude` values are those prefixed by a `-` and `include` values are
38+
a comma-separated list of field names, where `exclude` values are those prefixed by a `-` and `include` values are
3939
those with no prefix, e.g., `-geometry`, or `id,-geometry,properties`.
4040

4141
It is recommended that implementations provide exactly the `include` and `exclude` sets specified by the request,
4242
but this is not required. These values are only hints to the server as to the desires of the client, and not a
4343
contract about what the response will be. Implementations are still considered compliant if fields not specified as part of `include`
4444
are in the response or ones specified as part of `exclude` are. For example, implementations may choose to always
4545
include simple string fields like `id` and `type` regardless of the `exclude` specification. However, it is recommended
46-
that implementations honor excludes for attributes with more complex and arbitrarily large values
46+
that implementations honor excludes for fields with more complex and arbitrarily large values
4747
(e.g., `geometry`, `assets`). For example, some Item objects may have a geometry with a simple 5 point polygon, but these
4848
polygons can be very large when reprojected to EPSG:4326, as in the case of a highly-decimated sinusoidal polygons.
4949
Implementations are also not required to implement semantics for nested values whereby one can include an object, but
50-
exclude attributes of that object, e.g., include `properties` but exclude `properties.datetime`.
50+
exclude fields of that object, e.g., include `properties` but exclude `properties.datetime`.
5151

52-
No error must be returned if a specified field has no value for it in the catalog. For example, if the attribute
52+
No error must be returned if a specified field has no value for it in the catalog. For example, if the field
5353
"properties.eo:cloud_cover" is specified but there is no cloud cover value for an Item, a successful HTTP response
5454
must be returned and the Item entities will not contain that
55-
attribute.
55+
field.
5656

5757
If no `fields` are specified, the response is **must** be a valid
5858
[ItemCollection](https://github.com/radiantearth/stac-spec/tree/v1.0.0-rc.2/itemcollection/README.md). If a client excludes
59-
attributes that are required in a STAC Item, the server may return an invalid STAC Item. For example, if `type`
59+
fields that are required in a STAC Item, the server may return an invalid STAC Item. For example, if `type`
6060
and `geometry` are excluded, the entity will not even be a valid GeoJSON Feature, or if `bbox` is excluded then the entity
6161
will not be a valid STAC Item.
6262

63-
Implementations may return attributes not specified, e.g., id, but must avoid anything other than a minimal entity
63+
Implementations may return fields not specified, e.g., id, but must avoid anything other than a minimal entity
6464
representation.
6565

6666
This specification does not yet require the implementation of an "-ables" endpoint (like CQL2 does for queryables)
@@ -69,46 +69,154 @@ fields that can be selected, so implementations must provide this out-of-band. I
6969
fields in Item Properties to be prefixed with `properties.` or not, or support use of both the prefixed and non-prefixed
7070
name, e.g., `properties.datetime` or `datetime`.
7171

72-
## Include/Exclude Semantics
73-
74-
1. If `fields` attribute is specified with an empty object, or with both `include` and `exclude` set to null or an
75-
empty array, the recommended behavior is as if `include` was set to
76-
`["id", "type", "geometry", "bbox", "links", "assets", "properties.datetime"]`. This default is so that the entity
77-
returned is a valid STAC Item. Implementations may choose to add other properties, e.g., `created`, but the number
78-
of default properties attributes should be kept to a minimum.
79-
2. If only `include` is specified, these attributes are added to the default set of attributes (set union operation).
80-
3. If only `exclude` is specified, these attributes are subtracted from the union of the default set of attributes and
81-
the `include` attributes (set difference operation). This will result in an entity that is not a valid Item if any
82-
of the excluded attributes are in the default set of attributes.
83-
4. If both `include` and `exclude` attributes are specified, semantics are that a field must be included and **not**
84-
excluded. E.g., if `properties` is included and `properties.datetime` is excluded, then `datetime` must not appear
85-
in the attributes of `properties`.
72+
## Include/Exclude Semantics
73+
74+
1. If `fields` attribute is specified as an empty string (GET requests) or as an empty object or an object with both `include` and `exclude` set to either null or an
75+
empty array (for POST requests), then the recommended behavior is to include only fields
76+
`type`, `stac_version`, `id`, `geometry`, `bbox`, `links`, `assets`, and `properties.datetime`. If `properties.datetime` is null, then it is recommended to include `properties.start_datetime` and `properties.end_datetime`.
77+
These are the default fields to ensure a valid STAC Item is returned by default.
78+
Implementations may choose to include other properties, e.g., `properties.created`, but the number
79+
of default properties fields should be kept to a minimum.
80+
2. If only `include` is specified, these fields should be the only fields included.
81+
Any additional fields provided beyond those in the `include` list should be kept
82+
to a minimum, as the caller has explicitly stated they do not need them.
83+
3. If only `exclude` is specified, the specified fields should not be
84+
included, but every other field available for the
85+
Item should be included.
86+
4. If `exclude` is specified and `include` is null or an empty
87+
array, then the `exclude` fields should be excluded from the default set.
88+
5. For nested fields (e.g., `properties.datetime`), the most specific path
89+
should be honored first, and `include` should be preferred over `exclude`. For
90+
example:
91+
1. If a field is in `exclude`, and a nested field of that field is in
92+
`include`, the nested field should be included, but no other nested
93+
fields in the field should be included. For example, if `properties` is
94+
excluded and `properties.datetime` is included, then `datetime`
95+
should be the only nested field in `properties`.
96+
2. If a field is in `include`, and a nested field of that field is in `exclude`, the field
97+
should be included, and the nested field should be excluded. For example,
98+
if `properties` is included and `properties.datetime` is excluded, then
99+
`datetime` should not be in `properties`, but every other nested field should be.
100+
6. If the same field is present in both `include` and `exclude`, it should be included.
101+
7. If a field is not present in `include`, but it is present in `exclude`, it should be excluded.
102+
103+
### `null` vs. empty vs. missing
104+
105+
There is a semantic difference between a missing value (i.e., if `include` is not
106+
in the JSON object), and a `null` or empty value. The recommended behavior
107+
around missing vs. null and empty values is described in [the section
108+
above](#includeexclude-semantics), but is summarized here for reference. "ALL"
109+
means that all of the Item's fields should be returned; "DEFAULT" means the
110+
default set (i.e. the required fields for a valid STAC Item) should be returned.
111+
112+
| include | exclude | returned |
113+
| -- | -- | -- |
114+
| missing, `null`, or empty | missing, `null`, or empty | DEFAULT |
115+
| ["a", "b"] | missing, `null`, or empty | ["a", "b"] |
116+
| missing | ["a", "b"] | ALL except for ["a", "b"] |
117+
| `null` or empty | ["a", "b"] | DEFAULT except for ["a", "b"] |
118+
| ["a", "b"] | ["a"] | ["a", "b"] |
119+
| ["a"] | ["a", "b"] | ["a"] |
120+
| ["a.b"] | ["a"] | `a.b` should be the only field in `a` |
121+
| ["a"] | ["a.b"] | `a` should be included, but it should not include `a.b` |
122+
123+
In this example, `include` is missing:
124+
125+
```json
126+
{
127+
"fields": {
128+
"exclude": ["geometry"]
129+
}
130+
}
131+
```
132+
133+
In these two examples, `include` is `null` and empty, respectively:
134+
135+
```json
136+
{
137+
"fields": {
138+
"include": null,
139+
"exclude": ["geometry"]
140+
}
141+
}
142+
```
143+
144+
```json
145+
{
146+
"fields": {
147+
"include": [],
148+
"exclude": ["geometry"]
149+
}
150+
}
151+
```
152+
153+
The special case of both `include` and `exclude` missing is possible both in
154+
JSON and text. In JSON:
155+
156+
```json
157+
{
158+
"fields": {}
159+
}
160+
```
161+
162+
or
163+
164+
```json
165+
{
166+
"fields": null
167+
}
168+
```
169+
170+
In text:
171+
172+
```text
173+
?fields=
174+
```
175+
176+
It is not possible to differentiate between missing, null, and empty for the
177+
text representation, so implementations should assume the value is empty, NOT
178+
missing. For example, this is a case of `include` being empty (NOT missing):
179+
180+
```text
181+
?fields=-geometry
182+
```
86183

87184
## Examples
88185

89-
Return baseline fields. This **must** return valid STAC Item entities.
186+
### Default fields
187+
188+
Return the default fields. This should return valid STAC Item entities.
90189

91190
Query Parameters
92-
```http
191+
192+
```text
93193
?fields=
94194
```
95195

96196
JSON
197+
97198
```json
98199
{
99200
"fields": {
100201
}
101202
}
102203
```
103204

104-
This has a similar effect as an empty object for `fields`, but it is up to the discretion of the implementation
205+
### Explicitly get a valid STAC Item
206+
207+
Because implementations may choose to always include other fields (e.g.,
208+
extension-specific fields such as
209+
[sar](https://github.com/stac-extensions/sar)), this could has the same effect
210+
as an empty object for `fields`.
105211

106212
Query Parameters
107-
```http
108-
?fields=id,type,geometry,bbox,properties,links,assets
213+
214+
```text
215+
?fields=id,type,geometry,bbox,properties.datetime,links,assets,stac_version
109216
```
110217

111218
JSON
219+
112220
```json
113221
{
114222
"fields": {
@@ -117,22 +225,27 @@ JSON
117225
"type",
118226
"geometry",
119227
"bbox",
120-
"properties",
228+
"properties.datetime",
121229
"links",
122-
"assets"
230+
"assets",
231+
"stac_version",
123232
]
124233
}
125234
}
126235
```
127236

128-
Exclude `geometry` from the baseline fields. This **must** return an entity that is not a valid GeoJSON Feature or a valid STAC Item.
237+
### Exclude geometry
238+
239+
Exclude `geometry` from the full item. This will return an entity that is not a valid GeoJSON Feature or a valid STAC Item.
129240

130241
Query Parameters
131-
```http
242+
243+
```text
132244
?fields=-geometry
133245
```
134246

135247
JSON
248+
136249
```json
137250
{
138251
"fields": {
@@ -143,17 +256,21 @@ JSON
143256
}
144257
```
145258

146-
To return the `id`, `type`, `geometry`, and the Properties attribute `eo:cloud_cover`.
147-
This **must** return a valid STAC Item, as the includes are added to the default includes.
148-
Explicitly specifying `id`, `type`, and `geometry` has not effect as these are default fields,
149-
but `properties.eo:cloud_cover` is not a default field and thereby should be in the response.
259+
### Minimal subset
260+
261+
Return the `id`, `type`, `geometry`, and the Properties field `eo:cloud_cover`.
262+
This is not guaranteed not return a valid STAC Item, since not all required Item
263+
fields are included, but an implementor may choose to return a valid STAC
264+
item anyways.
150265

151266
Query Parameters
152-
```http
267+
268+
```text
153269
?fields=id,type,geometry,properties.eo:cloud_cover
154270
```
155271

156272
JSON
273+
157274
```json
158275
{
159276
"fields": {
@@ -167,19 +284,24 @@ JSON
167284
}
168285
```
169286

287+
### Exclude a nested fiels
288+
170289
To include `id` and all the properties fields, except for the `foo` field.
171290

172291
Query Parameters
173-
```http
292+
293+
```text
174294
?fields=id,properties,-properties.foo
175295
```
176296

177297
also valid:
178-
```http
298+
299+
```text
179300
?fields=+id,+properties,-properties.foo
180301
```
181302

182303
JSON
304+
183305
```json
184306
{
185307
"fields": {

0 commit comments

Comments
 (0)