Skip to content

OR queries more validation #6785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
Nov 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
effced9
Add composite filters. Port of https://github.com/firebase/firebase-a…
MarkDuckworth Jun 24, 2022
eba3b84
Cleanup: lint
MarkDuckworth Jun 24, 2022
33a0896
Support encoding and decoding Composite Filters
MarkDuckworth Jun 29, 2022
9f6e216
Formatting
MarkDuckworth Jun 30, 2022
6556240
Merge remote-tracking branch 'origin/markduckworth/or-queries' into m…
MarkDuckworth Jul 26, 2022
78a8900
Fix issue with readonly array.
MarkDuckworth Jul 26, 2022
80fc05d
WIP: serving results from index.
MarkDuckworth Aug 5, 2022
8b6f05f
Perform DNF transform
MarkDuckworth Aug 10, 2022
152fc18
Cleanup and tests from https://github.com/firebase/firebase-android-s…
MarkDuckworth Aug 10, 2022
8b0bfbd
Merge branch 'markduckworth/or-queries' of github.com:firebase/fireba…
MarkDuckworth Aug 12, 2022
1c948cb
use index if we have composite filters
MarkDuckworth Aug 12, 2022
2cad46c
Merge branch 'markduckworth/or-queries' into markduckworth/or-queries…
MarkDuckworth Aug 12, 2022
266f7f6
Fix bad import
MarkDuckworth Aug 12, 2022
c2d7cd2
Update the orderBy filtering mechanism for OR queries and steps to se…
MarkDuckworth Aug 15, 2022
41d0e29
Adding tests
MarkDuckworth Aug 15, 2022
90f7c50
Test for DNF computation.
MarkDuckworth Aug 16, 2022
ff52ecc
Fix implementation of isFlatConjunction
MarkDuckworth Aug 16, 2022
dfda9f8
Adding integration tests for validation of composite queries.
MarkDuckworth Aug 17, 2022
9bc3336
IndexDb manager tests for partial and full index.
MarkDuckworth Aug 17, 2022
d66abd4
Test query does not include documents with missing fields
MarkDuckworth Aug 17, 2022
e33263d
Run prettier
MarkDuckworth Aug 17, 2022
01c3487
Add missing awaits
MarkDuckworth Aug 17, 2022
7d56b57
Fixing argument validation for and . Updating tests for asserted err…
MarkDuckworth Aug 17, 2022
d01ff74
Prettier
MarkDuckworth Aug 18, 2022
186fd92
Removing duplicate test cases.
MarkDuckworth Aug 18, 2022
409b19c
Update proto with OR operator.
MarkDuckworth Aug 22, 2022
131f4be
Update protos with OR operator.
MarkDuckworth Aug 23, 2022
a514358
Testing query with indexes
MarkDuckworth Aug 23, 2022
0ab215b
Add compile.sh to compile proto updates.
MarkDuckworth Aug 23, 2022
638b0ad
Adding new script to run browser tests against the emulator.
MarkDuckworth Aug 23, 2022
86c9057
Update stringifyFilter to support CompositeFilters
MarkDuckworth Aug 23, 2022
3501135
Lint and formatting
MarkDuckworth Aug 23, 2022
4adf98d
Refactor core/target.ts, extracting classes Bound, OrderBy, and Filte…
MarkDuckworth Aug 24, 2022
60f4276
Formatting
MarkDuckworth Aug 24, 2022
476be61
Making public API changes to ensure that only QueryFilterConstraints …
MarkDuckworth Aug 24, 2022
a0e5d30
Remove QueryNonFilterConstraint from exports.
MarkDuckworth Aug 25, 2022
77afd20
Add overloads for query and deprecate existing query overload that su…
MarkDuckworth Aug 26, 2022
3862740
Merge branch 'markduckworth/or-queries' of github.com:firebase/fireba…
MarkDuckworth Oct 26, 2022
efd98c7
Implementing public API for OR queries that does not need the depreca…
MarkDuckworth Oct 27, 2022
94250da
Implementing 3rd iteration of the public API for OR queries.
MarkDuckworth Oct 28, 2022
ff0eede
Compute IN expansion.
MarkDuckworth Oct 28, 2022
66bf749
Fix edge case a == 1 && a == 6
MarkDuckworth Nov 3, 2022
968d133
Logic utils tests for in expansion.
MarkDuckworth Nov 3, 2022
5f37954
Fixed rogue comment.
MarkDuckworth Nov 8, 2022
ab5cad2
Comment cleanup.
MarkDuckworth Nov 8, 2022
5fa9c76
Comment cleanup.
MarkDuckworth Nov 8, 2022
9c6395b
Test refactoring.
MarkDuckworth Nov 8, 2022
09046e7
Collapsing duplicate tests that covered with and without client side …
MarkDuckworth Nov 9, 2022
ed0e192
Additional validation and additional validation tests for where(..), …
MarkDuckworth Nov 12, 2022
99c7ef8
Fixed type-o in error message.
MarkDuckworth Nov 12, 2022
789c481
Merge branch 'markduckworth/or-queries' of github.com:firebase/fireba…
MarkDuckworth Nov 12, 2022
c3c7afb
Tagging all new OR Query API members as internal until server support…
MarkDuckworth Nov 14, 2022
fb662cd
Constrained the parameters of the query() function implementation. It…
MarkDuckworth Nov 15, 2022
8405c97
Lint and formatting.
MarkDuckworth Nov 15, 2022
99fb4b9
query() implementation cleaner param names and types.
MarkDuckworth Nov 16, 2022
bfaac63
Changing queryConstraint param from optional to union undefined.
MarkDuckworth Nov 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 0 additions & 17 deletions common/api-review/firestore-lite.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@ export type AddPrefixToKeys<Prefix extends string, T extends Record<string, unkn
[K in keyof T & string as `${Prefix}.${K}`]+?: T[K];
};

// @public
export function and(...queryConstraints: QueryFilterConstraint[]): QueryCompositeFilterConstraint;

// @public
export function arrayRemove(...elements: unknown[]): FieldValue;

Expand Down Expand Up @@ -203,9 +200,6 @@ export type NestedUpdateFields<T extends Record<string, unknown>> = UnionToInter
[K in keyof T & string]: ChildUpdateFields<K, T[K]>;
}[keyof T & string]>;

// @public
export function or(...queryConstraints: QueryFilterConstraint[]): QueryCompositeFilterConstraint;

// @public
export function orderBy(fieldPath: string | FieldPath, directionStr?: OrderByDirection): QueryOrderByConstraint;

Expand All @@ -230,17 +224,9 @@ export class Query<T = DocumentData> {
withConverter<U>(converter: FirestoreDataConverter<U>): Query<U>;
}

// @public
export function query<T>(query: Query<T>, compositeFilter: QueryCompositeFilterConstraint, ...queryConstraints: QueryNonFilterConstraint[]): Query<T>;

// @public
export function query<T>(query: Query<T>, ...queryConstraints: QueryConstraint[]): Query<T>;

// @public
export class QueryCompositeFilterConstraint {
readonly type: 'or' | 'and';
}

// @public
export abstract class QueryConstraint {
abstract readonly type: QueryConstraintType;
Expand Down Expand Up @@ -268,9 +254,6 @@ export class QueryFieldFilterConstraint extends QueryConstraint {
readonly type = "where";
}

// @public
export type QueryFilterConstraint = QueryFieldFilterConstraint | QueryCompositeFilterConstraint;

// @public
export class QueryLimitConstraint extends QueryConstraint {
readonly type: 'limit' | 'limitToLast';
Expand Down
17 changes: 0 additions & 17 deletions common/api-review/firestore.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@ export type AddPrefixToKeys<Prefix extends string, T extends Record<string, unkn
[K in keyof T & string as `${Prefix}.${K}`]+?: T[K];
};

// @public
export function and(...queryConstraints: QueryFilterConstraint[]): QueryCompositeFilterConstraint;

// @public
export function arrayRemove(...elements: unknown[]): FieldValue;

Expand Down Expand Up @@ -327,9 +324,6 @@ export function onSnapshotsInSync(firestore: Firestore, observer: {
// @public
export function onSnapshotsInSync(firestore: Firestore, onSync: () => void): Unsubscribe;

// @public
export function or(...queryConstraints: QueryFilterConstraint[]): QueryCompositeFilterConstraint;

// @public
export function orderBy(fieldPath: string | FieldPath, directionStr?: OrderByDirection): QueryOrderByConstraint;

Expand Down Expand Up @@ -359,17 +353,9 @@ export class Query<T = DocumentData> {
withConverter<U>(converter: FirestoreDataConverter<U>): Query<U>;
}

// @public
export function query<T>(query: Query<T>, compositeFilter: QueryCompositeFilterConstraint, ...queryConstraints: QueryNonFilterConstraint[]): Query<T>;

// @public
export function query<T>(query: Query<T>, ...queryConstraints: QueryConstraint[]): Query<T>;

// @public
export class QueryCompositeFilterConstraint {
readonly type: 'or' | 'and';
}

// @public
export abstract class QueryConstraint {
abstract readonly type: QueryConstraintType;
Expand Down Expand Up @@ -397,9 +383,6 @@ export class QueryFieldFilterConstraint extends QueryConstraint {
readonly type = "where";
}

// @public
export type QueryFilterConstraint = QueryFieldFilterConstraint | QueryCompositeFilterConstraint;

// @public
export class QueryLimitConstraint extends QueryConstraint {
readonly type: 'limit' | 'limitToLast';
Expand Down
55 changes: 41 additions & 14 deletions packages/firestore/src/lite-api/query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ export abstract class QueryConstraint extends AppliableConstraint {
* apply (e.g. {@link orderBy}, {@link limit}).
* @throws if any of the provided query constraints cannot be combined with the
* existing or new constraints.
* @internal TODO remove this internal tag with OR Query support in the server
*/
export function query<T>(
query: Query<T>,
Expand All @@ -154,24 +155,21 @@ export function query<T>(

export function query<T>(
query: Query<T>,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
filterOrQueryConstraints: any,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
...nonFilters: any
queryConstraint: QueryCompositeFilterConstraint | QueryConstraint | undefined,
...additionalQueryConstraints: Array<
QueryConstraint | QueryNonFilterConstraint
>
): Query<T> {
let queryConstraints: AppliableConstraint[] = [];
if (filterOrQueryConstraints instanceof AppliableConstraint) {
queryConstraints.push(filterOrQueryConstraints);
} else if (Array.isArray(filterOrQueryConstraints)) {
queryConstraints.concat(filterOrQueryConstraints as AppliableConstraint[]);
}

if (nonFilters !== undefined) {
queryConstraints = queryConstraints.concat(
nonFilters as QueryNonFilterConstraint[]
);
if (queryConstraint instanceof AppliableConstraint) {
queryConstraints.push(queryConstraint);
}

queryConstraints = queryConstraints.concat(additionalQueryConstraints);

validateQueryConstraintArray(queryConstraints);

for (const constraint of queryConstraints) {
query = constraint._apply(query);
}
Expand Down Expand Up @@ -278,6 +276,7 @@ export function where(
* `QueryCompositeFilterConstraint`s are created by invoking {@link or} or
* {@link and} and can then be passed to {@link query} to create a new query
* instance that also contains the `QueryCompositeFilterConstraint`.
* @internal TODO remove this internal tag with OR Query support in the server
*/
export class QueryCompositeFilterConstraint extends AppliableConstraint {
/**
Expand Down Expand Up @@ -358,6 +357,7 @@ export type QueryNonFilterConstraint =
* `QueryFilterConstraint`s are created by invoking {@link or} or {@link and}
* and can then be passed to {@link query} to create a new query instance that
* also contains the `QueryConstraint`.
* @internal TODO remove this internal tag with OR Query support in the server
*/
export type QueryFilterConstraint =
| QueryFieldFilterConstraint
Expand All @@ -371,6 +371,7 @@ export type QueryFilterConstraint =
* for OR operation. These must be created with calls to {@link where},
* {@link or}, or {@link and}.
* @returns The created {@link QueryCompositeFilterConstraint}.
* @internal TODO remove this internal tag with OR Query support in the server
*/
export function or(
...queryConstraints: QueryFilterConstraint[]
Expand All @@ -394,6 +395,7 @@ export function or(
* for AND operation. These must be created with calls to {@link where},
* {@link or}, or {@link and}.
* @returns The created {@link QueryCompositeFilterConstraint}.
* @internal TODO remove this internal tag with OR Query support in the server
*/
export function and(
...queryConstraints: QueryFilterConstraint[]
Expand Down Expand Up @@ -1214,7 +1216,32 @@ export function validateQueryFilterConstraint(
) {
throw new FirestoreError(
Code.INVALID_ARGUMENT,
`Function ${functionName}() requires AppliableContraints created with a call to 'where(...)', 'or(...)', or 'and(...)'.`
`Function ${functionName}() requires AppliableConstraints created with a call to 'where(...)', 'or(...)', or 'and(...)'.`
);
}
}

function validateQueryConstraintArray(
queryConstraint: AppliableConstraint[]
): void {
const compositeFilterCount = queryConstraint.filter(
filter => filter instanceof QueryCompositeFilterConstraint
).length;
const fieldFilterCount = queryConstraint.filter(
filter => filter instanceof QueryFieldFilterConstraint
).length;

if (
compositeFilterCount > 1 ||
(compositeFilterCount > 0 && fieldFilterCount > 0)
) {
throw new FirestoreError(
Code.INVALID_ARGUMENT,
'InvalidQuery. When using composite filters, you cannot use ' +
'more than one filter at the top level. Consider nesting the multiple ' +
'filters within an `and(...)` statement. For example: ' +
'change `query(query, where(...), or(...))` to ' +
'`query(query, and(where(...), or(...)))`.'
);
}
}
61 changes: 61 additions & 0 deletions packages/firestore/test/integration/api/validation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1399,7 +1399,68 @@ apiDescribe('Validation:', (persistence: boolean) => {
).to.throw(
"Invalid query. You cannot use 'not-in' filters with 'in' filters."
);

// Multiple top level composite filters
expect(() =>
// @ts-ignore
query(coll, and(where('a', '==', 'b')), or(where('b', '==', 'a')))
).to.throw(
'InvalidQuery. When using composite filters, you cannot use ' +
'more than one filter at the top level. Consider nesting the multiple ' +
'filters within an `and(...)` statement. For example: ' +
'change `query(query, where(...), or(...))` to ' +
'`query(query, and(where(...), or(...)))`.'
);

// Once top level composite filter and one top level field filter
expect(() =>
// @ts-ignore
query(coll, or(where('a', '==', 'b')), where('b', '==', 'a'))
).to.throw(
'InvalidQuery. When using composite filters, you cannot use ' +
'more than one filter at the top level. Consider nesting the multiple ' +
'filters within an `and(...)` statement. For example: ' +
'change `query(query, where(...), or(...))` to ' +
'`query(query, and(where(...), or(...)))`.'
);
});

validationIt(
persistence,
'passing non-filters to composite operators fails',
db => {
const compositeOperators = [
{ name: 'or', func: or },
{ name: 'and', func: and }
];
const nonFilterOps = [
limit(1),
limitToLast(2),
startAt(1),
startAfter(1),
endAt(1),
endBefore(1),
orderBy('a')
];

for (const compositeOp of compositeOperators) {
for (const nonFilterOp of nonFilterOps) {
const coll = collection(db, 'test');
expect(() =>
query(
coll,
compositeOp.func(
// @ts-ignore
nonFilterOp
)
)
).to.throw(
`Function ${compositeOp.name}() requires AppliableConstraints created with a call to 'where(...)', 'or(...)', or 'and(...)'.`
);
}
}
}
);
});
});

Expand Down