Skip to content

[2.8/3] Add limitToLast implementation and tests. #2296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions packages/firebase/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8079,14 +8079,26 @@ declare namespace firebase.firestore {
): Query;

/**
* Creates and returns a new Query where the results are limited to the
* specified number of documents.
* Creates and returns a new Query that only returns the first matching
* documents.
*
* @param limit The maximum number of items to return.
* @return The created Query.
*/
limit(limit: number): Query;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original API proposal suggests to make this method as deprecated. Should we do that and introduce limitToFirst()? In that case, I would also leave the comments for limit as they were.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the decision was to keep limit and add limitToLast only.

See the comments in go/limitToLast

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Do you mind updating the proposed API so this is immediately apparent? The API proposal should be the source of truth and we should in general update the proposal to match the decision from the API council.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some edit suggestions there.


/**
* Creates and returns a new Query that only returns the last matching
* documents.
*
* You must specify at least one `orderBy` clause for `limitToLast` queries,
* otherwise an exception will be thrown thrown during execution.
*
* @param limit The maximum number of items to return.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This documentation should mention that the query requires an orderBy() clause.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

You must specify at least one `orderBy` clause for `limitToLast` queries, otherwise an exception will be thrown thrown during execution.

If you like your version better, we should still make it "an exception will be thrown".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found your version better, as always..

* @return The created Query.
*/
limitToLast(limit: number): Query;

/**
* Creates and returns a new Query that starts at the provided document
* (inclusive). The starting position is relative to the order of the query.
Expand Down
16 changes: 14 additions & 2 deletions packages/firestore-types/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1083,14 +1083,26 @@ export class Query {
): Query;

/**
* Creates and returns a new Query that's additionally limited to only
* return up to the specified number of documents.
* Creates and returns a new Query that only returns the first matching
* documents.
*
* @param limit The maximum number of items to return.
* @return The created Query.
*/
limit(limit: number): Query;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure that both API definitions/comments are in sync (they seemed to be slightly different even before this PR).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


/**
* Creates and returns a new Query that only returns the last matching
* documents.
*
* Queries with `limitToLast` must have at least one `orderBy` clause on
* one of the document fields, or an Exception will throw during execution.
*
* @param limit The maximum number of items to return.
* @return The created Query.
*/
limitToLast(limit: number): Query;

/**
* Creates and returns a new Query that starts at the provided document
* (inclusive). The starting position is relative to the order of the query.
Expand Down
29 changes: 21 additions & 8 deletions packages/firestore/src/api/database.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ import {
validateOptionalArgType,
validateOptionalArrayElements,
validateOptionNames,
validatePositiveNumber,
validateStringEnum,
valueDescription
} from '../util/input_validation';
Expand Down Expand Up @@ -1539,14 +1540,15 @@ export class Query implements firestore.Query {
limit(n: number): firestore.Query {
validateExactNumberOfArgs('Query.limit', arguments, 1);
validateArgType('Query.limit', 'number', 1, n);
if (n <= 0) {
throw new FirestoreError(
Code.INVALID_ARGUMENT,
`Invalid Query. Query limit (${n}) is invalid. Limit must be ` +
'positive.'
);
}
return new Query(this._query.withLimit(n), this.firestore);
validatePositiveNumber('Query.limit', 1, n);
return new Query(this._query.withLimitToFirst(n), this.firestore);
}

limitToLast(n: number): firestore.Query {
validateExactNumberOfArgs('Query.limitToLast', arguments, 1);
validateArgType('Query.limitToLast', 'number', 1, n);
validatePositiveNumber('Query.limitToLast', 1, n);
return new Query(this._query.withLimitToLast(n), this.firestore);
}

startAt(
Expand Down Expand Up @@ -1826,6 +1828,7 @@ export class Query implements firestore.Query {
complete: args[currArg + 2] as CompleteFn
};
}
this.validateHasExplicitOrderByForLimitToLast(this._query);
return this.onSnapshotInternal(options, observer);
}

Expand Down Expand Up @@ -1861,9 +1864,19 @@ export class Query implements firestore.Query {
};
}

private validateHasExplicitOrderByForLimitToLast(query: InternalQuery): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is certainly up for debate, but at least from an implementation standpoint it would be easier to do this validation in limitToLast itself. This would also follow the precedent we use for cursors - startAt().orderBy() throws on invocation, which is a little unfriendly to our users but makes our code cleaner.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not aware there was a precedent already, otherwise I'd say the current way is better because it makes user's job easier.

I'll raise this in chat to see how others feel about this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per our standup discussion, let's stay consistent with the rest of the Query methods (and explore the option to allow startAt().order()). Let's figure out what is feasible and hopefully we can leave this PR as is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plan to address this later: b/143499666 and keep this PR as-is for now, is that OK?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that is ok. Please do make sure that this is feasible before we expose limitToLast() publicly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

if (query.hasLimitToLast() && query.explicitOrderBy.length === 0) {
throw new FirestoreError(
Code.UNIMPLEMENTED,
'limitToLast() queries require specifying at least one orderBy() clause'
);
}
}

get(options?: firestore.GetOptions): Promise<firestore.QuerySnapshot> {
validateBetweenNumberOfArgs('Query.get', arguments, 0, 1);
validateGetOptions('Query.get', options);
this.validateHasExplicitOrderByForLimitToLast(this._query);
return new Promise(
(resolve: Resolver<firestore.QuerySnapshot>, reject: Rejecter) => {
if (options && options.source === 'cache') {
Expand Down
99 changes: 84 additions & 15 deletions packages/firestore/src/core/query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ import { Code, FirestoreError } from '../util/error';
import { isNullOrUndefined } from '../util/types';
import { Target } from './target';

export enum LimitType {
First = 'F',
Last = 'L'
}

/**
* Query encapsulates all the query attributes we support in the SDK. It can
* be run against the LocalStore, as well as be converted to a `Target` to
Expand All @@ -55,6 +60,7 @@ export class Query {
readonly explicitOrderBy: OrderBy[] = [],
readonly filters: Filter[] = [],
readonly limit: number | null = null,
readonly limitType: LimitType = LimitType.First,
readonly startAt: Bound | null = null,
readonly endAt: Bound | null = null
) {
Expand Down Expand Up @@ -133,6 +139,7 @@ export class Query {
this.explicitOrderBy.slice(),
newFilters,
this.limit,
this.limitType,
this.startAt,
this.endAt
);
Expand All @@ -148,18 +155,33 @@ export class Query {
newOrderBy,
this.filters.slice(),
this.limit,
this.limitType,
this.startAt,
this.endAt
);
}

withLimitToFirst(limit: number | null): Query {
return new Query(
this.path,
this.collectionGroup,
this.explicitOrderBy.slice(),
this.filters.slice(),
limit,
LimitType.First,
this.startAt,
this.endAt
);
}

withLimit(limit: number | null): Query {
withLimitToLast(limit: number | null): Query {
return new Query(
this.path,
this.collectionGroup,
this.explicitOrderBy.slice(),
this.filters.slice(),
limit,
LimitType.Last,
this.startAt,
this.endAt
);
Expand All @@ -172,6 +194,7 @@ export class Query {
this.explicitOrderBy.slice(),
this.filters.slice(),
this.limit,
this.limitType,
bound,
this.endAt
);
Expand All @@ -184,6 +207,7 @@ export class Query {
this.explicitOrderBy.slice(),
this.filters.slice(),
this.limit,
this.limitType,
this.startAt,
bound
);
Expand All @@ -202,6 +226,7 @@ export class Query {
this.explicitOrderBy.slice(),
this.filters.slice(),
this.limit,
this.limitType,
this.startAt,
this.endAt
);
Expand All @@ -227,15 +252,20 @@ export class Query {
// example, use as a dictionary key, but the implementation is subject to
// collisions. Make it collision-free.
canonicalId(): string {
return this.toTarget().canonicalId();
return `${this.toTarget().canonicalId()}|lt:${this.limitType}`;
}

toString(): string {
return `Query(target=${this.toTarget().toString()})`;
return `Query(target=${this.toTarget().toString()}; limitType=${
this.limitType
})`;
}

isEqual(other: Query): boolean {
return this.toTarget().isEqual(other.toTarget());
return (
this.toTarget().isEqual(other.toTarget()) &&
this.limitType === other.limitType
);
}

docComparator(d1: Document, d2: Document): number {
Expand Down Expand Up @@ -264,8 +294,12 @@ export class Query {
);
}

hasLimit(): boolean {
return !isNullOrUndefined(this.limit);
hasLimitToFirst(): boolean {
return !isNullOrUndefined(this.limit) && this.limitType === LimitType.First;
}

hasLimitToLast(): boolean {
return !isNullOrUndefined(this.limit) && this.limitType === LimitType.Last;
}

getFirstOrderByField(): FieldPath | null {
Expand Down Expand Up @@ -304,17 +338,52 @@ export class Query {
return this.collectionGroup !== null;
}

/**
* Converts this `Query` instance to it's corresponding `Target`
* representation.
*/
toTarget(): Target {
if (!this.target) {
this.target = new Target(
this.path,
this.collectionGroup,
this.orderBy,
this.filters,
this.limit,
this.startAt,
this.endAt
);
if (this.limitType === LimitType.First) {
this.target = new Target(
this.path,
this.collectionGroup,
this.orderBy,
this.filters,
this.limit,
this.startAt,
this.endAt
);
} else {
// Flip the orderBy directions since we want the last results
const orderBys = [] as OrderBy[];
for (const orderBy of this.orderBy) {
const dir =
orderBy.dir === Direction.DESCENDING
? Direction.ASCENDING
: Direction.DESCENDING;
orderBys.push(new OrderBy(orderBy.field, dir));
}

// We need to swap the cursors to match the now-flipped query ordering.
const startAt = this.endAt
? new Bound(this.endAt.position, !this.endAt.before)
: null;
const endAt = this.startAt
? new Bound(this.startAt.position, !this.startAt.before)
: null;

// Now return as a LimitType.First query.
this.target = new Target(
this.path,
this.collectionGroup,
orderBys,
this.filters,
this.limit,
startAt,
endAt
);
}
}
return this.target!;
}
Expand Down
48 changes: 42 additions & 6 deletions packages/firestore/src/core/sync_engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,9 @@ import {
import * as objUtils from '../util/obj';
import { SortedSet } from '../util/sorted_set';
import { ListenSequence } from './listen_sequence';
import { Query } from './query';
import { Query, LimitType } from './query';
import { SnapshotVersion } from './snapshot_version';
import { Target } from './target';
import { TargetIdGenerator } from './target_id_generator';
import { Transaction } from './transaction';
import {
Expand Down Expand Up @@ -216,7 +217,8 @@ export class SyncEngine implements RemoteSyncer, SharedClientStateSyncer {
this.sharedClientState.addLocalQueryTarget(targetId);
viewSnapshot = queryView.view.computeInitialSnapshot();
} else {
const queryData = await this.localStore.allocateQuery(query);
const queryData = await this.localStore.allocateTarget(query.toTarget());

const status = this.sharedClientState.addLocalQueryTarget(
queryData.targetId
);
Expand Down Expand Up @@ -305,8 +307,18 @@ export class SyncEngine implements RemoteSyncer, SharedClientStateSyncer {
const queryView = this.queryViewsByQuery.get(query)!;
assert(!!queryView, 'Trying to unlisten on query not found:' + query);

// TODO(wuandy): Note this does not handle the case where multiple queries
// map to one target, and user request to unlisten on of the queries.
// If it's not the only query mapped to the target, only clean up
// query view and keep the target active.
const queries = this.queriesByTarget[queryView.targetId];
if (queries.length > 1) {
this.queriesByTarget[queryView.targetId] = queries.filter(
q => !q.isEqual(query)
);
this.queryViewsByQuery.delete(query);
return;
}

// No other queries are mapped to the target, clean up the query and the target.
if (this.isPrimary) {
// We need to remove the local query target first to allow us to verify
// whether any other client is still interested in this target.
Expand Down Expand Up @@ -983,7 +995,7 @@ export class SyncEngine implements RemoteSyncer, SharedClientStateSyncer {
assert(!!target, `Target for id ${targetId} not found`);
queryData = await this.localStore.allocateTarget(target!);
await this.initializeViewAndComputeSnapshot(
target!.toTargetQuery(),
this.synthesizeTargetToQuery(target!),
targetId,
/*current=*/ false
);
Expand All @@ -996,6 +1008,30 @@ export class SyncEngine implements RemoteSyncer, SharedClientStateSyncer {
return activeQueries;
}

/**
* Creates a `Query` object from the specified `Target`. There is no way to
* obtain the original `Query`, so we synthesize a `Query` from the `Target`
* object.
*
* The synthesized result might be different from the original `Query`, but
* since the synthesized `Query` should return the same results as the
* original one (only the presentation of results might differ), and this is
* only used for multi-tab, the potential difference will not cause issues.
*/
// PORTING NOTE: Multi-tab only
private synthesizeTargetToQuery(target: Target): Query {
return new Query(
target.path,
target.collectionGroup,
target.orderBy,
target.filters,
target.limit,
LimitType.First,
target.startAt,
target.endAt
);
}

// PORTING NOTE: Multi-tab only
getActiveClients(): Promise<ClientId[]> {
return this.localStore.getActiveClients();
Expand Down Expand Up @@ -1061,7 +1097,7 @@ export class SyncEngine implements RemoteSyncer, SharedClientStateSyncer {
assert(!!target, `Query data for active target ${targetId} not found`);
const queryData = await this.localStore.allocateTarget(target!);
await this.initializeViewAndComputeSnapshot(
target!.toTargetQuery(),
this.synthesizeTargetToQuery(target!),
queryData.targetId,
/*current=*/ false
);
Expand Down
Loading