Skip to content

Fix missing query results when it is a match after a local patch mutation. #1957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
6 commits merged into from
Jul 12, 2019
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 40 additions & 1 deletion packages/firestore/src/local/local_documents_view.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ import {
MaybeDocumentMap,
maybeDocumentMap,
NullableMaybeDocumentMap,
nullableMaybeDocumentMap
nullableMaybeDocumentMap,
documentKeySet
} from '../model/collections';
import { Document, MaybeDocument, NoDocument } from '../model/document';
import { DocumentKey } from '../model/document_key';
Expand All @@ -37,6 +38,7 @@ import { MutationQueue } from './mutation_queue';
import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { RemoteDocumentCache } from './remote_document_cache';
import { PatchMutation } from '../model/mutation';

/**
* A readonly view of the local state of all documents we're tracking (i.e. we
Expand Down Expand Up @@ -208,6 +210,7 @@ export class LocalDocumentsView {
): PersistencePromise<DocumentMap> {
// Query the remote documents and overlay mutations.
let results: DocumentMap;
let mutationBatches: MutationBatch[];
return this.remoteDocumentCache
.getDocumentsMatchingQuery(transaction, query)
.next(queryResults => {
Expand All @@ -218,7 +221,43 @@ export class LocalDocumentsView {
);
})
.next(matchingMutationBatches => {
mutationBatches = matchingMutationBatches;
// It is possible that a PatchMutation can make a document a match to the query,
// but the version in `remoteDocumentCache` is not a match yet (waiting for server
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: s/remoteDocumentCache/the RemoteDocumentCache/ (since it is a widely used concept in Firestore)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// to ack). To handle this, we find all document keys affected by some`PatchMutation`s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/somePatchMutations/the PatchMutations/

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// that are not in `result` yet, and back fill them via `remoteDocumentCache.getEntries`,
// otherwise those `PatchMutations` will be ignored because no base document can be found,
// and lead to missing result for the query.
let missingBaseDocEntriesForPatching = documentKeySet();
for (const batch of matchingMutationBatches) {
for (const mutation of batch.mutations) {
const key = mutation.key;
// Only process documents belonging to the collection.
if (!query.path.isImmediateParentOf(key.path)) {
continue;
}
if (
mutation instanceof PatchMutation &&
results.get(mutation.key) === null
) {
missingBaseDocEntriesForPatching = missingBaseDocEntriesForPatching.add(
mutation.key
);
}
}
}
return this.remoteDocumentCache.getEntries(
transaction,
missingBaseDocEntriesForPatching
);
})
.next(baseDocsForPatching => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this variable name is perhaps slightly misleading; without context, it sounds like it contains all base documents, but in fact it's only a subset of those documents (those that have pending writes making them potentially match). Consider renaming (docsWithPendingWrites? docsWithLocalPatches?).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to docsWithPendingPatches.

baseDocsForPatching.forEach((key, doc) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: if you decide to follow the suggestion to create a helper function, perhaps you can move this filtering step there as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As i wrote in other reply, i'd like to keep the block as-is.

if (doc !== null && doc instanceof Document) {
results = results.insert(key, doc);
}
});
for (const batch of mutationBatches) {
for (const mutation of batch.mutations) {
const key = mutation.key;
// Only process documents belonging to the collection.
Expand Down
37 changes: 37 additions & 0 deletions packages/firestore/test/unit/specs/query_spec.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,4 +98,41 @@ describeSpec('Queries:', [], () => {
hasPendingWrites: true
});
});

specTest(
'Ensure correct query results for latency-compensated updates',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe 'Latency-compensated updates are included in query results'?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

[],
() => {
const fullQuery = Query.atPath(ResourcePath.fromString('collection'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be Query.atPath(path('collection'))

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

const filteredQuery = fullQuery.addFilter(filter('match', '==', true));
const docA = doc('collection/a', 1000, { match: false });
const docAv2Local = doc(
'collection/a',
1000,
{ match: true },
{ hasLocalMutations: true }
);

return (
spec()
.userListens(fullQuery)
.watchAcksFull(fullQuery, 1000, docA)
.expectEvents(fullQuery, { added: [docA] })

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that if you do userUnlisten(fullQuery) here then you can remove the event right below and further narrow down your test case.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure i understand..can you explain more?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am myself no longer fully sold on my idea. But let me explain what I had in mind:

    spec()
          .withGcEnabled(false)
          .userListens(fullQuery)
          .watchAcksFull(fullQuery, 1000, docA)
          .expectEvents(fullQuery, { added: [docA] })
          .userUnlistens(fullQuery)
          .userPatches('collection/a', { match: true })
          .userListens(filteredQuery)
          .expectEvents(filteredQuery, {
            added: [docAv2Local],
            fromCache: true,
            hasPendingWrites: true
          })

I believe this would work and it would remove the event in line 124 that is not directly related to your change. The reason I am no longer fully sold on this is that you would have to run the test with GC disabled to make this work. Otherwise, the SDK would delete docA from the RemoteDocumentCache as the user unlistens.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, can confirm what you described. Without GC disabled this is not working.

I am keep it as-is as that is slightly more likely what happens in real world.

// patch docA so that it will now match the filtered query.
.userPatches('collection/a', { match: true })
.expectEvents(fullQuery, {
modified: [docAv2Local],
hasPendingWrites: true
})
// Make sure docA shows up in filtered query.
.userListens(filteredQuery)
.expectEvents(filteredQuery, {
added: [docAv2Local],
fromCache: true,
hasPendingWrites: true
})
);
}
);
});