Skip to content

Support descending queries #6075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 28, 2022
Merged

Support descending queries #6075

merged 3 commits into from
Mar 28, 2022

Conversation

schmidt-sebastian
Copy link
Contributor

@schmidt-sebastian schmidt-sebastian commented Mar 16, 2022

This adds code to order all query results by document key, which is needed to return the correct result for limit queries.

Port of firebase/firebase-android-sdk#3499

@changeset-bot
Copy link

changeset-bot bot commented Mar 16, 2022

⚠️ No Changeset found

Latest commit: 09a3ca0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Mar 16, 2022

Size Report 1

Affected Products

  • @firebase/app

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser13.8 kB13.8 kB+1 B (+0.0%)
    esm518.0 kB18.0 kB+1 B (+0.0%)
    main18.9 kB18.9 kB+1 B (+0.0%)
    module13.8 kB13.8 kB+1 B (+0.0%)
  • @firebase/auth-compat

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser20.1 kB20.1 kB+1 B (+0.0%)
    esm526.9 kB26.9 kB+1 B (+0.0%)
    main29.5 kB29.5 kB+1 B (+0.0%)
    module20.1 kB20.1 kB+1 B (+0.0%)
  • @firebase/auth/cordova

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser180 kB180 kB+1 B (+0.0%)
    module180 kB180 kB+1 B (+0.0%)
  • @firebase/auth/internal

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser164 kB164 kB+1 B (+0.0%)
    esm5213 kB213 kB+1 B (+0.0%)
    main180 kB180 kB+1 B (+0.0%)
    module164 kB164 kB+1 B (+0.0%)
  • @firebase/auth/react-native

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser165 kB165 kB+1 B (+0.0%)
    module165 kB165 kB+1 B (+0.0%)
  • @firebase/firestore

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser248 kB252 kB+3.30 kB (+1.3%)
    esm5309 kB313 kB+3.52 kB (+1.1%)
    main498 kB503 kB+5.45 kB (+1.1%)
    module248 kB252 kB+3.30 kB (+1.3%)
    react-native249 kB252 kB+3.30 kB (+1.3%)
  • @firebase/firestore-lite

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser73.1 kB73.2 kB+60 B (+0.1%)
    esm586.5 kB86.6 kB+64 B (+0.1%)
    main126 kB126 kB+76 B (+0.1%)
    module73.1 kB73.2 kB+60 B (+0.1%)
    react-native73.3 kB73.4 kB+60 B (+0.1%)
  • @firebase/functions

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser8.99 kB8.99 kB+1 B (+0.0%)
    esm511.1 kB11.1 kB+1 B (+0.0%)
    main11.8 kB11.8 kB+1 B (+0.0%)
    module8.99 kB8.99 kB+1 B (+0.0%)
  • @firebase/functions-compat

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser1.67 kB1.68 kB+1 B (+0.1%)
    esm51.83 kB1.83 kB+1 B (+0.1%)
    main2.20 kB2.20 kB+1 B (+0.0%)
    module1.67 kB1.68 kB+1 B (+0.1%)
  • @firebase/messaging

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser20.8 kB20.8 kB+1 B (+0.0%)
    esm526.2 kB26.2 kB+1 B (+0.0%)
    main26.8 kB26.8 kB+1 B (+0.0%)
    module20.8 kB20.8 kB+1 B (+0.0%)
  • @firebase/messaging-compat

    TypeBase (7405e7d)Merge (57d637a)Diff
    browser2.08 kB2.08 kB+1 B (+0.0%)
    esm52.51 kB2.51 kB+1 B (+0.0%)
    main2.90 kB2.90 kB+1 B (+0.0%)
    module2.08 kB2.08 kB+1 B (+0.0%)
  • bundle

    43 size changes

    TypeBase (7405e7d)Merge (57d637a)Diff
    analytics (logEvent)40.0 kB40.0 kB+1 B (+0.0%)
    app-check (CustomProvider)33.7 kB33.7 kB+1 B (+0.0%)
    app-check (ReCaptchaEnterpriseProvider)35.9 kB35.9 kB+1 B (+0.0%)
    app-check (ReCaptchaV3Provider)35.8 kB35.8 kB+1 B (+0.0%)
    auth (Anonymous)63.5 kB63.5 kB+3 B (+0.0%)
    auth (EmailAndPassword)67.6 kB67.6 kB+3 B (+0.0%)
    auth (GoogleFBTwitterGitHubPopup)87.3 kB87.3 kB+3 B (+0.0%)
    auth (GooglePopup)87.1 kB87.1 kB+3 B (+0.0%)
    auth (GoogleRedirect)87.3 kB87.3 kB+3 B (+0.0%)
    auth (Phone)73.6 kB73.6 kB+3 B (+0.0%)
    database (Append to a list of data)143 kB143 kB+2 B (+0.0%)
    database (Filtering data)142 kB142 kB+2 B (+0.0%)
    database (Listen for child events)158 kB158 kB+2 B (+0.0%)
    database (Listen for value events + Detach listeners)158 kB158 kB+2 B (+0.0%)
    database (Listen for value events)158 kB158 kB+2 B (+0.0%)
    database (Read data once)150 kB150 kB+2 B (+0.0%)
    database (Save data as transactions)160 kB160 kB+2 B (+0.0%)
    database (Sort data)144 kB144 kB+2 B (+0.0%)
    database (Write data)143 kB143 kB+2 B (+0.0%)
    firestore (Persistence)259 kB262 kB+3.12 kB (+1.2%)
    firestore (Query Cursors)201 kB202 kB+929 B (+0.5%)
    firestore (Query)202 kB203 kB+929 B (+0.5%)
    firestore (Read data once)191 kB192 kB+929 B (+0.5%)
    firestore (Realtime updates)193 kB194 kB+929 B (+0.5%)
    firestore (Transaction)176 kB176 kB+692 B (+0.4%)
    firestore (Write data)175 kB176 kB+692 B (+0.4%)
    firestore-lite (Query Cursors)66.3 kB66.3 kB+62 B (+0.1%)
    firestore-lite (Query)69.3 kB69.4 kB+62 B (+0.1%)
    firestore-lite (Read data once)53.8 kB53.8 kB+62 B (+0.1%)
    firestore-lite (Transaction)71.1 kB71.2 kB+62 B (+0.1%)
    firestore-lite (Write data)56.6 kB56.6 kB+3 B (+0.0%)
    functions (call)27.6 kB27.6 kB+2 B (+0.0%)
    messaging (send + receive)43.3 kB43.3 kB+2 B (+0.0%)
    performance (trace)47.7 kB47.7 kB+1 B (+0.0%)
    remote-config (getAndFetch)42.3 kB42.3 kB+2 B (+0.0%)
    storage (getBytes)35.9 kB35.9 kB+2 B (+0.0%)
    storage (getDownloadURL)38.0 kB38.0 kB+2 B (+0.0%)
    storage (getMetadata)37.4 kB37.4 kB+2 B (+0.0%)
    storage (list + listAll)36.9 kB36.9 kB+2 B (+0.0%)
    storage (updateMetadata)37.7 kB37.7 kB+2 B (+0.0%)
    storage (uploadBytes)42.2 kB42.2 kB+2 B (+0.0%)
    storage (uploadBytesResumable)51.7 kB51.7 kB+2 B (+0.0%)
    storage (uploadString)42.5 kB42.5 kB+2 B (+0.0%)

  • firebase

    23 size changes

    TypeBase (7405e7d)Merge (57d637a)Diff
    firebase-analytics.js105 kB105 kB+147 B (+0.1%)
    firebase-app-check.js90.0 kB90.1 kB+4 B (+0.0%)
    firebase-app-compat.js26.4 kB26.4 kB+2 B (+0.0%)
    firebase-app.js81.4 kB81.5 kB+152 B (+0.2%)
    firebase-auth-compat.js123 kB123 kB+2 B (+0.0%)
    firebase-auth-cordova.js463 kB463 kB+2 B (+0.0%)
    firebase-auth-react-native.js491 kB492 kB+170 B (+0.0%)
    firebase-auth.js412 kB412 kB+3 B (+0.0%)
    firebase-compat.js775 kB778 kB+3.17 kB (+0.4%)
    firebase-database.js603 kB603 kB+4 B (+0.0%)
    firebase-firestore-compat.js300 kB303 kB+3.16 kB (+1.1%)
    firebase-firestore-lite.js250 kB250 kB+87 B (+0.0%)
    firebase-firestore.js818 kB820 kB+2.55 kB (+0.3%)
    firebase-functions-compat.js7.95 kB7.95 kB+2 B (+0.0%)
    firebase-functions.js31.1 kB31.1 kB+3 B (+0.0%)
    firebase-messaging-compat.js36.4 kB36.4 kB+3 B (+0.0%)
    firebase-messaging-sw.js101 kB101 kB+169 B (+0.2%)
    firebase-messaging.js99.2 kB99.4 kB+170 B (+0.2%)
    firebase-performance-standalone-compat.es2017.js86.5 kB86.5 kB+3 B (+0.0%)
    firebase-performance-standalone-compat.js64.0 kB64.0 kB+3 B (+0.0%)
    firebase-performance.js117 kB117 kB+147 B (+0.1%)
    firebase-remote-config.js106 kB106 kB+146 B (+0.1%)
    firebase-storage.js146 kB146 kB+1 B (+0.0%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/oUo4t2zV67.html

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Mar 16, 2022

Size Analysis Report 1

This report is too large (746,042 characters) to be displayed here in a GitHub comment. Please use the below link to see the full report on Google Cloud Storage.

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/BH4PZ6Xif8.html

Copy link
Contributor

@ehsannas ehsannas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some index manager test failures.

}
break;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should port this change (adding a break statement) to the Android SDK.

segmentValue = filterValue;
segmentInclusive = filterInclusive;
}
function targetGetAscendingBound(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comments about what the function does.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for targetGetDescendingBound

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

});
});
}
);
}).next(() => result as DocumentKeySet | null);
}).next(() => result as DocumentKey[] | null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is as DocumentKey[] needed here? result is already of type DocumentKey[]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are casting to DocumentKey[] | null. This means that the function returns PersistencePromise<DocumentKey[] | null> rather than PersistencePromise<DocumentKey[]> | PersistencePromise<null>, which is unfortunately a differen type.

@@ -79,6 +79,18 @@ export function fieldIndexGetDirectionalSegments(
return fieldIndex.fields.filter(s => s.kind !== IndexKind.CONTAINS);
}

/**
* Returns the order of the document key component.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returns the order of the document key component for the given index

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -906,15 +935,17 @@ export class IndexedDbIndexManager implements IndexManager {
this.uid,
bounds[i].arrayValue,
bounds[i].directionalValue,
new Uint8Array(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing the encoding of an empty string is an empty byte array

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OrderedCodeWriter writes bytes, so this is the byte array that sorts before any other byte arrays.

* The document key this entry points to. This entry is encoded by an ordered
* encoder to match the key order of the index.
*/
orderedDocumentKey: Uint8Array;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it true that if the order was always ASC, then using EncodedResourcePath below would have sufficed? in order words: is the only reason we need this is because of DESC indexes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. EncodedResourcePath already does some gymnastics to make sure that paths order naturally, even if there the slashes would yield our of order results normally. With EncodedResourcePath, we get: a/b, a/c, aa/c instead of a/b, aa/b, a/c for example (example-ish, since the sorting is off here too)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I changed the documentKey entry to no longer use EncodedResourcePath since we no longer rely on any ordering.

@github-actions
Copy link
Contributor

Changeset File Check ⚠️

  • Changeset formatting error in following file:
    Some packages have been changed but no changesets were found. Run `changeset add` to resolve this error.
    If this change doesn't need a release, run `changeset add --empty`.
    

Copy link
Contributor

@ehsannas ehsannas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still seeing these two test failures:

  1. Target Bounds
    less than query:

  2. Target Bounds
    greater than or equals query:

@schmidt-sebastian
Copy link
Contributor Author

I fixed the test failures. Sorry for not catching them earlier. I think we should write better test helpers to reduce the repetitiveness of the bounds tests, but I can't get myself to do that on my current setup.

@schmidt-sebastian schmidt-sebastian merged commit 46fe136 into master Mar 28, 2022
@schmidt-sebastian schmidt-sebastian deleted the mrschmidt/descorder branch March 28, 2022 16:53
@firebase firebase locked and limited conversation to collaborators Apr 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants