Skip to content

Refinement of Providers into Providers and ExternProviders #2469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 40 additions & 33 deletions src/query.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,22 +71,27 @@ are cheaply cloneable; insert an `Rc` if necessary).

If, however, the query is *not* in the cache, then the compiler will
call the corresponding **provider** function. A provider is a function
implemented in a specific module and **manually registered** into the
[`Providers`][providers_struct] struct during compiler initialization.
The macro system generates the [`Providers`][providers_struct] struct,
which acts as a function table for all query implementations, where each
implemented in a specific module and **manually registered** into either
the [`Providers`][providers_struct] struct (for local crate queries) or
the [`ExternProviders`][extern_providers_struct] struct (for external crate queries)
during compiler initialization. The macro system generates both structs,
which act as function tables for all query implementations, where each
field is a function pointer to the actual provider.

**Note:** The `Providers` struct is generated by macros and acts as a function table for all query implementations.
It is **not** a Rust trait, but a plain struct with function pointer fields.
[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html
[extern_providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.ExternProviders.html

**Note:** Both the `Providers` and `ExternProviders` structs are generated by macros and act as function tables for all query implementations.
They are **not** Rust traits, but plain structs with function pointer fields.

**Providers are defined per-crate.** The compiler maintains,
internally, a table of providers for every crate, at least
conceptually. Right now, there are really two sets: the providers for
queries about the **local crate** (that is, the one being compiled)
and providers for queries about **external crates** (that is,
dependencies of the local crate). Note that what determines the crate
that a query is targeting is not the *kind* of query, but the *key*.
conceptually. There are two sets of providers:
- The `Providers` struct for queries about the **local crate** (that is, the one being compiled)
- The `ExternProviders` struct for queries about **external crates** (that is,
dependencies of the local crate)

Note that what determines the crate that a query is targeting is not the *kind* of query, but the *key*.
For example, when you invoke `tcx.type_of(def_id)`, that could be a
local query or an external query, depending on what crate the `def_id`
is referring to (see the [`self::keys::Key`][Key] trait for more
Expand Down Expand Up @@ -119,56 +124,58 @@ they define both a `provide` and a `provide_extern` function, through

### How providers are set up

When the tcx is created, it is given the providers by its creator using
the [`Providers`][providers_struct] struct. This struct is generated by
the macros here, but it is basically a big list of function pointers:

[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html
When the tcx is created, it is given both the local and external providers by its creator using
the `Providers` struct from `rustc_middle::util`. This struct contains both the local and external providers:

```rust,ignore
struct Providers {
type_of: for<'tcx> fn(TyCtxt<'tcx>, DefId) -> Ty<'tcx>,
// ... one field for each query
pub struct Providers {
pub queries: crate::query::Providers, // Local crate providers
pub extern_queries: crate::query::ExternProviders, // External crate providers
pub hooks: crate::hooks::Providers,
}
```

Each of these provider structs is generated by the macros and contains function pointers for their respective queries.

#### How are providers registered?

The `Providers` struct is filled in during compiler initialization, mainly by the `rustc_driver` crate.
The provider structs are filled in during compiler initialization, mainly by the `rustc_driver` crate.
But the actual provider functions are implemented in various `rustc_*` crates (like `rustc_middle`, `rustc_hir_analysis`, etc).

To register providers, each crate exposes a [`provide`][provide_fn] function that looks like this:

[provide_fn]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/fn.provide.html

```rust,ignore
pub fn provide(providers: &mut Providers) {
*providers = Providers {
type_of,
// ... add more providers here
..*providers
};
pub fn provide(providers: &mut rustc_middle::util::Providers) {
providers.queries.type_of = type_of;
// ... add more local providers here

providers.extern_queries.type_of = extern_type_of;
// ... add more external providers here

providers.hooks.some_hook = some_hook;
// ... add more hooks here
}
```

- This function takes a mutable reference to the `Providers` struct and sets the fields to point to the correct provider functions.
- You can also assign fields individually, e.g. `providers.type_of = type_of;`.
- You can assign fields individually for each provider type (local, external, and hooks).

#### Adding a new provider

Suppose you want to add a new query called `fubar`. You would:

1. Implement the provider function:
```rust,ignore
fn fubar<'tcx>(tcx: TyCtxt<'tcx>, key: DefId) -> Fubar<'tcx> { ... }
fn fubar<'tcx>(tcx: TyCtxt<'tcx>, key: LocalDefId) -> Fubar<'tcx> { ... }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and then make this a DefId

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tshepang suggest to make it LocalDefId. #2465 (comment)

I just went through the source code and realized that the key for local providers is pretty much LocalDefId. Which one are we going to choose?

```
2. Register it in the `provide` function:
```rust,ignore
pub fn provide(providers: &mut Providers) {
*providers = Providers {
fubar,
..*providers
};
pub fn provide(providers: &mut rustc_middle::util::Providers) {
providers.queries.fubar = fubar;
// If you need an external provider:
providers.extern_queries.fubar = extern_fubar;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not really helpful, I'd prefer to remove this and instead have a section on how the query system interacts with metadata for extern providers (you don't have to write that now, you can just remove the example)

}
```

Expand Down