Move JSON grammar -> regex grammar conversion to the router

[danieldk]

What does this PR do?

This change moves the JSON grammar -> regex grammar conversion to the router by adding a dependency on the outlines-core Rust crate. In contrast to the Python implementation, the conversions are not LRU-cached since they seem to be fast enough:

simple schema           time:   [5.8293 µs 5.8307 µs 5.8320 µs]
                        change: [-13.166% -12.884% -12.641%] (p = 0.00 < 0.05)
                        Performance has improved.

complex schema          time:   [14.875 µs 14.881 µs 14.887 µs]
                        change: [-2.1637% -1.9914% -1.7852%] (p = 0.00 < 0.05)
                        Performance has improved.

Using the schemas from:
https://github.com/dottxt-ai/outlines-core/blob/main/benchmarks/bench_json_schema.py

Some additional remarks:

• For now, I haven't removed the JSON type from the v3 protobuf for compatibility. Instead, an error emitted and the grammar is ignored.
• Another behavioral change is that when a grammar is invalid, the request is rejected with an error, rather than continuing with the request with a .* grammar.
• The Git dependency is necessary because the latest outlines-core is not in crates.io yet, @ErikKaum recommended to use the latest version from Git.
• The regex -> automaton compilation has not been moved to the router yet, since it hasn't been ported to Python yet.
• Added some type annotations while learning about the code.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[ErikKaum]

Awesome 🔥

I had a quick look and it seems to be good 👍 I haven't run it locally thou (yet).

For reference, this was an error we couldn't catch before, that we now should be able to capture: #2280

[OlivierDehaene]

If you can update the snapshots this looks good!

[drbh]

LGTM! great improvement!

*just need some small changes to make CI happy