Skip to content

Releases: huggingface/text-generation-inference

v0.2.1

07 Feb 14:41
@OlivierDehaene OlivierDehaene
v0.2.1
2fe5e1b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.2.1

Fix

  • server: fix bug with repetition penalty when using GPUs and inference mode
Loading

v0.2.0

03 Feb 11:56
@OlivierDehaene OlivierDehaene
v0.2.0
20c3c59
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.2.0

Features

  • router: support Token streaming using Server Side Events
  • router: support seeding
  • server: support gpt-neox
  • server: support santacoder
  • server: support repetition penalty
  • server: allow the server to use a local weight cache

Breaking changes

  • router: refactor Token API
  • router: modify /generate API to only return generated text

Misc

  • router: use background task to manage request queue
  • ci: docker build/push on update
Loading