Skip to content

fix context size check in sglang model #787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

xgwang
Copy link
Contributor

@xgwang xgwang commented May 29, 2025

  1. add exact the same logic as in vllm model. otherwise it will fail in sglang with confusing error
  2. it is duplicated now, shall we look into any improvement?

Copy link
Member

@clefourrier clefourrier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thanks for the fix!
I'm approving for now but I think @NathanHB 's working on a refacto of a number of legacy items in the code and hopefully we'll have a better system for this once it's merged

@HuggingFaceDocBuilderDev
Copy link
Collaborator

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@xgwang
Copy link
Contributor Author

xgwang commented Jun 6, 2025

Hi, thanks for the fix! I'm approving for now but I think @NathanHB 's working on a refacto of a number of legacy items in the code and hopefully we'll have a better system for this once it's merged

great to hear that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants