Skip to content

llama : add llama_lora_adapter_clear #8653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 24, 2024

Conversation

ngxson
Copy link
Collaborator

@ngxson ngxson commented Jul 23, 2024

Ref discussion: #8636 (comment)

llama_lora_adapter_clear() can be used when user want to switch the adapter, but don't know which adapters are loaded into llama_context to be removed.

For simple task switching application, user can write their own code based on this function:

void switch_my_adapter(struct llama_context * ctx, struct llama_lora_adapter * adapter) {
  llama_lora_adapter_clear(ctx);
  llama_lora_adapter_set(ctx, adapter, 1.0);
}

@ngxson ngxson requested a review from ggerganov July 23, 2024 17:45
@ngxson ngxson mentioned this pull request Jul 23, 2024
4 tasks
@zhipenghan
Copy link

Thank you for implementing this change. Now, users can manage adaptors outside of the library as if the existing adaptors were invisible.

@ngxson ngxson merged commit b115105 into ggml-org:master Jul 24, 2024
53 checks passed
@ngxson
Copy link
Collaborator Author

ngxson commented Jul 24, 2024

Oops sorry I didn't modify the merge commit message to specify module name

arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants