Skip to content

ggml : allocate graphs in a context #2392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 26, 2023
Merged

ggml : allocate graphs in a context #2392

merged 8 commits into from
Jul 26, 2023

Conversation

slaren
Copy link
Member

@slaren slaren commented Jul 25, 2023

  • Adds ggml_new_object to allocate objects in a context (internal to ggml)
  • Adds ggml_new_graph, ggml_build_forward_ctx, ggml_graph_overhead to support allocating graphs in a context
  • Modifies ggml_graph_compute_with_ctx to use ggml_new_object instead of a tensor
  • Modifies llama.cpp to allocate the eval graph on the compute context

Fixes ggml-org/ggml#299

slaren and others added 2 commits July 26, 2023 11:34
Co-authored-by: Georgi Gerganov <[email protected]>
@slaren slaren merged commit 5488fb7 into master Jul 26, 2023
@slaren slaren deleted the ggml-ctx-graph branch July 26, 2023 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ggml : improve API to allow allocating compute graphs on the heap
2 participants