Search results

Support Shared Cache

Cache

Feature requestRequest for a new featureRequest for a new feature

Status: Open.

#35876 In huggingface/transformers;

· LoserCheems opened on Jan 24, 2025

LLaMa 3 8B - offloaded_static cache - layer_device_map TypeError

bug

Cache

Core: PipelineInternals of the library; Pipeline.Internals of the library; Pipeline.

WIPLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progressLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

Status: Open.

#34327 In huggingface/transformers;

· mikethebos opened on Oct 23, 2024

The same situation as #31377 occurred when using Qwen/Qwen2-VL-7B-Instruct

bug

Cache

Multimodal

Status: Open.

#33399 In huggingface/transformers;

· toondata opened on Sep 10, 2024

The _crop_past_key_values function should be a member function of Cache.

Cache

Feature requestRequest for a new featureRequest for a new feature

Status: Open.

#33378 In huggingface/transformers;

· jiqing-feng opened on Sep 9, 2024

Support <code>StaticCache</code> in assisted generation

Cache

Generation

Good Difficult Issue

Status: Open.

#32946 In huggingface/transformers;

· gante opened on Aug 22, 2024

static cache implementation is not compatible with attn_implementation==flash_attention_2

bug

Cache

Feature requestRequest for a new featureRequest for a new feature

Status: Open.

#32040 In huggingface/transformers;

· faaany opened on Jul 18, 2024

Skipping cudagraphs for unknown reason

Cache

CompilationIssues related to torchdynamo and torchinductorIssues related to torchdynamo and torchinductor

Feature requestRequest for a new featureRequest for a new feature

Good Second IssueIssues that are more difficult to do than "Good First" issues - give it a try if you want!Issues that are more difficult to do than "Good First" issues - give it a try if you want!

Status: Open.

#31645 In huggingface/transformers;

· hychiang-git opened on Jun 26, 2024

Problems when using SinkCache for model.generate()

Cache

Generation

WIPLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progressLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

Status: Open.

#31381 In huggingface/transformers;

· Zoeyyao27 opened on Jun 12, 2024

Implement kv cache sparsity like H2O with attention score

Cache

Feature requestRequest for a new featureRequest for a new feature

Status: Open.

#30758 In huggingface/transformers;

· HarryWu99 opened on May 11, 2024