Tags: jeffbolznv/llama.cpp
Tags
convert : refactor rope scaling handling (ggml-org#18013) * refactor rope scaling handling * ws-- * missed a couple * use find_hparam
ggml-alloc : fix reuse-parent logic for misaligned sizes (ggml-org#17884)
metal: SSM kernel improvements (ggml-org#17876) * feat: Add a batched version of ssm_conv This was done using Claude Code. It found a number of optimizations around how the threads were organized, resulting in a huge performance boost! Branch: Mamba2SSD Signed-off-by: Gabe Goodhart <[email protected]> * feat: Optimized SSM_SCAN kernel for metal This used Claude Code and resulted in a modest performance improvement while maintaining correctness. Branch: Mamba2SSD Signed-off-by: Gabe Goodhart <[email protected]> * test: Add test-backend-ops perf tests for SSM_CONV Branch: SSMKernelImprovements Signed-off-by: Gabe Goodhart <[email protected]> * test: Real representitive tests for SSM_CONV Branch: SSMKernelImprovements Signed-off-by: Gabe Goodhart <[email protected]> * refactor: Use function constant for ssm_conv batch size Branch: SSMKernelImprovements Signed-off-by: Gabe Goodhart <[email protected]> * test: backend op tests for ssm_scan from granite4 1b-h Branch: SSMKernelImprovements Signed-off-by: Gabe Goodhart <[email protected]> * style: remove commented out templates Branch: SSMKernelImprovements Signed-off-by: Gabe Goodhart <[email protected]> * feat: float4 version of ssm_conv_batched Branch: SSMKernelImprovements Signed-off-by: Gabe Goodhart <[email protected]> * fix: Add missing ggml_metal_cv_free Signed-off-by: Gabe Goodhart <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]> --------- Signed-off-by: Gabe Goodhart <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]>
ggml-cpu: remove duplicate conditional check 'iid' (ggml-org#17650)
vulkan: Reduce temporary memory usage for TOP_K (ggml-org#17623) - Compute row size for the temp buffer based on the output of the first pass. - Update shader addressing math to use the output row size - Pass the output row size as "ncols_output", what used to be "ncols_output" is now "k" For the common case of K=40 and src0=(200000,1,1,1), this reduces the temporary buffer from about 3.2MB to 500KB.
PreviousNext