llama.cpp adds FP16 activation support for Qualcomm Hexagon inference

This edition covers the gap since the 08:00 UTC report. llama.cpp release b11276 expands its Qualcomm Hexagon backend with FP16 activation kernels, improving support for on-device model execution on Snapdragon-class hardware.

AI developments in this edition

Copy Markdown
  1. llama.cpp b11276 adds FP16 SILU, GELU, GEGLU and SWIGLU ops to Hexagon

    llama.cpp release b11276 adds F16 support for several activation operations in its ggml-hexagon HTP backend, including SILU, GELU, GELU_QUICK, GEGLU and SWIGLU. The project says these paths were verified on-device on a QRD8850 using CPU-diffed correctness tests; SWIGLU_OAI builds across the supported DSP variants but was not yet covered by an on-device F16 test in this change. The practical effect is broader native FP16 execution for local models using the Qualcomm Hexagon backend, reducing a compatibility gap for Snapdragon-based inference. The contributor also reported token-generation benchmarks for Llama 3.2 1B and Qwen2.5 models, but those are project-reported measurements rather than independent performance results.

Last checked

RSS