llama.cpp fixes ModernBERT reranker pooling and incorrect scores
One practical inference update landed in this hour: llama.cpp now preserves ModernBERT classifier pooling mode, fixing incorrect scores for rerankers that use CLS pooling instead of mean pooling.

AI developments in this edition
Copy Markdownllama.cpp adds classifier pooling support for ModernBERT rerankers
llama.cpp release b11273 adds support for the `classifier_pooling` setting used by ModernBERT rerankers. Previously, llama.cpp always mean-pooled token embeddings before the classification head; models configured for CLS pooling could convert and load without warning but return incorrect scores. The change writes the pooling mode into GGUF metadata and selects mean or CLS pooling at inference. In the contributor’s comparison, the max difference for IBM Granite’s CLS-pooled reranker fell from 1.94 to 0.23 versus Transformers, with most remaining error attributed to a separate activation mismatch. For local RAG and search stacks using llama.cpp reranking, this removes a silent correctness problem for a class of ModernBERT models.