Don't let the LLM speak, just probe it (blog.j11y.io)

A new method has emerged for efficiently probing large language models (LLMs) without relying on the slow and costly text generation process. By extracting the hidden state at the last token of the input prompt—typically around 70% of the way up the model's layers—researchers can use a small multi-layer perceptron (MLP) to classify responses based on criteria articulated in natural language. This approach bypasses traditional question-judging by leveraging the model’s internal geometry, where decisions are made during the forward pass, allowing for rapid and inexpensive evaluations. The significance of this technique lies in its capability to serve as a universal classifier using a single frozen model, with no need for criterion-specific training. This method not only retains the advanced understanding of modern LLMs but also enhances efficiency for tasks that involve complex structural questions, like discerning intent or tone in texts. By optimizing the way content is processed, the researchers propose improvements in evaluating multiple criteria on large datasets with far less computational overhead, promising a transformative impact on how LLMs can be deployed in analytic applications within the AI/ML community.