AI labs have based their reputations on model benchmarking, a system of evaluations designed to measure the performance of AI products. However, Irene Solaiman, Hugging Face’s chief policy officer, stated at TechCrunch Disrupt last week that the methods behind these evaluations are complex and still maturing.
Solaiman highlighted that a key aspect of trusting evaluation results is their reproducibility and the level of access to the model itself. Reproducibility in model evaluations remains a debated topic within the industry.
Furthermore, Solaiman noted that evaluations are an evolving science, and there is currently no clear method to meaningfully compare them to each other. She also mentioned that evaluations can become ineffective relatively quickly, and a more thriving ecosystem is needed.
The concept of embedded evaluators, where third-party observers are placed within AI organisations, has gained popularity. However, doubts persist regarding their neutrality and effectiveness, particularly following Anthropic's decision to hire Accenture as its first evaluator, which drew criticism.
Evaluations are now expanding beyond just the model to include the entire system, such as the harness and agent. Solaiman explained that a weaker model with a superior harness could outperform a more advanced model. She added that agentic evaluations are challenging and costly, with more evaluations available for models than for harnesses or holistic agents.