Salesforce Announces the World’s First LLM Benchmark for CRM

Salesforce today announced the world’s first LLM benchmark for CRM to help businesses evaluate the rapidly growing number of large language models (LLMs) for use in their customer relationship management (CRM) systems.

The new benchmark is a comprehensive evaluation framework that measures the performance of LLMs against four key measures: accuracy, cost, speed, and trust and safety. It’s been specifically designed to evaluate common sales and service use cases, including prospecting, lead nurturing, as well as sales opportunity and service case summaries. The benchmark also includes a public leaderboard to help professionals decide which LLM is best for their CRM needs. Salesforce will continue to incorporate new use case scenarios into the benchmark and enhance its evaluation of LLMs, which will soon include fine-tuned LLMs.

“As AI continues to evolve, enterprise leaders are saying it’s important to find the right mix of performance, accuracy, responsibility, and cost to unlock the full potential of generative AI to drive business growth,” said Silvio Savarese, EVP & Chief Scientist, Salesforce AI Research. “Salesforce’s new LLM Benchmark for CRM is a significant step forward in the way businesses assess their AI strategy within the industry. It not only provides clarity on next-generation AI deployment but also can accelerate time to value for CRM-specific use cases. Our commitment is to continuously evolve this benchmark to keep pace with technological advancements, ensuring it remains relevant and valuable.”