Paris – February 11, 2025: MLCommons, in partnership with the AI Confirm Basis, immediately launched v1.1 of AILuminate, incorporating new French language capabilities into its first-of-its-kind AI security benchmark.
The brand new replace – which was introduced on the Paris AI Motion Summit – marks the subsequent step in the direction of a world customary for AI security and comes as AI purchasers throughout the globe search to guage and restrict product danger in an rising regulatory panorama.
Like its v1.0 predecessor, the French LLM model 1.1 was developed collaboratively by AI researchers and business specialists, guaranteeing a trusted, rigorous evaluation of chatbot danger that may be instantly integrated into firm decision-making.
“Corporations all over the world are more and more incorporating AI of their merchandise, however they don’t have any frequent, trusted technique of evaluating mannequin danger,” mentioned Rebecca Weiss, Government Director of MLCommons. “By increasing AILuminate’s language capabilities, we’re guaranteeing that international AI builders and purchasers have entry to the kind of unbiased, rigorous benchmarking confirmed to scale back product danger and enhance business security.”
Just like the English v1.0, the v1.1 French mannequin of AILuminiate assesses LLM responses to over 24,000 French language check prompts throughout twelve classes of hazards behaviors – together with violent crime, hate, and privateness. In contrast to a lot of peer benchmarks, not one of the LLMs evaluated are given advance entry to particular analysis prompts or the evaluator mannequin. This ensures a methodological rigor unusual in customary educational analysis and an empirical evaluation that may be trusted by business and academia alike.
“Constructing protected and dependable AI is a world downside – and all of us have an curiosity in coordinating on our method,” mentioned Peter Mattson, Founder and President of MLCommons. “At the moment’s launch marks our dedication to championing an answer to AI security that’s international by design and is a primary step towards evaluating security issues throughout numerous languages, cultures, and worth programs.”
The AILuminate benchmark was developed by the MLCommons AI Risk and Reliability working group, a workforce of main AI researchers from establishments together with Stanford College, Columbia College, and TU Eindhoven, civil society representatives, and technical specialists from Google, Intel, NVIDIA, Microsoft, Qualcomm Applied sciences, Inc., and different business giants dedicated to a standardized method to AI security. Cognizant that AI security requires a coordinated international method, MLCommons additionally collaborated with worldwide organizations such because the AI Confirm Basis to design the AILuminate benchmark.
“MLCommons’ work in pushing the business towards a world security customary is extra essential now than ever,” mentioned Nicolas Miailhe, Founder and CEO of PRISM Eval. “PRISM is proud to help this work with our newest Conduct Elicitation Know-how (BET), and we look ahead to persevering with to collaborate on this essential trustbuilding effort – in France and past.”
At present obtainable in English and French, AILuminate might be made obtainable in Chinese language and Hindi later this 12 months.
