MLCommons has enhanced AILuminate LLM v1.1, adding French language capabilities to industry-leading AI safety benchmarks.
Takeaway Points
- MLCommons has enhanced AILuminate LLM v1.1, adding French language capabilities to industry-leading AI safety benchmarks.
- This was announced at the Paris AI Action Summit.
- The AILuminate benchmark was developed by the MLCommons AI Risk and Reliability working group.
MLCommons AILuminate LLM v1.1
MLCommons, in partnership with the AI Verify Foundation, said on Monday that it has released v1.1 of AILuminate, incorporating new French language capabilities into its first-of-its-kind AI safety benchmark. The new update, which was announced at the Paris AI Action Summit, marks the next step towards a global standard for AI safety and comes as AI purchasers across the globe seek to evaluate and limit product risk in an emerging regulatory landscape.
Rebecca Weiss, Executive Director of MLCommons, said, “Companies around the world are increasingly incorporating AI in their products, but they have no common, trusted means of comparing model risk. By expanding AILuminate’s language capabilities, we are ensuring that global AI developers and purchasers have access to the type of independent, rigorous benchmarking proven to reduce product risk and increase industry safety.”
Peter Mattson, Founder and President of MLCommons, Commented, “Building safe and reliable AI is a global problem – and we all have an interest in coordinating on our approach. Today’s release marks our commitment to championing a solution to AI safety that’s global by design and is a first step toward evaluating safety concerns across diverse languages, cultures, and value systems.”
Other comment
Nicolas Miailhe, Founder and CEO of PRISM Eval, said, “MLCommons’ work in pushing the industry toward a global safety standard is more important now than ever. PRISM is proud to support this work with our latest Behavior Elicitation Technology (BET), and we look forward to continuing to collaborate on this important trustbuilding effort – in France and beyond.”
The AILuminate Developers
According to the report, The AILuminate benchmark was developed by the MLCommons AI Risk and Reliability working group, a team of leading AI researchers from institutions including Stanford University, Columbia University, and TU Eindhoven, civil society representatives, and technical experts from Google, Intel, NVIDIA, Microsoft, Qualcomm Technologies, Inc., and other industry giants committed to a standardized approach to AI safety.
Availability
The AILuminate is Currently available in English and French, and will be made available in Chinese and Hindi later this year, the company said.
About MLCommons
MLCommons is the world’s leader in AI benchmarking. An open engineering consortium supported by over 125 members and affiliates, MLCommons has a proven record of bringing together academic, industry, and civil society to measure and improve AI. The foundation for MLCommons began with the MLPerf benchmarks in 2018, which rapidly scaled as a set of industry metrics to measure machine learning performance and promote transparency of machine learning techniques. Since then, MLCommons has continued to use collective engineering to build the benchmarks and metrics required for better AI – ultimately helping to evaluate and improve the accuracy, safety, speed, and efficiency of AI technologies.
