AI Safety Evaluations Have Significant Limitations, Report Finds

Despite the increasing demand for AI safety and accountability, current tests and benchmarks may not be sufficient. Generative AI models, which produce text, images, music, and videos, are under scrutiny for their mistakes and unpredictability. Various organizations are proposing new benchmarks to ensure these models' safety.

Emerging Tools and Evaluations

Recently, Scale AI, NIST, and the U.K. AI Safety Institute have developed tools to assess model risks. However, a report by the Ada Lovelace Institute (ALI) suggests these evaluations, while useful, might be inadequate.

Insights from the Ada Lovelace Institute

ALI conducted a study involving interviews with experts from academic labs, civil society, and AI vendors, along with an audit of recent AI safety research. The study found that current evaluations are often non-exhaustive, easily manipulated, and do not necessarily reflect real-world performance.

Expert Opinions on AI Safety

Elliot Jones, senior researcher at ALI, emphasized that products in other sectors undergo rigorous testing to ensure safety before deployment. The research aimed to highlight the limitations of current AI safety evaluations and explore their use for policymakers and regulators.

Limitations and Challenges

The study revealed disagreements within the AI industry about the best evaluation methods. Some tests only assessed model alignment with lab benchmarks, not real-world impact. For example, while a model may perform well on a state bar exam, that doesn’t mean it’ll be able to solve more open-ended legal challenges. Data contamination, where benchmarks overestimate performance due to training on the same data, is also a concern.

Issues with Red-Teaming

“Red-teaming” involves tasking individuals or groups to attack a model to identify vulnerabilities and flaws. However, it lacks standardization, making its effectiveness hard to assess. The manual, costly nature of red-teaming poses challenges, especially for smaller organizations.

Industry Pressures

Pressure to release models quickly and reluctance to conduct thorough tests contribute to inadequate evaluations. A source mentioned that within companies, the push for rapid releases often overshadows the need for comprehensive evaluations.

Path Forward

ALI suggests increased public-sector involvement and clear articulation of evaluation requirements by regulators. Governments should mandate public participation in developing evaluations and support third-party testing with regular access to models and data.

Conclusion: The Quest for AI Safety

Safety is not an inherent property of models; it requires understanding the context of use, access, and adequacy of safeguards. Evaluations can identify risks but cannot guarantee complete safety. Many experts agree that evaluations can highlight potential dangers but cannot prove a model is entirely safe.