Scale AI has released DualChem, the first benchmark created under a partnership with the Center for Advancing Innovation and Standards for Super Intelligence at the National Institute of Standards and Technology. The standardized test contains 600 chemistry questions designed to evaluate how AI models handle dual-use scenarios and measure the assistance they provide to novice users. Evaluations of roughly 30 frontier models showed average accuracy of 50 percent, with top performers reaching about 67 percent. Researchers noted that models often continue to answer questions they identify as potentially risky, indicating a gap in understanding the relationship between risk and benefit levels.
Read the full story at fedscoop.com.
Photo: Republica / Pixabay
