In September 2024, the Department of Commerce’s Bureau of Industry and Security (BIS) released a proposed rule that would establish reporting requirements for entities developing advanced AI models or advanced computing clusters. They issued a public request for comments, inviting individuals and organizations to provide feedback and suggest improvements to the proposed rule.

Palisade Research submitted a comment, focusing on recommendations that could strengthen the reporting requirements for entities developing dual-use foundation models. We believe that AI capabilities are improving rapidly, and it’s essential for the US federal government to acquire information that allows it to prepare for AI-related threats to national security and public safety.

In our comment, we outline 5 core recommendations that can improve the effectiveness and robustness of the reporting requirements. We believe such recommendations could promote transparency into frontier AI development, enabling the government to better understand and prepare for potential safety and security threats. 

Our full response is available here.

Below, we present a summary of our five core recommendations:

  1. Establish a protected and/or anonymous reporting mechanism for employees at entities developing dual-use foundation models. Allow employees to submit concerns to ai_reporting@bis.doc.gov. Employees should be empowered to disclose: (a) any information about how the company might be inaccurate or misleading in its reports to BIS and (b) other information pertaining to the safety and reliability of dual-use foundation models, or activities or risks that present concerns regarding U.S. national security. Ideally, this platform would be both protected (entities developing dual-use foundation models would be prohibited from retaliating against individuals who use this platform for legitimate purposes) and anonymous. If making the reporting mechanism protected is not feasible, we believe an anonymous reporting mechanism would still provide substantial value. Entities should also affirm in their reports to BIS that (a) they have made employees aware of this reporting mechanism and (b) their policies (e.g., NDAs, non-disclosure agreements) will not prohibit, punish, or discourage employees from using this mechanism.

  2. Establish a regular interview program with employees at entities producing dual-use foundation models. BIS should conduct interviews on a quarterly basis with employees at companies developing dual-use foundation models. BIS would select these employees from a roster of employees, selecting individuals from multiple teams to get a diverse array of knowledge. In these interviews, BIS would ask employees to answer questions relating to dual-use foundation models, their capabilities, concerns regarding safety and security, and expectations about future progress in AI that could produce novel safety and security threats. For additional details about this proposal, see this paper.

  3. Require capability forecasts. In addition to requiring information about red-team testing, BIS should require companies to provide their best estimates of when they anticipate they or others will develop dual-use foundation models with certain kinds of security-relevant capabilities1. This would allow the US industrial base and defense establishment to make more informed predictions about future advances in AI systems and their implications for national defense.

  4. Require responses to a Summary Form that is legible to non-experts. In addition to requiring reports of red-team testing, we recommend that BIS require entities to submit a short Summary Form. The Summary Form would be accessible to non-technical audiences and highlight the most important defense-relevant information. We include example questions that could be asked in the full response.

  5. Amend the notification conditions such that entities must notify BIS of major capability improvements that pose imminent security risks. Some advances in AI capabilities may occur suddenly, and it may be essential for BIS to learn of these advances before the start of a new quarter. Therefore, we recommend that BIS require entities to report any major capability improvements that have imminent implications for national defense within 5 days.

Palisade’s full response is available here. If you have questions, feel free to contact us at policy@palisaderesearch.org.

  1. For example, those specified under the EO definition of dual-use foundation models: (1) Substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons; (2) Enabling powerful offensive cyber operations through automated vulnerability discovery and exploitation against a wide range of potential targets of cyberattacks; or (3) Permitting the evasion of human control or oversight through means of deception or obfuscation. ↩