The Biden Administration’s new Executive Order on Safe, Secure and Trustworthy Artificial Intelligence is a landmark presidential directive in addressing a powerful technology that evolves by the day.

As with any government action, response to the comprehensive 100-plus-page document has been all over the board, from scathing to praiseworthy.

Experts across industries raise concerns around competition, impacts to open-source projects, the potential dampening of innovation, cybersecurity ramifications, how rules will play out across value chains (not just technology developers themselves) and much more.

At the same time, many agree we are at a pivotal moment in history and it’s clear the administration intends to move quickly.

“Biden is starting from a favorable position: Even most AI business leaders agree that some regulation is necessary,” said Jaysen Gillespie, head of analytics and data science at deep learning company RTB House. “Given the context behind his Executive Order, the President has a real opportunity to establish leadership on what may be the most important topic of this century.”

Wide-sweeping action around AI

In the order, Biden requires developers of AI systems to do the following:

• Share their safety test results and other critical information with the U.S. government.
• Develop standards, tools and tests to help ensure AI systems are safe, secure and trustworthy.
• Establish standards and best practices for detecting AI-generated content and authenticating official content.
• Build an advanced cybersecurity program to develop AI tools to find and fix vulnerabilities in critical software.

The President also calls on Congress to pass bipartisan data privacy legislation and orders action around the following:

• Accelerating the development and use of privacy-preserving techniques.
• Strengthening privacy-preserving research and technologies.
• Evaluating how agencies collect and use commercially available information.
• Developing guidelines for federal agencies to evaluate the effectiveness of privacy-preserving techniques.

The sweeping order further gives directives around advancing equity and civil rights, consumer and worker protection, responsible government use of AI and promoting innovation and competition.

To emphasize its commitment to responsible, safe AI, the administration has also rolled out a new AI.gov website.

Substance of testing, enforcing standards

Many concerns begin with the foundational testing requirement, which is up for interpretation.

“What will be interesting will be what these tests will require, what results will be required and what AI companies fall under this requirement,” said attorney Colin S. Levy.

Similarly, the order lacks mention of any specific implementation and enforcement mechanisms — so while it proposes a vision for how AI should be regulated, it doesn’t have much to legally back those measures, he noted.

“Of course, it is good to see the executive order address both risks and benefits of AI, but both of those will shift as AI continues to rapidly advance,” said Levy.

Paul Barrett, deputy director of the NYU Stern Center for Business and Human Rights, agreed that training and red teaming of models “makes good sense.”

However, it's not clear that the federal government has the resources to assess the “vastly complicated” training process or the adequacy of red-teaming and other necessary testing, he said. And how will standards be enforced?

Also, instructing the use of content-authentication tools is important, but in many cases, such tools are not yet effective for detecting AI-generated content.

In the end, though, “President Biden is sending a valuable message that certain AI systems create immediate risks that demand immediate attention,” said Barrett.

What AI systems pose risk?

Others question how we ultimately determine whether AI systems pose a serious risk to national security and public health and safety.

Jeff Williams, cofounder and CTO of Contrast Security, pointed out that “almost any AI” could flood critical agencies with requests indistinguishable from human ones.

“The opportunities to undermine national security are endless,” he said.

Also, the “rigorous standards” for red-team safety testing will be a very difficult challenge, he noted, adding that the government should include guidance from the Open Worldwide Application Security Project (OWASP).

Williams further lamented that there are scant details on AI transparency and explainable AI.

“Consumers need to understand and interpret the predictions made by ML models,” he said. “They have the right to know about the software and models they are trusting with the most important things in their life.”

But it’s a noble effort. “I’m impressed that the White House has stepped in relatively quickly to address AI threats,” Williams said. “Historically, the reaction from government has been too weak and years too late to make a difference.”

Lofty cybersecurity goals

Experts also contend that Biden’s cybersecurity orders are lofty, particularly when it comes to developing AI tools to find and fix vulnerabilities in critical software.

While watermarking of AI-generated content is possible — via embedded patterns and metadata, for instance — threat actors can bypass these controls, said David Brauchler, principal security consultant at NCC Group.

There is currently no meaningful way to prevent AI content from “masquerading as human-created content,” he said.

Brauchler added that AI has substantial potential to increase the effectiveness of security controls in code review, active monitoring and vulnerability detection, but “there is still a long road before this technology will become viable.”

Williams of Contrast agreed, contending that the White House is “grasping at straws to think that AI will magically solve cybersecurity issues in the near term.”

For known vulnerabilities and attacks, there are techniques with better assurance and capability than that offered by AI, he pointed out. For unknown threats, meanwhile, AI is “not a particularly good match,” since there is no training data on zero-days.

“So why do we think that AI can help us here?” Williams posited.

Responsible AI more critical than ever

Other cybersecurity experts praised the order for its emphasis on responsible AI innovation.

AI has been “transformative” for modern technology and recent developments have lowered the barrier for both innovators and adversaries, noted Drew Bagley, VP and counsel for privacy and cyber policy at leading cybersecurity software company CrowdStrike.

Defenders rely on AI to detect and prevent cyberattacks at scale in an era dominated by malware-free attacks and zero-day exploits, he pointed out. At the same time, attackers are increasingly using large language models (LLMs) to move more quickly and scale operations.

“Ultimately, it’s critical that AI can and should be leveraged in a responsible way,” said Bagley.

The natural language interface of LLMs can make cybersecurity roles and responsibilities more broadly accessible, helping to close the cybersecurity skills gap and improve response times, “boosting proactive security.”

“This is why investing in responsible AI innovation is more critical than ever,” said Bagley.

Fostering innovation

In the end, experts caution that the order should not stifle innovation.

“It’s clear the Biden administration is choosing to 'go big' on AI and pursue an unprecedented whole of government approach on this issue,” said Matthew Mittelsteadt, research fellow with the Mercatus Center, a nonprofit think tank.

The Order’s ultimate impact will depend on how the rules interact with the industry and existing regulations, he said. When follow-up rules are crafted, agencies should be wary of “regulatory cudgels that might depress AI adoption or bind the hands of the developers.”

Of equal importance, the order should not race to create new regulation that slows down innovation and requires official certification, “treating AIs today like weapons or pharmaceuticals,” said Kevin Bocek, VP of ecosystem and community at cybersecurity company Venafi.

“We need to promote research and innovation to achieve outcomes of standards, security and safety instead of racing to apply rules and regulations from the last century,” he said.