A new report from edge cloud platform provider Fastly suggests AI crawlers made up almost 80% of all AI bot traffic in recent months – with crawlers from Meta making up more than half of total traffic.

Fastly’s Q2 2025 Threat Insights Report, which covered AI bot traffic from mid-April to mid-July, showed crawlers – a type of bot that gathers data from across the web to AI models – making up the majority of traffic.

AI crawlers for Meta, Google, and OpenAI accounted for a combined 95% of all AI crawler request volume between April and July. But it was Meta that led the pack, accounting for double (52%) the amount of traffic that Google (23%) and OpenAI (20%) each were responsible for.

While crawler bots operate similarly to search engine crawlers – in that they systematically scan sites to collect content – the practice of using crawlers for AI has become increasingly controversial in the wake of early days of generative AI development, where model developers willfully scrape every corner of the internet for training data for content, often without rights holder permissions.

According to Fastly’s report, Meta’s crawler activity appears to be increasing, with traffic demand over the analyzed time period showing an upward trend, which goes against the norm for such bots.

“Most crawlers exhibit irregular patterns of activity, with extended periods of moderate or low request volumes interspersed with sustained increases lasting days or even weeks, often around 2–3 times the baseline rate,” the report reads. “The only consistent exception is Common Crawl’s CCBot, which follows a more regular and predictable crawling schedule.”

Fetcher bots, meanwhile, which access content in response to user queries, made up just 20% of AI bot traffic. Fetcher bots are employed by AI apps that have web search functionality, like ChatGPT and Perplexity, which, according to Fastly, is driving massive real-time request volumes.

In some instances, fetcher bot request volume exceeded 39,000 requests per minute, placing huge pressure on unprotected origin infrastructure and overwhelming servers – and in some cases mimicking the effects of distributed denial-of-service (DDoS) attacks without the malicious intent.

North America was found to have received the heaviest skew of AI crawler traffic, accounting for nearly 90% of observed activity by Fastly. Every other region, including Europe, Asia, and Latin America, saw a significantly smaller share. Automated bot traffic as a whole represented 37% of observed activity across Fastly’s global network during Q2.

“AI bots are reshaping how the internet is accessed and experienced, introducing new complexities for digital platforms,” said Arun Kumar, senior security researcher at Fastly.

“Whether scraping for training data or delivering real-time responses, these bots create new challenges for visibility, control, and cost. You can’t secure what you can’t see, and without clear verification standards, AI-driven automation risks are becoming a blind spot for digital teams. Businesses need the tools and insights to manage automated traffic with the same precision and urgency as any other infrastructure or security risk.”

AI bots are becoming an increasingly top-of-the-line issue in the internet world, with platform owners disgruntled about their content being taken without permission.

In the past month alone, Reddit blocked the Wayback Machine – an internet archive tool used to archive older websites and pages for preservation – from accessing the platform over claims AI model developers were using it to access its data to build out their training datasets. And Cloudflare is in the midst of a highly publicized spat with Perplexity, the SoftBank-backed AI search app, over claims the latter uses “undeclared crawlers” to get around the provider’s AI scraping restrictions, an allegation Perplexity disputes.