The operator of PatronView, a 1.5 million-page database of American philanthropists, said on August 7, 2026, that automated traffic accounted for more than 99 percent of pages served during a measured week. The first-person account documents a year of attempts to identify and limit crawlers while preserving access for actual visitors.
During the cited week, the server answered 2.5 million external requests and delivered 1.28 million full pages. Browser analytics recorded only 5,977 pageviews, or roughly one observed human view for every 214 pages served. The comparison also illustrates why JavaScript analytics can understate server load: many bots never execute the tracking code.
PatronView compiles profiles from public IRS filings, donor walls and annual reports. Its owner acknowledges scraping those sources periodically, while distinguishing that collection from crawlers repeatedly sweeping the full site. Search-optimization bots were initially blocked through `robots.txt` and Cloudflare firewall rules.
The traffic later became more distributed. In November 2025, about 4,000 apparent visitors each loaded one fundraising page without a referrer. On April 22, the site received 3.6 million requests from 361,844 IP addresses, nearly all geolocated to China. Cloudflare’s managed challenge handled 1.18 million requests in ten hours, but many clients passed it. The operator then blocked China, followed by Vietnam and Singapore after similar traffic appeared.
Those country-wide blocks are a blunt response with obvious risk to legitimate users. The author justified them by PatronView’s audience: 95.9 percent of real search traffic came from the United States and 1.3 percent from Canada. The figures and classifications come from one operator’s logs and analytics, not an independent audit, and an untracked request is not automatically proven malicious.
AI search crawlers presented a measurable cost-benefit problem. In one week, Anthropic’s Claude-SearchBot requested 420,680 pages while Claude-User, used when a person asks the service to fetch a page, corresponded to 12 visitors. The author calculated 35,000 pages crawled for each referral and reported serving 4.63 gigabytes to the crawler versus 175 kilobytes to referred users.
After PatronView blocked Claude-SearchBot at the firewall, requests fell from roughly 60,000 a day to about 25 attempts, suggesting that crawler respected the denial. The operator adopted pages-crawled-per-visitor-referred as a comparison metric, judging Google worthwhile and Bing less efficient but defensible.
The experience shows that bot management is not one filter. It combines server logs, analytics, rate patterns, geography, user agents, firewall responses and an operator’s tolerance for collateral blocking. PatronView’s numbers cannot represent the whole web, but they expose the infrastructure burden that a modest human audience can hide. Server-side measurement remains essential.
