Who Is the Web For When Bots Do the Browsing?

Bots are the butt of many jokes today and an easy icebreaker for a web discussion that is serious, slightly chaotic, and unwilling to pretend anyone has it figured out. Of course, beneath the jokes is a bigger question: what happens when websites serve both people and the agents acting for them?

I heard this topic surface at GoLive 2026, Pantheon’s inaugural conference, where Nick Thompson, CEO of The Atlantic, kept the panelists on their toes despite the playful mood. He traded questions with Stephanie Cohen, Chief Strategy Officer at Cloudflare, Matthew Duncan, former Future of Work leader at Microsoft, and Jody Schwartz, SVP of People at Pantheon.

Here are three things I learned about bots and the future of work that may be useful for the marketer reading this blog. 

How to treat bots as a new audience

For years, “bot” meant nuisance: something to block or filter. But now bots fall into three unofficial categories: good, bad and evil. The pressing question is: Which bots are we trying to reach, what do we want them to understand, and how do we block them if we have to?

“Bots may be your customer,” said Cohen of Cloudflare. In fact, their data suggests bots now outnumber human traffic. For content marketers, that metric changed the content brief. A page may be read by a person, summarized by an agent, compared with competitors, and used to shape a recommendation before a human ever visits. Cohen’s image was memorable: the web becomes an iceberg, with people seeing the tip while activity happens underneath between agents.

To put some bot police in place, Cohen pointed to WebBotAuth. It lets verified AI agents cryptographically sign requests using HTTP Message Signatures. In plain English, it helps sites verify an agent’s identity instead of trusting a self-reported user-agent string. From there, a site can decide which bots to admit and what they can access.

Then Thompson offered a deliberately provocative thought experiment. There are three ways to scrape content:

• Generous: respects the site’s wishes.
• Aggressive: ignores them but remains identifiable.
• Evil: actively disguises the bot as human.

Sadly, some companies today choosing the third option makes the governance problem concrete: the web cannot rely on good intentions alone.

That teed up Cohen’s provocative proposal: let’s charge companies for bad bots. If one bot misbehaves, accountability should travel back to its company behind it. One practical response is an allowlist (adopted by The Atlantic too): admit only verified bots. But that can also block legitimate bots, making shared verification standards essential.

When AI kills the org chart

The industry backdrop is already visible: job cuts (Microsoft takes the cake, offering buyouts to 60K employees), flatter teams, functions being folded together, and constant pressure to produce more with fewer people. Pantheon’s State of the Web 2026 complicates the job-cut narrative: AI accelerated content, code, and policy creation. But only 44% of marketing leaders said AI makes them genuinely faster; 36% said time shifted to other work, and 79% said manual validation makes AI harder to scale. 

In other words, we are all in the middle of transformation, and it will take time to see the productivity explanation, which is often offered as a reason for job cuts.

But reducing headcount is not the same as redesigning work.

One example from the conversation: Cohen described Cloudflare eliminating product marketing as a standalone function. The work did not simply vanish. The old boundary between product and marketing stopped making as much sense to Cloudflare when AI could help turn product information into an initial customer-facing draft.

Duncan put it more sharply: “I think the org chart is dead. We need to move to a work chart.” The line landed because it named what job-cut headlines often obscure: the question is not only how many people remain, but how work gets divided between people and agents. Adding AI to a broken process only gives us a faster broken process, with fancy formatting.

The future org chart may be less about titles and more about decision rights. Who owns the outcome? Who can stop the machine? Who makes the ambiguous call?

Trust is the new brand moat

After all the talk about agents, Thompson of The Atlantic asked the question I kept thinking about: “How do you maintain the human connection with the website?”

It was the sharpest question because it refused to treat the web as purely technical. As pertinent to a journalist, he kept asking where value accumulates, whether a brand’s relationship could be reduced to its bot, and how a publisher maintains agency when agents read on behalf of humans.

Jody Schwartz, SVP of People at Pantheon, offered the answer in a phrase that belongs on every brand brief: “The moat of what makes you, you.” She connected that moat to authenticity, personality, judgment, and the desire to know there is a real perspective behind the experience.

That matters because AI is making information and content more abundant, not necessarily more meaningful. The strongest brands (the ones that will survive) will use AI to remove friction while keeping their voice, judgment and personality intact.

As a former journalist, I keep coming back to the difference between information and perspective. An agent can summarize the story. It cannot automatically supply the reason a reader should trust this source, this voice or this point of view.

My takeaway: let agents handle more of the scavenger hunt. Keep the judgment, taste, and relationship human. Then take one workflow and map where AI can accelerate the work, where a person must decide, and what context, permissions, review, attribution, and rollback need to travel with it.

That is Pantheon’s “move fast, stay in control” point of view in practice, and a useful test of whether your web operating model is ready for the AI era. 

Check out where Pantheon’s headed in the AI era of the web.