Six Months of LLM Agents Doing My SEO Monitoring: The Honest Scorecard

The stack in plain words

A scheduler fires a few times a day. Each task launches a headless LLM agent session with one narrow job: pull fresh Search Console data and write a seven-day summary, check that every monitored URL still returns 200, diff the titles and structured data of key pages against yesterday, probe each site’s llms.txt and schema markup for conformity. The output lands in a Telegram chat, and a copy goes to a journal file the next session can read.

The piece that made it scale: a single registry file that describes each site. Add a site to the registry and it inherits the whole monitoring suite the same day. My 5,200-page catalog and this portfolio run the identical stack.

What agents are good at

Collection and cross-referencing, without fatigue. Every morning the GSC agent compares the last seven days to the seven before, names the top and bottom movers, checks indexation counts against a known baseline, and flags anything that moved more than the noise. I have not opened Search Console for a routine check since February.

The second strength is cross-checking claims against the live site. My recap agent does not just say a page lost clicks; it fetches the page and reports whether the title, meta description and schema are still intact before speculating. That single habit kills most false alarms about “something broke”.

The morning my agent invented a cause

In June, the recap flagged a French page with a weak click-through rate and explained it confidently: Google’s AI Overviews were absorbing the clicks. One problem. AI Overviews were not deployed in France. The page’s real issue was ordinary: it sat at position 6 to 9 on its money queries, and low CTR is what position 6 to 9 looks like.

The agent did not lie; it pattern-matched. CTR erosion plus 2026 plus SEO equals AI Overview in most of what the model has read, so that became the explanation. It inherited the ambient hype the way it inherits everything else.

The fix was a standing rule, not a better model: no cause may be named without checking the live state first. Which market is this? Is the feature even deployed there? What is the actual position on the queries that matter? Since that rule shipped, diagnosis quality changed more than any prompt tweak had achieved.

The bottleneck nobody talks about: deciding

By July the pipeline had a different failure. Every morning recap ended with a proposed action, each one reasonable, each one waiting for my yes. Eight recaps piled up unanswered. The analysis machine ran perfectly and nothing happened, because the scarce resource was never analysis. It was my decisions.

We restructured: daily recaps are now information only, and decisions batch into one weekly session with a hard cap of three, argued from clean data windows. That last part matters more than it sounds. Seven-day comparisons breathe with the weather; a title change that looked like a failure in the daily noise turned out, on proper before-and-after windows, to have improved CTR. The rollback I almost approved would have undone a working change.

The rules that survived six months

1. Verify live state before any claim. Fresh GSC data and the actual served HTML, never memory or assumption.
2. Agents inform, humans decide. Nothing ships to production without a named human decision.
3. Every change carries its rollback. Local backup of the previous state plus a written revert path, before pushing.
4. Decide weekly, on long windows. Daily data is for spotting fires, not for judging experiments.
5. Alert on failure, not just success. A monitoring task that dies silently is worse than no monitoring, because you trust it.

Is it worth it?

Costs, honestly: several days of setup, prompt maintenance when a report drifts in quality, and the occasional absurd false positive. My quota monitor once alerted on an error pattern it had found in its own log output.

Against that: I watch two properties and a 168-location network’s presence with a morning glance at a chat thread, anomalies surface the day they happen instead of the week I remember to look, and the whole system transfers to any new site by adding one registry entry. For a solo operator, that trade is not close.

FAQ

Which model runs the agents?

Claude, in headless sessions launched by the OS scheduler. The pattern is model-agnostic: narrow task, fresh data pull, structured output, one delivery channel.

Do the agents change anything on the sites?

Not on their own. They read, compare and report. Changes go through scripts a human triggers after a decision, each with a backup and a written rollback.

What should the first agent do?

The daily Search Console recap. It is the highest signal for the least plumbing, and it teaches you immediately where your agents will overreach.

The monitoring described here is what caught the llms.txt conformity gap in the previous article. The two systems are one stack.