Copied!
Blog

Claude can’t actually read half the web. Here’s the fix

Ask an AI to read a modern website and it often reads an empty shell. Here’s why plain fetching fails, and the eight MCP servers that fix web research and scraping.

← Blog
Earth at night seen from space with glowing city lights
Photo on Unsplash

September 29, 2026

MCP servers for web research

You paste a link. The AI confidently summarizes it. The summary is wrong, or thin, or describes a loading spinner. Nobody warned you it couldn’t actually see the page.

Here’s what happened: the server sent a near-empty HTML shell and a pile of JavaScript. A browser runs that script and builds the real page. A basic fetch tool never does. So on single-page apps, JS-rendered content, lazy-loaded sections and anything behind an interaction, the AI reads the shell and improvises the rest.

The fix isn’t a smarter model, it’s a better reader. These are the MCP servers that give Claude one.

Disclosure, up front: we build Enrichr, so it appears on this list. We’ve tried to say exactly why, and to be straight about where the others are the better tool.

In this list
  1. *Enrichr
  2. Firecrawl
  3. Playwright
  4. Brave Search
  5. Fetch
  6. Chrome DevTools
  7. PubMed
  8. Google Drive

01 Enrichrour pick

Enrichr renders the page in a real browser, then reports on what appeared: the post-JavaScript DOM, structured data, headings, links, assets, even the animations and canvas content. scrape_url is the quick way to get clean markdown from one page; crawl_site and batch_scrape handle many. It also includes OCR and PDF parsing for content that isn’t HTML at all. Best when the page is dynamic and you need to trust what you’re reading.

02 Firecrawl

Turns pages and whole sites into LLM-friendly markdown and structured data. A popular choice for large scraping and crawling jobs where clean text is the goal.

03 Playwright

General browser control. When the content is behind a click, a login you’re authorized to use or a multi-step flow, Playwright lets Claude perform the steps and read the result.

04 Brave Search

Search is a different job from reading. Brave’s search API gives Claude a way to find pages in the first place, then hand them to a reader.

05 Fetch

The lightweight reference server for pulling a URL’s content and converting it to markdown. Fast and simple, and fine for static pages. Just know its limits: it isn’t a browser.

06 Chrome DevTools

Read network requests and the console. Sometimes the data you want isn’t in the page at all; it’s in the API call the page makes, and DevTools shows you which one.

07 PubMed

For scientific and medical research, a dedicated source beats scraping every time: search articles, read metadata, follow citations from the primary literature.

08 Google Drive

Your own documents are part of “the web” for research purposes. Pair external sources with the notes and papers you already have.

A quick test to run on any reader: give it a page you know is JavaScript-heavy and ask what’s in the third section. If the answer is vague or wrong, it read the shell. And scrape responsibly: respect robots rules and terms, throttle your requests and avoid personal data.