NEW ScrapingAnt MCP for Claude Code, Cursor & Windsurf — try it free →
★★★★★ 5.0 on Capterra

News scraper API. Every story as HTML, Markdown or JSON.

Scrape news articles from the sources you choose with one API call. Take the raw HTML for your own parser, the article as Markdown, or the headline, author and publish date as JSON. Headless Chrome and rotating proxies are built in.

10,000 article fetches a month free · failed requests cost 0 · no card required

# The full page, exactly as the publisher serves it. 1 credit.
$ curl 'https://api.scrapingant.com/v2/general' \
    --data-urlencode 'x-api-key=YOUR_KEY' \
    --data-urlencode 'url=https://news.example.com/markets/chipmaker-earnings' \
    --data-urlencode 'browser=false'
# → <!doctype html><html>… headline, body, bylines and
#   schema.org NewsArticle metadata, as served </html>
# The same article as Markdown, ready for search, summaries or an LLM.
$ curl 'https://api.scrapingant.com/v2/markdown' \
    --data-urlencode 'x-api-key=YOUR_KEY' \
    --data-urlencode 'url=https://news.example.com/markets/chipmaker-earnings' \
    --data-urlencode 'browser=false'
# → { "url": "https://news.example.com/markets/chipmaker-earnings",
#     "markdown": "# Chipmaker beats forecasts\n\nBy J. Rivera …" }
# No parser at all: name the fields, get JSON.
$ curl 'https://api.scrapingant.com/v2/extract' \
    --data-urlencode 'x-api-key=YOUR_KEY' \
    --data-urlencode 'url=https://news.example.com/markets/chipmaker-earnings' \
    --data-urlencode 'extract_properties=title,author,published_at,summary'
# → { "title": "Chipmaker beats forecasts", "author": "J. Rivera",
#     "published_at": "2026-11-17T08:30:00Z", "summary": "…" }
# Headlines from a section page, then the full text of each story.
import requests

API = "https://api.scrapingant.com/v2"
KEY = "YOUR_KEY"

stories = requests.get(f"{API}/extract", params={
    "x-api-key": KEY,
    "url": "https://news.example.com/technology",
    "extract_properties": "stories(list: headline, url, published_at)",
}).json()["stories"]

for story in stories:
    text = requests.get(f"{API}/markdown", params={
        "x-api-key": KEY, "url": story["url"],
    }).json()["markdown"]
    save(story, text)  # your database, your alerts
your-news-pipeline MARKETS 2 min ago Chipmaker beats forecasts as AI demand surges By J. Rivera · The Daily Ledger "title": "Chipmaker beats forecasts…", "author": "J. Rivera", "published_at": "2026-11-17T08:30:00Z", "companies": ["…", "…"], "summary": "…" api.scrapingant.com · HTML · Markdown · JSON
article URL /v2/general raw HTML HTML <h1>Chipmaker beats…</h1> <time datetime="2026-11-17"> <script type="application/ld+json"> "@type": "NewsArticle" your parser takes it from here
Raw extraction

Raw pages. Your parser.

Already have a parser, or want full control over what you keep? /v2/general returns the complete HTML of any news page, so your BeautifulSoup, Cheerio or XPath code works unchanged. You stop running browsers and proxies, and keep every line of your extraction logic.

Many news sites publish the headline, author, publish and update times as schema.org NewsArticle metadata inside the page. With the raw HTML you read those fields directly, for 1 credit per article. Need the text without the markup? /v2/markdown returns the same page as Markdown.

  • Raw HTML from 1 credit per page, with JavaScript rendering when a site needs it
  • Works with the tools you already use: BeautifulSoup, lxml, Cheerio, Scrapy pipelines, newspaper-style article parsers
  • Images, links, embedded JSON and metadata all arrive intact, nothing is summarized or dropped
How the Web Scraping API works →
National press Local outlets Trade journals Newsrooms /v2/extract AI extraction JSON title author published_at section summary
AI extraction

One schema. Every newsroom.

No parser yet, or tired of fixing one after every redesign? ScrapingAnt's /v2/extract endpoint takes a list of fields, such as title,author,published_at,summary, and returns them as JSON from any article page. Add a source to your watchlist by adding its URL, not by writing another parser.

  • Lists too: pull every headline, link and timestamp from a section page in one call
  • Go beyond the page: ask for companies_mentioned, people_quoted or sentiment
  • Full text as Markdown from /v2/markdown, ready for search, summaries and embeddings
How AI extraction works →
url=news.example.com/business proxy_country=US Fed holds rates US edition proxy_country=GB FTSE hits record UK edition proxy_country=JP 日経平均が上昇 Japan edition one URL · the edition local readers see country targeting at no extra cost
Browsers and proxies included

Any source. Any country's edition.

News sites rate-limit crawlers, render stories with JavaScript and show different front pages by region. ScrapingAnt runs the headless Chrome cluster and rotates the IPs for you, so one request works the same on a wire service and a small-town weekly. Add proxy_country to read the edition local readers see.

A non-browser datacenter fetch costs 1 credit. JavaScript rendering costs 10 credits with datacenter proxies or 125 with residential proxies. See the full credit-cost breakdown.

  • Rotating datacenter and residential IPs across 100+ countries, with no geotargeting surcharge
  • Headless Chrome for live blogs, infinite scroll and client-rendered stories
  • Failed requests cost 0 credits, so a blocked fetch never eats your budget
YOUR NEWS CRAWLER your schedule, your storage, your alerts 1 Feeds and section pages every N minutes 2 New links only dedupe by URL 3 Fetch and extract in parallel your_alert(company="…", story=…) → minutes after it publishes
Built for news crawlers

From feed to database. In an afternoon.

A news crawler is three steps: read the feeds and section pages, keep the links you have not seen, fetch and extract each new story. ScrapingAnt covers the fetching and extracting, so the code you own is a scheduler and a table. You can have stories landing in your database the same day you sign up.

Polling 200 feeds every 15 minutes takes about 576,000 credits a month at 1 credit per feed. Every account starts with 10,000 free credits, enough to prove the pipeline before you pay anything.

  • Unlimited concurrency: fetch a morning's worth of stories as fast as your workers can send them
  • No index and no delay tier: each request reads the live page, corrections included
  • Python and JavaScript clients, an MCP server for AI agents, and plain HTTP for everything else
Guide: how to scrape Google News →

Six things teams build with a news scraper.

Same API, same credit pool. The sources and the questions are yours.

Media monitoring & PR

Know when a client, a competitor or an executive is in the news, across national press, local outlets and trade journals. Pair it with the brand monitoring API for mentions beyond news.

Financial news signals

Feed earnings stories, analyst notes and company announcements into your models minutes after they publish, with the publish time attached to every record.

AI training data & RAG

Build a fresh news corpus for fine-tuning or retrieval. Article text comes back as Markdown, ready to chunk and embed, so your model answers with today's facts.

News aggregators & apps

Power a news app, a niche newsletter or an industry digest from the sources your readers trust, including the small ones no ready-made feed carries.

Competitor & market intelligence

Track product launches, funding rounds, hires and regulatory moves in your industry. Extract company names and deal sizes straight into your CRM or data warehouse.

Research & news archives

Collect coverage of an event, a topic or an election for academic and policy research. Keep dated snapshots with the web archival API before stories are edited or pulled.

News scraper questions.

What is a news scraper?

A news scraper automatically collects headlines, article text, authors and publish dates from news websites. ScrapingAnt is the API you build one on: send a URL, get the page back as raw HTML, Markdown or JSON.

How do I scrape news articles?

Fetch a section page or RSS feed to get the article links, then send each link to the API with the fields you want, such as title,author,published_at. Run it on a schedule and store only the new stories.

Can I get the raw HTML of a news article?

Yes. /v2/general returns the full HTML of the page for 1 API credit, so you can parse it with your own code and keep the publisher's metadata.

What is the difference between a news scraper API and a news API?

A news API sells a fixed feed from the sources its vendor picked. A news scraper API fetches the pages you name, so you get full article text from local, trade and non-English outlets that feeds leave out.

Is there an article extractor API?

Yes. The /v2/extract endpoint works as an article extraction API: pass a list of fields and it returns them as JSON from any article page, without a template per publisher.

Can I build a news crawler with a scraping API?

Yes. Your scheduler asks ScrapingAnt for section pages and feeds, queues the links it has not seen and fetches each new article. The API handles browsers, proxies and blocks, so the crawler is a few dozen lines of code.

How much does a news scraper cost?

A plain article fetch costs 1 API credit, and every account gets 10,000 free credits a month with no card required. Failed requests cost zero.

Pricing

Industry leading pricing that scales with your business.

Compare plans side by side. Every tier includes 10,000 free credits to start.
👈Swipe to compare all 5 plans👉
Plans
Enthusiast
100K credits / mo
$19/mo
★ Most Popular
Startup
500K credits / mo
$49/mo
Business
3M credits / mo
$249/mo
Business Pro
8M credits / mo
$599/mo
Custom
10M+ credits / mo
$699+/mo
Monthly API credits 100,000 500,000 3,000,000 8,000,000 10M+
Support channel Email Priority email Priority email Priority email Priority + dedicated
Integration help Docs only Custom code snippets Debug sessions Priority debug sessions Full enterprise onboarding
Expert assistance — included included included included
Custom proxy pools — — included included included
Custom anti-bot avoidances — — included included included
Dedicated account manager — — included included included
Start Free Start Free → Start Free Start Free Talk to Sales
⚡
Hit your limit mid-month?
Restart your plan instantly — no waiting for the next billing cycle. Credits refresh the moment you pay, so scraping never has to stop.
✓10,000 free credits every month
✓No credit card required
✓Pay only for successful scrapes — failed requests cost 0
Customers

What teams are saying.

From solo developers shipping side projects to enterprise pipelines at Fortune 500s.

★★★★★ 5.0 on Capterra →
★★★★★

“Onboarding and API integration was smooth and clear. Everything works great. The support was excellent.”

Illia K.
Android Software Developer
★★★★★

“Great communication with co-founders helped me to get the job done. Great proxy diversity and good price.”

Andrii M.
Senior Software Engineer
★★★★★

“This product helps me to scale and extend my business. The setup is easy and support is really good.”

Dmytro T.
Senior Software Engineer
FAQ

News scraping API FAQ.

Anything else? Talk to us — we read every email.

What is a news scraping API?

A news scraping API is a managed HTTP endpoint that fetches pages from news websites and returns them as HTML, Markdown or structured JSON. It replaces the parts of a news scraper that are painful to run yourself: headless browsers, proxy rotation, anti-bot handling and a parser for every publisher. With ScrapingAnt, /v2/general returns the page, /v2/markdown returns it as Markdown, and /v2/extract returns the fields you ask for as JSON.

Do you offer a ready-made news feed like a news API?

No, and that is the point. A news API gives every customer the same index, with the sources, delay and article snippets its vendor chose. ScrapingAnt fetches the live page from the sources you choose, at the moment you ask, and returns the full article. Teams pick us when the outlets they care about are missing from news feeds, when they need full text and not a 200-character snippet, or when they want to own the pipeline.

Can I use my own parser instead of AI extraction?

Yes, and many teams do. Request the page from /v2/general and you get the complete HTML to run through BeautifulSoup, Cheerio, XPath or an article-parsing library. A plain fetch costs 1 credit, which makes raw extraction the cheapest way to collect news at volume. Use /v2/extract only for the sources or fields where writing a parser is not worth it.

Which fields can I extract from a news article?

Any field you can name. Common choices are title, author, published_at, section, summary and tags, plus lists such as companies_mentioned or people_quoted. Pass them in extract_properties and the AI extraction endpoint returns them as JSON. For the full text, request the article as Markdown.

How fresh is the news data?

As fresh as your schedule. Every request fetches the live page, so you read a story the way it stands right now, including corrections and updates. There is no shared index and no delay tier: poll a breaking-news section every minute or a trade journal once a day.

Can I scrape Google News?

Yes. Google News publishes RSS feeds for searches, topics and locations, and you can fetch them through the API from any proxy country. Our guide on how to scrape Google News walks through the feed URLs, the parsing code and a full Python example. Use the feed to discover stories, then fetch each publisher's article for the full text.

Can I get local editions and news in other languages?

Yes. Add proxy_country to route the request through that country and see the edition local readers see, with local headlines and language. Country targeting costs nothing extra, on datacenter and residential proxies alike.

How much does it cost to scrape 100,000 news articles a month?

Fetching 100,000 articles without a browser through datacenter proxies takes 100,000 credits, which fits the Startup plan ($49/month, 500,000 credits) five times over. With JavaScript rendering the same volume takes 1M credits. These are fetch budgets: failed requests cost zero, while retries and repeat fetches use credits.

AI extraction adds ceil((Markdown input characters + output characters) / 30) credits on top of the fetch. A common pattern keeps it cheap: extract headline, author and date fields from short pages, and fetch the full article text as Markdown. See the full credit-cost breakdown.

Does it work on JavaScript-heavy news sites?

Yes. Set browser=true and the page loads in headless Chrome, so live blogs, infinite-scroll section pages and client-rendered articles come back fully rendered. Many news sites serve complete HTML without it, so try browser=false first and pay 1 credit instead of 10.

Can my AI agent read the news through ScrapingAnt?

Yes. The ScrapingAnt MCP server gives Claude, Cursor and other MCP clients tools that fetch any page as Markdown, text or HTML. Point your agent at a news URL and it reads the current story instead of relying on training data.

Is it legal to scrape news websites?

Collecting publicly available news pages for analysis, monitoring and research is common practice, but news content is protected by copyright. Store and analyze what you collect, link back to the source, and do not republish full articles without a license. Check the terms of the sites you work with and ask your counsel about your specific use.

Talk to us

Following thousands of news sources?

Custom volume pricing, help designing your crawler, or a one-off dataset of historical coverage to get started. Drop us a line and a real human gets back within a few hours.

“Our clients are pleasantly surprised by the response speed of our team.”

Oleg Kulyk
Founder, ScrapingAnt

A real human replies within a few hours · we don't share your email

Thanks — we'll be in touch shortly.
Something went wrong submitting the form. Please try again or email us directly.

Ready to scrape the web?

10,000 free credits every month. No credit card. Pay only for successful requests.

Sign up in under 30 seconds — no card, no commitment.