Documentation

Quickstart (Basic)

This page shows how to make a ScrapingAnt request with curl.

Before you begin

First of all, you have to log in to the ScrapingAnt service.

You’ll be able to use the service with a free Basic plan and with 1000 API requests per month (no payment information required).

Making request

The curl command includes JSON with the url to be fetched (url), and the flag specifying return type (return_text). Here we will try to scrape https://example.com:
curl --request POST \
    --url https://api.scrapingant.com/v1/general \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --header 'x-api-key: API_KEY' \
    --data '{ "url" : "https://example.com", "return_text" : false }' 
You should see a response similar to the following:
{
   "content":"<!DOCTYPE html><html><head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\">\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\">\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n\n\n</body></html>",
   "cookies":""
} 
If you don’t want to bother yourself with html tags, you can specify "return_text": true, and the result will be as following:
{
   "content":"Example Domain\n\nThis domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.\n\nMore information...",
   "cookies":""
} 

The content JSON property contains exact HTML/Text response from the scraped site and the cookies property contains cookies that been returned from the target site.

Examples

Still have questions?

You can find more details about our API and have more monthly API requests at ScrapingAnt RapidAPI page.

Or just reach us by support@scrapingant.com