Selenium Cookies in Python: Set, Save and Validate Scraped Data

Updated 2026-09-27

Replaced mixed-language API examples, cookie insertion on about:blank and untested session recipes with runnable extraction tests. The evidence includes Chrome and Firefox cookie restoration, incorrect-data cases, and a separate ScrapingAnt test that receives and resends synthetic cookies. The original URL and publication date are preserved.

To set a cookie in Selenium Python, first navigate to its target origin, then call driver.add_cookie() with a name/value dictionary. Navigate or refresh afterward so the next request uses it. To reuse a session, save the relevant cookies, restore them on the same origin in the new browser, and check the resulting data.

That last step matters. In our controlled catalog, every extraction returned HTTP 200 and four product rows. Restoring only the session cookie still selected member prices—but in USD when the intended dataset was EUR. A successful navigation, an authenticated-looking page and the expected row count all missed the error.

This guide shows the cookie operations, a complete runnable save/restore example, and the measured failures behind that distinction. The catalog and accounts are synthetic; the results demonstrate mechanisms, not production-site success rates.

Set, read and delete cookies on the correct origin

Python's methods are called directly on driver. There is no driver.manage().cookies() chain in these Python examples.

Task	Selenium Python method	What to check
Add or replace a cookie	driver.add_cookie(cookie_dict)	Open the intended origin first; keep domain and path narrow
Read an associated cookie	driver.get_cookie(name)	The result can be None; presence does not prove a valid server session
Read associated cookies	driver.get_cookies()	This is a snapshot for the current document, not every cookie in the browser
Delete a named cookie	driver.delete_cookie(name)	Revisit the data page and verify the resulting state
Delete associated cookies	driver.delete_all_cookies()	Use the intended origin; this does not clear local storage

The Selenium cookie guide documents these operations. The WebDriver specification defines them in relation to the active document. In both tested browsers, adding a cookie on about:blank, or specifying an unrelated domain, raised InvalidCookieDomainException.

A reliable sequence is: navigate to a lightweight page on the intended origin, confirm you stayed on that origin, add the cookie, then load the extraction page. An unexpected redirect to a different login host is a reason to stop and inspect the flow. Broadening a cookie's domain is not a fix for that mismatch.

Run a complete save-and-restore example

Download or check out the tested example revision, then change into examples/selenium-set-cookies before running the commands below. That directory includes the fixture, browser setup and cookie helpers. These commands require Python 3.12, Chrome and openssl; Selenium Manager may download the matching browser driver:

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-lock.txt
python quickstart.py
./run.sh

The measured environment used Selenium 4.49.0 and Python 3.12.10. The primary browser runs used Chrome 154.0.8037.57 and Firefox 156.0.1 on macOS 26.6.2 arm64. The dependency lock, browser/driver versions and raw outputs are in the packet; these are the tested versions, not a promise that they remain current.

Here is the complete quickstart.py. fixture_server, browser, extract and the snapshot functions are helpers supplied in that directory; the cookie methods themselves are ordinary Selenium calls.

"""Run with: python quickstart.py. Only synthetic localhost data is used."""
import json
import tempfile
from pathlib import Path

from catalog_fixture import fixture_server
from browser_matrix import browser, extract
from cookie_state import save_snapshot, restore_snapshot


def main():
    with fixture_server() as (server, base), tempfile.TemporaryDirectory() as tmp:
        snapshot = Path(tmp) / 'cookies.json'
        with browser('chrome') as driver:
            # This fixture issues a demo session; no real login is performed.
            driver.get(base + '/seed')
            driver.get(base + '/catalog/')
            driver.delete_cookie('demo_region')
            driver.add_cookie({
                'name': 'demo_region', 'value': 'eu', 'path': '/',
                'secure': True, 'sameSite': 'Lax',
            })
            save_snapshot(driver, snapshot, base)

        with browser('chrome') as driver:
            driver.get(base + '/blank')
            restored = restore_snapshot(driver, snapshot, base)
            result = extract(driver, server, base, 'quickstart', 4)
            print(json.dumps({
                'restored': restored,
                'matching_records': result['matching_records'],
                'records': result['records'],
            }, indent=2))
            assert result['assertion_passed']


if __name__ == '__main__':
    main()

The fixture's /seed route issues a synthetic session and region preference. The example explicitly replaces the region cookie, saves the state, closes the first browser, and restores the snapshot into a fresh browser. /blank establishes the destination origin before restoration. extract() then navigates to /catalog/ and checks the four expected SKU/currency/price tuples.

Selected fields from the actual quickstart output:

{
  "restored": {
    "restored": 2,
    "skipped_expired": 0
  },
  "matching_records": 4
}

The complete capture also contains the four extracted records. Prices use integer minor units: the expected EUR member values are 800, 1600, 2400 and 3200. These are fixed synthetic price schedules, not currency conversions or observations from a shop.

The temporary snapshot is removed on exit. For repeated jobs, the helper accepts a caller-chosen private file path; exclude that path from Git and logs. The local fixture generates a temporary self-signed certificate, and its test browsers accept it. Use normal certificate verification for real target sites.

What the cookie experiments changed in the extracted data

We ran three rounds in each primary browser. Each round contained nine extraction cases and four cookie diagnostics: 54 extraction observations and 78 browser checks in total. Each extraction case therefore has six observations, and every one of those observations returned four rows with HTTP 200.

The correctness check matches each expected SKU, currency and price once. Duplicate copies of one product cannot substitute for missing products. The offline regression tests cover that distinction, alongside missing products, wrong currency and reordered records.

Cookie state	Rows returned	Correct member-EUR records	What happened
No cookies	4	0/4	Retail USD schedule
Server-issued session and region	4	4/4	Intended member EUR schedule
Complete snapshot, fresh browser	4	4/4	Intended state restored
Session cookie only	4	0/4	Member prices, wrong currency: USD
JavaScript-visible cookies only	4	0/4	EUR preference retained; HttpOnly session omitted, so retail prices
Session cookie with the wrong path	4	0/4	Session not sent to the catalog path
Expired session filtered before insertion	4	0/4	Only the remaining preference was restored
Server-revoked session	4	0/4	Cookie still present; fixture no longer accepted the session
Session cookie deleted	4	0/4	Retail pricing returned

All six observations per case agreed. See the full report, Chrome captures and Firefox captures. Repeating a deterministic fixture on one machine does not estimate how often these problems occur on real websites.

Two failures deserve different recovery paths. A missing region preference calls for restoring the required extraction state. A revoked session calls for re-establishing a valid session through the site's intended flow. Re-inserting the same cookie bytes did not make our revoked demo session valid.

For a production catalog, define checks appropriate to the data contract: currency and region, expected account role, required columns, unique product identifiers, and plausible values. The test here uses exact known prices because it owns the fixture. A live collector should not blindly apply those synthetic constants to changing prices.

Save the attributes and scope you actually need

The packet's JSON snapshot helper records the source origin and the cookies returned for the current document. On restore, it checks the destination scheme, host and port, validates the snapshot, filters already-expired entries, and preserves supported path, Secure, HttpOnly, SameSite and expiry fields.

It deliberately accepts only the exact source host and omits domain during insertion to keep restoration tied to that host. It refuses parent-domain or foreign-domain cookies instead of guessing their intended scope. This makes it a narrow example for this catalog, not an exporter for arbitrary browser sessions. Applications that need legitimate multi-subdomain cookies require an explicit domain policy.

Attribute or state	Consequence for extraction
Domain and path	Determine where the browser can send the cookie; a catalog request can miss a cookie that exists elsewhere
HttpOnly	Hides the cookie from document.cookie; it does not hide it from the tested WebDriver cookie API
Expiry	An expired snapshot entry is skipped by this helper; refreshing the file does not renew the server's session
Secure	Restricts transmission to secure contexts, subject to browser rules; the fixture uses HTTPS
SameSite	Controls cross-site cookie sending; copying the field does not test an entire cross-site login flow
Server-side session validity	The server can reject a session even while its cookie remains in the browser

The Set-Cookie reference describes the attribute rules. In our HttpOnly diagnostic, WebDriver returned both demo cookie names, while document.cookie exposed only the region cookie. That is why a JavaScript-only export lost the member session.

Saving cookie JSON also does not capture local storage, session storage, or an entire browser profile. If those are part of your application state, investigate them separately; see Selenium local storage. The fixture's expiry case tests filtering a past timestamp before insertion, not waiting for the browser to expire a cookie naturally. Cross-site SameSite behavior, partitioned cookies, SSO and MFA were not tested.

The helper writes owner-only files on POSIX systems and refuses a final-path symlink. JSON is not encryption. Store real cookie snapshots as credentials, avoid printing their values, and keep both them and API keys out of version control. Windows permissions and browser behavior were not tested in this experiment.

Receive and send cookies with ScrapingAnt

If you want ScrapingAnt to retrieve pages for the extraction workflow, cookie state can travel between requests without running this local Selenium browser. ScrapingAnt accepts cookies as name/value pairs through the cookies parameter, with multiple pairs separated by semicolons. ScrapingAnt's extended endpoint returns received cookies in its cookies field, so you can pass the relevant pairs into a later request.

We tested that round trip separately against a public cookie echo endpoint using two synthetic values. The probe made four calls in each datacenter mode: a baseline, a cookie-setting URL, a new request without replay, and a request explicitly sending the received pairs. All eight API responses and target statuses were 200.

Step	Without a browser	With a browser
Baseline	No demo cookies echoed	No demo cookies echoed
Set endpoint	Both expected pairs received and echoed	Both expected pairs received and echoed
Separate call without replay	No demo cookies echoed	No demo cookies echoed
Explicitly replay received pairs	Both values echoed by the target	Both values echoed by the target
Total measured credits for four calls	4	40

The capture contains all eight credit receipts, totaling 44 credits. The returned body field was html; the received cookie pairs were in cookies. The product credit rules for these settings are: A request without a browser through a datacenter proxy costs 1 API credit. A request with JavaScript rendering through a datacenter proxy costs 10 API credits.

Inspect the non-live plan first:

python scrapingant_cookies.py

To reproduce the paid probe, supply SCRAPINGANT_API_KEY through your environment or secret manager, then explicitly opt in:

./run.sh --scrapingant

It permits at most eight API attempts and makes no automatic retries. An existing output file blocks accidental reruns. The key goes in the API authentication header; only selected statuses, credit counts, synthetic cookie names and boolean comparisons are saved. The README explains output handling and deliberate reruns.

Use this as a receive/select/replay pattern. Keep only the cookie pairs needed by the intended target and let the HTTP client encode the cookies parameter. The name/value string does not carry a WebDriver cookie's domain, path, expiry, HttpOnly, Secure or SameSite metadata. Do not flatten cookies from several sites into one string or send them to a different target.

The echo test confirms cookie transport in the two measured modes. It does not establish that a real login survives a change of browser, network or other session state, and it does not renew expired or revoked authentication. ScrapingAnt is unnecessary for the local fixture or a direct Selenium workflow you already operate. For remote page retrieval with explicit cookie chaining, start with the ScrapingAnt cookie documentation.

Recheck the extraction contract when sessions change

Run ./run.sh to execute the 16 offline tests and the local Chrome suite. It does not make ScrapingAnt calls, even when an API key is configured. The committed results remain intact; reruns write to ignored run_output/. The packet documents the optional Firefox run and exact measured version.

For a saved-results check without starting browsers or spending credits:

python summarize.py > run_output/rebuilt-comparison.json

When adapting the examples, keep three checks separate: did navigation succeed, was the expected session state established, and did the extracted records satisfy the data contract? Our session-only case passed the first two and still returned the wrong currency. Validate the data after restoration, expiry handling and session renewal, before accepting the batch downstream.

For the next part of the scraper, see finding elements with Selenium and cookie management with Python Requests.

Examples tested on 2026-09-27 with Selenium 4.49.0, Chrome 154.0.8037.57, Firefox 156.0.1 and requests 2.34.2. Code and outputs: pinned evidence packet.

This article was drafted with AI assistance from a tested evidence packet. The named author is responsible for the code, measurements and corrections.