Selenium

Long-standing browser automation project whose WebDriver API controls real browsers, used for testing and scraping

Visit Selenium →
Tools & LibrariesNew
TopicsScrapingBrowserAutomationTestingWebdriver

Selenium is an umbrella project for tools and libraries that automate web browsers. Its best-known piece, WebDriver, uses the automation APIs the browser vendors provide to control a real browser, and it has official Python bindings. It has been the standard for browser test automation for many years and is also used for scraping pages that need JavaScript.

You no longer have to download browser drivers by hand: Selenium Manager, a command-line tool written in Rust, is shipped with every Selenium release and manages drivers and browsers automatically. Selenium also includes Grid for running tests across many machines.

See Python Web Scraping Tools Compared for how it compares with Playwright and the non-browser options.

More in Tools & Libraries

Ruff

An extremely fast Python linter and formatter written in Rust — replaces flake8, black, isort, and more in a single tool

FeaturedLinterFormatter

Poetry

Dependency management and packaging tool for Python projects with virtual environment handling

ToolsDependency-management

Mimic

Captures network traffic from mobile apps and generates a reusable Python client library for their APIs, without manual reverse-engineering

Reverse EngineeringAPI