selector-picker

Click any element on any page and get a robust CSS selector, XPath, and ready-to-paste BeautifulSoup / Parsel / Scrapy code.

selector-picker is a small Manifest V3 browser extension for people who write scrapers. Instead of squinting at DevTools and hand-tuning nth-child chains, you click the element you want and it hands you a minimal, stable selector plus the Python to use it:

• a robust CSS selector — prefers a unique id, then stable attributes (data-testid, name, aria-label, …), then meaningful classes, and only falls back to :nth-of-type / :nth-child when it truly has to;
• a short XPath keyed on an id, attribute, or text — plus the absolute XPath;
• ready-to-paste snippets for BeautifulSoup, Parsel, and Scrapy;
• the element's text preview and key attributes.

Everything is copyable with one click. No build step, no npm packages to run it — it is plain HTML, CSS, and vanilla JavaScript that loads straight from disk.

Screenshots

The popup and the in-page results panel are described in docs/README.md (capture your own so they match your theme):

• Popup — a Start picking button and the most recent pick with copy buttons.
• In-page panel — appears where you click, showing the highlighted element, its selectors, and the generated Python.

Install (from source, unpacked)

This extension is not published to the Chrome Web Store or Firefox Add-ons — you load it directly from a clone of this repository.

git clone https://github.com/python-web-scraping-com/selector-picker.git

Chrome / Edge / Brave

1. Go to chrome://extensions (or edge://extensions).
2. Turn on Developer mode (top-right).
3. Click Load unpacked and select the cloned selector-picker folder.
4. Pin the crosshair icon to your toolbar.

Firefox

1. Go to about:debugging#/runtime/this-firefox.
2. Click Load Temporary Add-on….
3. Select the manifest.json file inside the cloned folder.

Firefox loads temporary add-ons only until you restart the browser. The core picking and selector generation are standard WebExtension APIs, so it works in both engines; Chromium is the primary target.

Usage

1. Click the toolbar icon and press Start picking (or use the keyboard shortcut Alt+Shift+S).
2. Move your mouse — the element under the cursor is highlighted with an overlay and a tag label.
3. Click the element you want. A panel appears with the selector, both XPaths, and the Python snippets. Press Esc any time to cancel.
4. Hit Copy on whichever field you need. Re-open the popup to see your most recent pick again.

The generated Python carries a single discreet comment pointing back to the guides, e.g.:

# selector built with selector-picker — guides: https://python-web-scraping.com
from parsel import Selector

sel = Selector(text=html)
el = sel.css("article.product:nth-of-type(3) span.price").get()
# XPath alternative:
el = sel.xpath('//span[@data-testid="price"]').get()

How the selector algorithm works

The core lives in src/selector.js and is written so it can run both inside the extension and under Node for testing. For a clicked element it tries, in order, to find the shortest selector that resolves to exactly one element (verified with querySelectorAll at every step):

1. Unique id — if the element has a sensible, non-generated id that is unique in the document, use #id and stop.
2. Stable attribute — data-testid, data-test, data-qa, name, aria-label, itemprop, and friends, if one uniquely identifies the element.
3. Meaningful class combination — using classes that don't look machine-generated.
4. Descendant path — walk up the tree, adding the most stable fragment at each level and testing uniqueness, stopping the moment the path is unique. :nth-of-type is preferred over :nth-child, and both are used only when a class/attribute can't do the job.

Machine-generated tokens — CSS-in-JS classes like css-1a2b3c, hash blobs, and long digit runs like item-83920 — are detected and skipped so the selectors survive the next deploy. The XPath generator mirrors this: it keys the short XPath on an id, a stable attribute, or short leaf text, verifying uniqueness with document.evaluate, and anchors to the nearest id ancestor before it resorts to an absolute path.

Project layout

manifest.json              MV3 manifest
src/selector.js            selector + XPath + snippet engine (also unit-tested)
content/content.js         hover-highlight, click-to-pick, in-page results panel
popup/                     toolbar popup (start picking, show last pick)
background/                service worker (keyboard shortcut + on-demand injection)
icons/                     PNG icons + SVG source
tools/make_icons.py        regenerates the PNG icons (no dependencies)
test/                      Node + jsdom test suite

Development / testing

The DOM logic is fully unit-tested with Node's built-in test runner and jsdom — no live sites are ever contacted.

npm install   # installs jsdom (dev only; nothing is needed to run the extension)
npm test

The tests feed sample HTML to the selector engine and assert that each generated CSS selector uniquely resolves to the intended element and that every XPath resolves through document.evaluate.

To regenerate the toolbar icons:

python tools/make_icons.py

Going deeper

selector-picker gives you the selector — these guides on python-web-scraping.com show you what to do with it:

• Parsing HTML with BeautifulSoup — using select_one / select with the selectors this tool produces.
• BeautifulSoup vs lxml: which parser is faster? — picking the parser behind your selectors.
• Web scraping with Scrapy — where response.css(...) and response.xpath(...) fit in a spider.
• Scrapy vs BeautifulSoup: which to use — choosing between the two snippet styles.
• Extracting data with regular expressions — for the bits that selectors can't reach.
• Extracting JSON-LD & structured data — when the data is in a <script> rather than the markup.
• The Complete Guide to Python Web Scraping — the end-to-end fundamentals.

License

MIT — see LICENSE.