A good web scraping brief describes the decision you need to make and the data needed to make it. You do not need to select proxies or write a technical specification. You do need to identify representative sources, required fields, coverage, freshness, and the expected output.
The most useful starting material is a few real URLs plus a small example of the spreadsheet or record you want back. Include an awkward page as well as an easy one.
Explain the job in one sentence
Use a concrete outcome: “We need to identify independent businesses offering commercial HVAC maintenance in three counties.” That is more useful than “We need leads.” For products, explain whether the goal is a catalog import, competitive pricing, or stock alerts.
Describe what happens after delivery. Someone reviewing a spreadsheet needs different information from an application automatically updating prices. The latter also needs stronger identity, validation, and exception controls.
Supply representative sources and coverage
List the websites and example pages. Say whether you already have all the URLs or need the collector to discover records. Explain category, geography, date, and exclusion boundaries.
If access requires your account, identify that requirement early so the provider can review authorized access and handling. A publicly visible field and a field available inside your own account are different collection contexts.
Do not estimate volume solely from a headline search-result count. Ask the provider to validate achievable coverage during discovery and identify known gaps.
Define one row and every important field
For each field, give its meaning and an example. “Price” might mean the displayed sale price, a seller's offer, or the price including delivery. “Company” might mean a legal entity, brand, or individual branch.
An illustrative product brief could request product identifier, variant, retailer, displayed price, currency, availability, source URL, and observation time. A lead brief could request domain, location, relevant service, qualification status, supporting URL, and check date.
State what missing information should look like. Blank, unknown, not applicable, and collection failed are not interchangeable. Avoid requiring guesses to fill every cell.
Specify timing and delivery
Say whether you need a snapshot or recurring collection, and explain how old the data can be. Include the delivery destination and whether previous observations must remain available.
For a file, specify CSV, spreadsheet, or JSON and any import requirements. For an integration, identify the receiving system, permissions, record keys, and whether new data creates records or updates existing ones. Credentials should be transferred through an agreed secure channel, not pasted into an initial enquiry.
Agree what acceptance looks like
Request a representative sample before the full run. Check whether rows represent the right entity, values mean what you expected, and difficult cases are handled consistently. Agree how completeness, duplicates, errors, and source changes will be reported.
Your brief is ready when another person can explain what a successful delivery would contain without inventing missing rules. Send your brief to Lightning Digital, or review how our managed scraping service works.
