Extraction

Use extraction when you want fields, not just content.

When extraction is the right tool

Extraction is a strong fit for:

company and product profiles
pricing or policy extraction
structured enrichment for downstream systems
repeatable data collection from public pages

Design the request carefully

Good extraction quality usually depends more on request design than on retry count. Keep the request:

narrow enough to be realistic
specific about what should be extracted
backed by a schema that downstream systems can actually use

Generate a schema

Write the JSON schema yourself, as in the examples below, or generate a draft from your extraction goal. Schema generation requires the extract:schema:generate scope. Node.js and Python examples use the Bulkgrid SDKs. Set BULKGRID_API_KEY in your backend environment; cURL examples also use BULKGRID_BASE_URL=https://bulkgrid.com.

cURL

curl "$BULKGRID_BASE_URL/api/v1/extract/schema" \
  -H 'Content-Type: application/json' \
  -H "x-api-key: $BULKGRID_API_KEY" \
  -d '{"query":"Company name and product summary","allowNulls":true}'

Schema generation is currently available through the REST API; the SDK has no schema-generation method. The response contains schema and optional notes. Review the result before passing schema to /api/v1/extract. Provide a query, optional context, and allowNulls to control whether nulls are allowed. Generated schemas are validated before being returned; if validation fails, the service retries once and then returns a structured error. Review and tighten the draft, then test it with a small extraction run before expanding its scope. Schema generation prepares the request; submit the extraction separately to process your URLs.

Request examples

import { BulkgridClient } from '@bulkgrid/sdk';

const client = new BulkgridClient({
  apiKey: process.env.BULKGRID_API_KEY ?? '',
  baseUrl: process.env.BULKGRID_BASE_URL ?? 'https://bulkgrid.com',
});

const run = await client.extract({
  urls: ['https://example.com', 'https://example.com/pricing'],
  query: 'Extract company name, product summary, and pricing details',
  schema: {
    type: 'object',
    properties: {
      companyName: { type: 'string' },
      productSummary: { type: 'string' },
      pricing: { type: 'string' },
    },
    required: ['companyName'],
  },
  maxRetries: 3,
});

import os
from bulkgrid import BulkgridClient

with BulkgridClient(
    api_key=os.environ["BULKGRID_API_KEY"],
    base_url=os.environ.get("BULKGRID_BASE_URL", "https://bulkgrid.com"),
) as client:
    data = client.extract(
        {
            "urls": ["https://example.com", "https://example.com/pricing"],
            "query": "Extract company name, product summary, and pricing details",
            "schema": {
                "type": "object",
                "properties": {
                    "companyName": {"type": "string"},
                    "productSummary": {"type": "string"},
                    "pricing": {"type": "string"},
                },
                "required": ["companyName"],
            },
            "maxRetries": 3,
        }
    )
    print(data)

curl "$BULKGRID_BASE_URL/api/v1/extract" \
  -H 'Content-Type: application/json' \
  -H "x-api-key: $BULKGRID_API_KEY" \
  -d '{
    "type": "extract",
    "urls": [
      "https://example.com",
      "https://example.com/pricing"
    ],
    "query": "Extract company name, product summary, and pricing details",
    "schema": {
      "type": "object",
      "properties": {
        "companyName": { "type": "string" },
        "productSummary": { "type": "string" },
        "pricing": { "type": "string" }
      },
      "required": ["companyName"]
    },
    "maxRetries": 3
  }'

Workflow

submit the extraction request
store the run ID
poll GET /api/v1/runs/{runId}
fetch GET /api/v1/runs/{runId}/results
read extraction_data from the result records

Common quality problems

the schema asks for data the source does not contain
the query is too broad
the page requires interaction or access patterns the request does not account for

Recommendation

Start with the smallest schema that delivers value. Expand later once the output is stable.

Get started

Index

More

Access and operations

When extraction is the right tool

Design the request carefully

Generate a schema

Request examples

Workflow

Common quality problems

Recommendation

​When extraction is the right tool

​Design the request carefully

​Generate a schema

​Request examples

​Workflow

​Common quality problems

​Recommendation

When extraction is the right tool

Design the request carefully

Generate a schema

Request examples

Workflow

Common quality problems

Recommendation