> ## Documentation Index
> Fetch the complete documentation index at: https://bulkgrid.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# SDK

> Use the Bulkgrid TypeScript and Python SDKs from applications and workers.

Choose [TypeScript](#typescript) for Node.js or [Python](#python) for synchronous and asynchronous Python applications.

## TypeScript

Use the TypeScript SDK from your Node.js application or worker.

### Install

```bash theme={null}
npm install @bulkgrid/sdk@^0.2.0
```

Create a key with the scopes your workflow needs, then configure the client:

```js theme={null}
import { BulkgridClient } from '@bulkgrid/sdk';

const client = new BulkgridClient({
  apiKey: process.env.BULKGRID_API_KEY ?? '',
  baseUrl: process.env.BULKGRID_BASE_URL ?? 'https://bulkgrid.com',
});
```

The SDK uses API-key authentication. Do not pass an OAuth token as `apiKey`.

### Search indexed content

Requires `search:query` and accessible indexed content:

```js theme={null}
const response = await client.search({ query: 'authentication setup', limit: 5 });
console.log(response.results);
```

### Find a source and add it to a collection

Requires `sources:read`, `sources:write`, `collections:read`, `collections:write`, and `search:query`. Plan limits and collection restrictions still apply; a `403` is not a reason to retry with broader credentials automatically.

```js theme={null}
// Search the public catalog and sources owned by your workspace.
const { sources: matches } = await client.sources.search('https://example.com');
console.log(
  matches.map(source => ({
    identifier: source.identifier,
    subscribed: source.isSubscribed,
    owned: source.isOwned,
  })),
);

// For a website, create it or subscribe to the existing shared source.
const { source, created } = await client.sources.create({
  type: 'domain',
  identifier: 'https://example.com',
  visibility: 'public',
});
console.log({ sourceId: source.id, created });

const { collection } = await client.collections.create({ name: 'Research' });
await client.collections.addSource({ collectionId: collection.id, sourceId: source.id });

const status = await client.sources.status(source.id);
if (status.indexedItems > 0) {
  const response = await client.search({ query: 'authentication setup', collectionId: collection.id });
  console.log(response.executedQueries, response.results);
}
```

`created: false` means the source already existed. Adding a subscription does not guarantee indexing is complete. `sources.list({ search })` searches your subscribed and owned sources; `sources.search(query)` searches the public catalog and owned sources, with subscription flags.

Collection creation requires a qualifying plan. To use an existing collection, call `collections.list()` and select an appropriate accessible collection instead. `addSource` includes the whole source and keeps existing exclusions. Use `updateRules` for specific folders or URLs:

```js theme={null}
await client.collections.updateRules(collection.id, {
  add: [{ action: 'exclude', target: 'folder', source_id: source.id, target_id: '/archive' }],
});
const { rules, counts } = await client.collections.rules(collection.id);
console.log(rules, counts);
```

`removeSource({ collectionId, sourceId })` removes the source's rules returned by an initial read. It requires both `collections:read` and `collections:write`; rules added concurrently after that read are left alone. Removing a source from a collection does not remove its workspace subscription.

### GitHub sources and indexing

```js theme={null}
const { source: githubSource, run } = await client.sources.createGithub({
  repositoryUrl: 'https://github.com/example/docs',
  includePaths: ['docs/**'],
});
console.log(githubSource.id, run.id);

const page = await client.sources.documents(githubSource.id, { parent: '/', limit: 50 });
console.log(page.documents, page.nextOffset);
```

GitHub sources require a qualifying plan and are private to the workspace. Source creation returns before ingestion completes. `sources.recrawl(id)` requests a refresh and needs `sources:write` plus `runs:write`. `sources.changes(id, { runId, kind: 'changed' })` lists changes for a specific run; paginate with `offset` and the returned `nextOffset`.

### Create and monitor a crawl

Requires `runs:write`, `runs:read`, and `results:read`:

```js theme={null}
const run = await client.crawl({
  urls: ['https://example.com'],
  options: { formats: ['markdown'] },
});

const current = await client.runs.get(run.id);
console.log(current.status);
```

Poll until `completed`, `failed`, or `cancelled`, with a maximum wait time. Once complete:

```js theme={null}
const { results } = await client.runs.results(run.id, { page: 1, limit: 20 });
const firstResult = results.find(result => !result.error_message);
if (firstResult) {
  const markdown = await client.runs.getResultContent(run.id, firstResult.id, { type: 'markdown' });
  console.log(markdown);
}
```

### Available methods

| Method                               | Purpose                                        |
| ------------------------------------ | ---------------------------------------------- |
| `search(input)`                      | Search indexed content                         |
| `crawl(input)`                       | Crawl known URLs                               |
| `deepCrawl(input)`                   | Traverse from a starting URL                   |
| `extract(input)`                     | Extract structured fields                      |
| `runs.create(input)`                 | Create a run with an explicit `type`           |
| `runs.list()` / `runs.get(id)`       | Inspect runs                                   |
| `runs.cancel(id)` / `runs.retry(id)` | Stop or retry work                             |
| `runs.delete(id)`                    | Request asynchronous deletion of a stopped run |
| `runs.results(id)`                   | List result records                            |
| `runs.getResultContent(...)`         | Retrieve text content                          |
| `runs.getResultScreenshot(...)`      | Retrieve a signed screenshot URL               |

`crawl`, `deepCrawl`, and `extract` add the required run `type` automatically. Raw HTTP calls and `runs.create` must include it explicitly.

| Source methods                                                | Collection methods                                            |
| ------------------------------------------------------------- | ------------------------------------------------------------- |
| `sources.list(query?)`, `sources.search(query)`               | `collections.list()`                                          |
| `sources.create(input)`, `sources.createGithub(input)`        | `collections.create(input)`                                   |
| `sources.get(id)`, `sources.update(id, input)`                | `collections.get(id)`, `collections.update(id, input)`        |
| `sources.delete(id)`                                          | `collections.delete(id)`                                      |
| `sources.status(id)`, `sources.recrawl(id)`                   | `collections.addSource({ collectionId, sourceId })`           |
| `sources.documents(id, query?)`, `sources.changes(id, query)` | `collections.removeSource({ collectionId, sourceId })`        |
|                                                               | `collections.rules(id)`, `collections.updateRules(id, input)` |

`sources.delete` removes your subscription to a shared source or deletes an owned source according to server permissions. See [Source management](/docs/sources).

### Handle failures

```js theme={null}
import { BulkgridApiError } from '@bulkgrid/sdk';

try {
  await client.search({ query: 'release notes' });
} catch (error) {
  if (error instanceof BulkgridApiError) {
    console.error(error.status, error.data);
  }
  throw error;
}
```

Use `RequestOptions.signal` to cancel requests. The SDK does not automatically retry requests or poll runs. See [Retries](/docs/reliability-and-retries) before retrying a create request that may already have succeeded.

### Analyze a source before ingestion

Source analysis is asynchronous and may start a resource-consuming crawl. It requires `sources:discover` and `runs:write`; start it only within the approved source scope. These methods are available in the repository build; verify your installed package includes them.

```js theme={null}
const analysis = await client.sources.analyze({ url: 'https://example.com' });
console.log(analysis.id, analysis.runId, analysis.status);

// Check later using the analysis ID, not the crawl run ID.
const progress = await client.sources.getAnalysis(analysis.id);
console.log(progress.status, progress.urlCount, progress.paths, progress.warning);
```

Reading analysis requires `sources:discover`. Counts describe estimated coverage, not guaranteed storage or credits. Treat `partial` results and warnings as incomplete evidence. The SDK does not poll indefinitely or create a source subscription automatically.

After approved creation, use `sources.status(sourceId)` to distinguish source registration from indexing readiness. Use `collections.rules(collectionId)` before adding whole-source access with `collections.addSource({ sourceId, collectionId })`. A failed collection update does not undo source creation; preserve the returned source ID rather than repeating creation.

## Python

The Python SDK supports Python 3.10+ and offers the same source, collection, crawl, extraction, search, and run operations as the TypeScript SDK.

<Note>
  The Python package is prepared for its first PyPI release. Until it is published, install it from a local repository checkout with `python -m pip install ./packages/sdk-python`. After publication, use `python -m pip install bulkgrid`.
</Note>

### Search

```python theme={null}
import os
from bulkgrid import BulkgridClient

with BulkgridClient(api_key=os.environ["BULKGRID_API_KEY"]) as client:
    response = client.search({"query": "authentication setup", "limit": 5})
    print(response["results"])
```

Use a scoped API key. The default `base_url` is `https://bulkgrid.com`; you can override it for your deployment. Request and response dictionaries retain the API's field spelling, such as `collectionId` and `source_id`. Method names use Python conventions: `deep_crawl`, `sources.get_analysis`, `sources.create_github`, and `runs.get_result_content`.

### Async applications

```python theme={null}
import asyncio
import os
from bulkgrid import AsyncBulkgridClient


async def main():
    async with AsyncBulkgridClient(api_key=os.environ["BULKGRID_API_KEY"]) as client:
        response = await client.search({"query": "release notes", "limit": 3})
        print(response["results"])


asyncio.run(main())
```

Every sync operation has an async equivalent. Use context managers to release HTTP connections, or call `close()` / `await aclose()` explicitly.

### Types and errors

Request and response types are available from `bulkgrid.types`. Responses remain regular Python dictionaries; type hints support editor completion and static checking. The server validates requests; the SDK checks HTTP status and JSON object responses.

Catch `BulkgridAPIError` for non-success HTTP responses. It exposes `status_code`, `data`, `request_id`, and `retry_after`. `BulkgridTimeoutError` and `BulkgridConnectionError` identify network failures; `BulkgridResponseError` identifies invalid JSON responses.

The default network timeout is 60 seconds; configure it with `timeout=30.0`, for example. It does not wait for a crawl or extraction run to finish. Requests do not retry automatically: a timed-out write may already have started processing. Persist run and analysis IDs, use bounded polling, and inspect status before retrying.

### Sources, collections, and runs

The [Sources](/docs/sources), [Collections](/docs/collections), and [Runs and results](/docs/runs-and-results) guides include Python tabs. For example, analyze a source with `client.sources.analyze({"url": "https://example.com"})` and check it with `client.sources.get_analysis(analysis_id)`.

Use `client.collections.add_source(collection_id, source_id)` to include a source in a collection. `remove_source(collection_id, source_id)` removes all rules for that source, matching the TypeScript helper.

`client.runs.results(run_id, {"page": 1, "limit": 20})` returns one page of results. Text content comes from `get_result_content`; screenshot URLs come from `get_result_screenshot`, and `download_result_screenshot` returns image bytes. Check per-result failures and pagination metadata before consuming outputs.