Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.oathnet.org/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

OathNet uses cursor-based pagination for efficient retrieval of large datasets. This approach provides:
  • Consistent results - No duplicates or missed records when data changes
  • Better performance - Faster than offset-based pagination for large datasets
  • Stable ordering - Results maintain consistent order across pages
For new integrations, prefer the v2 search endpoints. Legacy endpoints may return nextCursorMark, while v2 endpoints return next_cursor.

​
How It Works

  1. Make your initial request
  2. Check for nextCursorMark (or next_cursor) in the response
  3. Pass the cursor as the cursor parameter in your next request
  4. Repeat until the cursor is null

​
Basic Example

import requests

def fetch_all_results(query, api_key):
    """Fetch all paginated results for a query."""
    all_results = []
    cursor = None

    while True:
        params = {"q": query}
        if cursor:
            params["cursor"] = cursor

        response = requests.get(
            "https://oathnet.org/api/service/v2/breach/search",
            params=params,
            headers={"x-api-key": api_key}
        ).json()

        if not response["success"]:
            raise Exception(response["message"])

        data = response["data"]
        all_results.extend(data["items"])

        print(f"Fetched {len(data['items'])} results "
              f"({len(all_results)}/{data['meta']['total']} total)")

        # Check for more pages
        cursor = data.get("next_cursor") or data.get("nextCursorMark")
        if not cursor:
            break

    return all_results

# Usage
results = fetch_all_results("user@example.com", API_KEY)
print(f"Total results: {len(results)}")

​
Response Structure

​
V2 Search Response

{
  "success": true,
  "message": "Request completed successfully",
  "data": {
    "items": [...],
    "meta": {
      "count": 100,
      "total": 1500,
      "has_more": true
    },
    "next_cursor": "AoJw3ZD..."
  }
}
FieldDescription
results_foundTotal matching records
results_shownRecords in this response
nextCursorMarkCursor for next page (null if last page)
next_cursor_markAlias for nextCursorMark

​
V2 Endpoints (Stealer, Victims)

{
  "data": {
    "items": [...],
    "meta": {
      "count": 25,
      "total": 1500,
      "has_more": true,
      "total_pages": 60
    },
    "next_cursor": "eyJsYXN0X2lkIjoiZG9jXzAwMSJ9"
  }
}
FieldDescription
meta.countRecords in this response
meta.totalTotal matching records
meta.has_moreWhether more pages exist
meta.total_pagesEstimated total pages
next_cursorCursor for next page

​
Controlling Page Size

Use page_size to control results per page: 
# Get 50 results per page (default is 25)
response = requests.get(
    "https://oathnet.org/api/service/v2/stealer/search",
    params={
        "q": "user@example.com",
        "page_size": 50
    },
    headers={"x-api-key": API_KEY}
)
Endpoint TypeDefaultMaximum
Breach Search1001000
V2 Stealer25100
V2 Victims25100

​
Pagination with Filters

Cursors work with all filter parameters: 
cursor = None

while True:
    response = requests.get(
        "https://oathnet.org/api/service/v2/stealer/search",
        params={
            "domain[]": "google.com",
            "from": "2024-01-01",
            "to": "2024-06-30",
            "page_size": 50,
            "cursor": cursor
        },
        headers={"x-api-key": API_KEY}
    ).json()

    # Process results...

    cursor = response["data"].get("next_cursor")
    if not cursor:
        break
Don’t change filters between pages. The cursor is specific to the original query. Changing filters will cause unexpected results.

​
Best Practices

For large datasets, process results as you fetch them instead of loading everything into memory:
def process_results_stream(query, api_key, processor):
    """Process results incrementally."""
    cursor = None
    total_processed = 0

    while True:
        params = {"q": query}
        if cursor:
            params["cursor"] = cursor

        response = requests.get(
            "https://oathnet.org/api/service/v2/breach/search",
            params=params,
            headers={"x-api-key": api_key}
        ).json()

        for result in response["data"]["items"]:
            processor(result)  # Process each result
            total_processed += 1

        cursor = response["data"].get("next_cursor") or response["data"].get("nextCursorMark")
        if not cursor:
            break

    return total_processed
Add delays between pagination requests to avoid rate limits:
import time

cursor = None
while True:
    response = fetch_page(query, cursor)
    process_results(response["data"]["results"])

    cursor = response["data"].get("nextCursorMark")
    if not cursor:
        break

    # Small delay between pages
    time.sleep(0.1)
For long-running jobs, save cursors to resume if interrupted:
import json

STATE_FILE = "pagination_state.json"

def save_state(cursor, processed_count):
    with open(STATE_FILE, "w") as f:
        json.dump({
            "cursor": cursor,
            "processed": processed_count
        }, f)

def load_state():
    try:
        with open(STATE_FILE) as f:
            return json.load(f)
    except FileNotFoundError:
        return {"cursor": None, "processed": 0}

# Resume from saved state
state = load_state()
cursor = state["cursor"]
processed = state["processed"]
Set a maximum number of results to fetch:
def fetch_limited_results(query, api_key, max_results=1000):
    """Fetch up to max_results records."""
    all_results = []
    cursor = None

    while len(all_results) < max_results:
        response = fetch_page(query, cursor)
        results = response["data"]["results"]

        remaining = max_results - len(all_results)
        all_results.extend(results[:remaining])

        cursor = response["data"].get("nextCursorMark")
        if not cursor:
            break

    return all_results

​
Common Issues

​
Cursor Expired

Cursors may expire after a period of inactivity: 
{
  "success": false,
  "message": "Cursor expired or invalid",
  "errors": {
    "cursor": "The provided cursor is no longer valid"
  }
}
Solution: Start pagination from the beginning.

​
Results Changed

If data is updated between pagination requests, you may see slightly different totals. This is normal and doesn’t affect result consistency.

​
Next Steps

Rate Limiting

Understand quotas when paginating large datasets