> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oathnet.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Pagination

> Navigate large result sets with cursor-based pagination

## Overview

OathNet uses **cursor-based pagination** for efficient retrieval of large datasets. This approach provides:

* **Consistent results** - No duplicates or missed records when data changes
* **Better performance** - Faster than offset-based pagination for large datasets
* **Stable ordering** - Results maintain consistent order across pages

<Note>
  For new integrations, prefer the v2 search endpoints. Some older `/service/*` routes return `nextCursorMark`, while v2 endpoints return `next_cursor`.
</Note>

## How It Works

1. Make your initial request
2. Check for `nextCursorMark` (or `next_cursor`) in the response
3. Pass the cursor as the `cursor` parameter in your next request
4. Repeat until the cursor is `null`

## Basic Example

<CodeGroup>
  ```python Python theme={null}
  import requests

  def fetch_all_results(query, api_key):
      """Fetch all paginated results for a query."""
      all_results = []
      cursor = None

      while True:
          params = {"q": query}
          if cursor:
              params["cursor"] = cursor

          response = requests.get(
              "https://oathnet.org/api/service/v2/breach/search",
              params=params,
              headers={"x-api-key": api_key}
          ).json()

          if not response["success"]:
              raise Exception(response["message"])

          data = response["data"]
          all_results.extend(data["items"])

          print(f"Fetched {len(data['items'])} results "
                f"({len(all_results)}/{data['meta']['total']} total)")

          # Check for more pages
          cursor = data.get("next_cursor") or data.get("nextCursorMark")
          if not cursor:
              break

      return all_results

  # Usage
  results = fetch_all_results("user@example.com", API_KEY)
  print(f"Total results: {len(results)}")
  ```

  ```javascript Node.js theme={null}
  async function fetchAllResults(query, apiKey) {
    const allResults = [];
    let cursor = null;

    while (true) {
      const params = new URLSearchParams({ q: query });
      if (cursor) params.set("cursor", cursor);

      const response = await fetch(
        `https://oathnet.org/api/service/v2/breach/search?${params}`,
        { headers: { "x-api-key": apiKey } }
      );
      const json = await response.json();

      if (!json.success) {
        throw new Error(json.message);
      }

      const { data } = json;
      allResults.push(...data.items);

      console.log(`Fetched ${data.items.length} results ` +
                  `(${allResults.length}/${data.meta.total} total)`);

      // Check for more pages
      cursor = data.next_cursor || data.nextCursorMark;
      if (!cursor) break;
    }

    return allResults;
  }

  // Usage
  const results = await fetchAllResults("user@example.com", API_KEY);
  console.log(`Total results: ${results.length}`);
  ```

  ```go Go theme={null}
  package main

  import (
      "encoding/json"
      "fmt"
      "net/http"
      "net/url"
  )

  func fetchAllResults(query, apiKey string) ([]map[string]interface{}, error) {
      var allResults []map[string]interface{}
      cursor := ""

      for {
          params := url.Values{"q": {query}}
          if cursor != "" {
              params.Set("cursor", cursor)
          }

          req, _ := http.NewRequest("GET",
              "https://oathnet.org/api/service/v2/breach/search?"+params.Encode(), nil)
          req.Header.Add("x-api-key", apiKey)

          client := &http.Client{}
          resp, err := client.Do(req)
          if err != nil {
              return nil, err
          }

          var data map[string]interface{}
          json.NewDecoder(resp.Body).Decode(&data)
          resp.Body.Close()

          results := data["data"].(map[string]interface{})["items"].([]interface{})
          for _, r := range results {
              allResults = append(allResults, r.(map[string]interface{}))
          }

          // Check for next page
          if next, ok := data["data"].(map[string]interface{})["next_cursor"]; ok && next != nil {
              cursor = next.(string)
          } else {
              break
          }
      }

      return allResults, nil
  }
  ```
</CodeGroup>

## Response Structure

### V2 Search Response

```json theme={null}
{
  "success": true,
  "message": "Request completed successfully",
  "data": {
    "items": [...],
    "meta": {
      "count": 100,
      "total": 1500,
      "has_more": true
    },
    "next_cursor": "AoJw3ZD..."
  }
}
```

| Field              | Description                              |
| ------------------ | ---------------------------------------- |
| `results_found`    | Total matching records                   |
| `results_shown`    | Records in this response                 |
| `nextCursorMark`   | Cursor for next page (null if last page) |
| `next_cursor_mark` | Alias for `nextCursorMark`               |

### V2 Endpoints (Stealer, Victims)

```json theme={null}
{
  "data": {
    "items": [...],
    "meta": {
      "count": 25,
      "total": 1500,
      "has_more": true,
      "total_pages": 60
    },
    "next_cursor": "eyJsYXN0X2lkIjoiZG9jXzAwMSJ9"
  }
}
```

| Field              | Description              |
| ------------------ | ------------------------ |
| `meta.count`       | Records in this response |
| `meta.total`       | Total matching records   |
| `meta.has_more`    | Whether more pages exist |
| `meta.total_pages` | Estimated total pages    |
| `next_cursor`      | Cursor for next page     |

## Controlling Page Size

Use `page_size` to control results per page:

```python theme={null}
# Get 50 results per page (default is 25)
response = requests.get(
    "https://oathnet.org/api/service/v2/stealer/search",
    params={
        "q": "user@example.com",
        "page_size": 50
    },
    headers={"x-api-key": API_KEY}
)
```

| Endpoint Type | Default | Maximum |
| ------------- | ------- | ------- |
| Breach Search | 100     | 1000    |
| V2 Stealer    | 25      | 100     |
| V2 Victims    | 25      | 100     |

## Pagination with Filters

Cursors work with all filter parameters:

```python theme={null}
cursor = None

while True:
    response = requests.get(
        "https://oathnet.org/api/service/v2/stealer/search",
        params={
            "domain[]": "google.com",
            "from": "2024-01-01",
            "to": "2024-06-30",
            "page_size": 50,
            "cursor": cursor
        },
        headers={"x-api-key": API_KEY}
    ).json()

    # Process results...

    cursor = response["data"].get("next_cursor")
    if not cursor:
        break
```

<Warning>
  **Don't change filters between pages.** The cursor is specific to the original query. Changing filters will cause unexpected results.
</Warning>

## Best Practices

<AccordionGroup>
  <Accordion title="Process Results Incrementally" icon="stream">
    For large datasets, process results as you fetch them instead of loading everything into memory:

    ```python theme={null}
    def process_results_stream(query, api_key, processor):
        """Process results incrementally."""
        cursor = None
        total_processed = 0

        while True:
            params = {"q": query}
            if cursor:
                params["cursor"] = cursor

            response = requests.get(
                "https://oathnet.org/api/service/v2/breach/search",
                params=params,
                headers={"x-api-key": api_key}
            ).json()

            for result in response["data"]["items"]:
                processor(result)  # Process each result
                total_processed += 1

            cursor = response["data"].get("next_cursor") or response["data"].get("nextCursorMark")
            if not cursor:
                break

        return total_processed
    ```
  </Accordion>

  <Accordion title="Handle Rate Limits Between Pages" icon="clock">
    Add delays between pagination requests to avoid rate limits:

    ```python theme={null}
    import time

    cursor = None
    while True:
        response = fetch_page(query, cursor)
        process_results(response["data"]["results"])

        cursor = response["data"].get("nextCursorMark")
        if not cursor:
            break

        # Small delay between pages
        time.sleep(0.1)
    ```
  </Accordion>

  <Accordion title="Save Cursors for Resumption" icon="floppy-disk">
    For long-running jobs, save cursors to resume if interrupted:

    ```python theme={null}
    import json

    STATE_FILE = "pagination_state.json"

    def save_state(cursor, processed_count):
        with open(STATE_FILE, "w") as f:
            json.dump({
                "cursor": cursor,
                "processed": processed_count
            }, f)

    def load_state():
        try:
            with open(STATE_FILE) as f:
                return json.load(f)
        except FileNotFoundError:
            return {"cursor": None, "processed": 0}

    # Resume from saved state
    state = load_state()
    cursor = state["cursor"]
    processed = state["processed"]
    ```
  </Accordion>

  <Accordion title="Limit Total Results" icon="hand">
    Set a maximum number of results to fetch:

    ```python theme={null}
    def fetch_limited_results(query, api_key, max_results=1000):
        """Fetch up to max_results records."""
        all_results = []
        cursor = None

        while len(all_results) < max_results:
            response = fetch_page(query, cursor)
            results = response["data"]["results"]

            remaining = max_results - len(all_results)
            all_results.extend(results[:remaining])

            cursor = response["data"].get("nextCursorMark")
            if not cursor:
                break

        return all_results
    ```
  </Accordion>
</AccordionGroup>

## Common Issues

### Cursor Expired

Cursors may expire after a period of inactivity:

```json theme={null}
{
  "success": false,
  "message": "Cursor expired or invalid",
  "errors": {
    "cursor": "The provided cursor is no longer valid"
  }
}
```

**Solution:** Start pagination from the beginning.

### Results Changed

If data is updated between pagination requests, you may see slightly different totals. This is normal and doesn't affect result consistency.

## Next Steps

<CardGroup cols={2}>
  <Card title="Rate Limiting" icon="gauge-high" href="/guides/rate-limiting">
    Understand quotas when paginating large datasets
  </Card>
</CardGroup>