> ## Documentation Index
> Fetch the complete documentation index at: https://allhandsai-cookbook-preview-pr-13.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Conversation Metrics

> Retrieve cost and token usage metrics for conversations via CLI tool.

<Card title="View source on GitHub" icon="github" href="https://github.com/OpenHands/enterprise-cookbook/tree/b73c7c4dec2be0f1ed2258deebc2e49013064c38/conversation-metrics" horizontal />

CLI tool to retrieve cost and token usage metrics for OpenHands conversations.

Supports both **V0** and **V1** APIs, automatically selecting the appropriate one based on the conversation version.

## Features

* **Auto-detects** conversation version (V0 vs V1) and uses the appropriate API
* **Graceful fallback** chain for metric retrieval
* **Displays** cost (USD), token counts, cache stats, and context window
* **JSON output** for programmatic use
* **API call logging** for debugging and development
* **Fixture-based testing** - high coverage via recorded API responses
* **Zero dependencies** - uses only Python standard library

## Installation

No installation required - just make the script executable:

```bash theme={null}
chmod +x oh-metrics
```

Requires Python 3.10+.

## Usage

### Set your API key

```bash theme={null}
export OH_API_KEY="your-api-key"
```

### Get metrics for a conversation

```bash theme={null}
./oh-metrics <conversation_id>
```

**Example output (V0 conversation):**

```text theme={null}
────────────────────────────────────────────────────────────
Conversation: 7f3d57c4e5b2434d9ca78e5e27311137
Title: Hello and a Programming Joke
API Version: V0 via V0 (events)
────────────────────────────────────────────────────────────
💰 Total Cost: $0.083175 USD

📊 Token Usage:
   Prompt tokens:     12,865
   Completion tokens: 117
   Total tokens:      12,982

🗄️  Cache:
   Cache read:        0
   Cache write:       12,740

📐 Context window:    200,000
────────────────────────────────────────────────────────────
```

**Example output (V1 conversation):**

```text theme={null}
────────────────────────────────────────────────────────────
Conversation: 72d40619b8534f9b9de6c3f17a71072d
Title: 📝 V0 API for Conversation Costs & Tokens
API Version: V1 via V1 (events)
────────────────────────────────────────────────────────────
💰 Total Cost: $13.576675 USD

📊 Token Usage:
   Prompt tokens:     17,577,146
   Completion tokens: 63,985
   Total tokens:      17,641,131

🗄️  Cache:
   Cache read:        17,022,477
   Cache write:       553,973

🧠 Reasoning tokens:  1,169

📐 Context window:    200,000
────────────────────────────────────────────────────────────
```

### JSON output

```bash theme={null}
./oh-metrics <conversation_id> --json
```

```json theme={null}
{
  "conversation_id": "7f3d57c4e5b2434d9ca78e5e27311137",
  "title": "Hello and a Programming Joke",
  "api_version": "V0",
  "api_used": "V0 (events)",
  "metrics": {
    "accumulated_cost": 0.083175,
    "accumulated_token_usage": {
      "prompt_tokens": 12865,
      "completion_tokens": 117,
      "cache_read_tokens": 0,
      "cache_write_tokens": 12740,
      "context_window": 200000
    }
  }
}
```

### Options

| Option | Short | Description |
| - | - | - |
| `--api-key KEY` | `-k` | API key (defaults to `OH_API_KEY` env var) |
| `--base-url URL` | `-u` | Base URL (default: `https://app.all-hands.dev`) |
| `--json` | `-j` | Output in JSON format |
| `--log-api-calls` | `-l` | Log all API requests/responses to `.oh/api-logs/` |
| `--help` | `-h` | Show help message |

### Logging API Calls

For debugging or development, you can log all API requests and responses:

```bash theme={null}
./oh-metrics <conversation_id> --log-api-calls
```

This creates timestamped files in `.oh/api-logs/YYYYMMDD-HHMMSS/`:

```text theme={null}
.oh/api-logs/20260227-145230/
├── 0001-request.json
├── 0001-response.json
├── 0002-request.json
└── 0002-response.json
```

Each request/response pair is numbered sequentially. The Authorization header is redacted from logged requests.

## API Endpoints Used

### For V1 Conversations

| Endpoint | Purpose |
| - | - |
| `GET /api/conversations/{id}` | Get conversation info (to determine version) |
| `GET /api/v1/app-conversations?ids={id}` | Get conversation with metrics |
| `GET /api/v1/conversation/{id}/events/search` | Fallback: find metrics in `ConversationStateUpdateEvent` |

### For V0 Conversations

| Endpoint | Purpose |
| - | - |
| `GET /api/conversations/{id}` | Get conversation info |
| `GET /api/conversations/{id}/events` | Get events with `llm_metrics` |
| `GET /api/conversations/{id}/trajectory` | Fallback for metrics |

## How Metrics Are Retrieved

The tool uses a fallback chain to find metrics:

1. **Check conversation version** via `/api/conversations/{id}`
2. **For V1 conversations**:
   * First try `/api/v1/app-conversations?ids={id}` which includes a `metrics` object
   * If metrics are all zeros, fall back to `/api/v1/conversation/{id}/events/search` and extract metrics from `ConversationStateUpdateEvent` at `value.stats.usage_to_metrics.agent`
3. **For V0 conversations** (or if V1 fails): Use `/api/conversations/{id}/events` and find the latest event with `llm_metrics`
4. **Last resort**: Use `/api/conversations/{id}/trajectory` and scan for `llm_metrics`

> **Note:** Some V1 conversations have metrics stored only in events (not in the app-conversations response). The fallback chain ensures these are still retrieved correctly.

## Metrics Explained

| Field | Description |
| - | - |
| `accumulated_cost` | Total cost in USD |
| `prompt_tokens` | Tokens sent to the LLM |
| `completion_tokens` | Tokens generated by the LLM |
| `cache_read_tokens` | Tokens read from prompt cache |
| `cache_write_tokens` | Tokens written to prompt cache |
| `reasoning_tokens` | Tokens used for reasoning (if applicable) |
| `context_window` | Model's context window size |

## Architecture

The library is organized into separate modules:

```text theme={null}
oh_api/
├── __init__.py    # Public API exports
├── client.py      # Base HTTP client with logging and fixture support
├── v0.py          # V0 API driver
├── v1.py          # V1 API driver
└── metrics.py     # High-level metrics retrieval with fallback chain
```

### Using the Library Programmatically

```python theme={null}
from oh_api import APIClient, get_conversation_metrics

# Create a client
client = APIClient(
    base_url="https://app.all-hands.dev",
    api_key="your-api-key",
    log_api_calls=True,  # Optional: enable logging
)

# Get metrics
metrics = get_conversation_metrics(client, "conversation-id")
if metrics:
    print(f"Cost: ${metrics.accumulated_cost:.6f}")
    print(f"Tokens: {metrics.total_tokens:,}")
```

### Using the V0/V1 Drivers Directly

```python theme={null}
from oh_api import APIClient
from oh_api.v0 import V0Driver
from oh_api.v1 import V1Driver

client = APIClient(base_url="...", api_key="...")

# V0 API
v0 = V0Driver(client)
conv = v0.get_conversation("conv-id")
events = v0.get_events("conv-id", limit=50)

# V1 API
v1 = V1Driver(client)
conv = v1.get_conversation("conv-id")
if conv and conv.metrics:
    print(f"Cost: ${conv.metrics.accumulated_cost}")
```

## Testing

The test suite uses recorded API fixtures to achieve high coverage without making real API calls:

```bash theme={null}
# Run tests
uv run pytest tests/ -v

# Run with coverage
uv run pytest tests/ --cov --cov-report=term-missing
```

Current coverage: **89%** of library code.

### Creating New Fixtures

1. Run the CLI with `--log-api-calls` to capture real API responses
2. Copy relevant response files to `tests/fixtures/`
3. Rename following the pattern: `GET__api_path_q_param=value.json`

## License

MIT License - see [LICENSE](https://github.com/OpenHands/enterprise-cookbook/blob/b73c7c4dec2be0f1ed2258deebc2e49013064c38/LICENSE) for details.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.