Skip to content

Commit 3761b6c

Browse files
codegen-sh[bot]codegen-sh[bot]
authored
Add comprehensive documentation for Agent Run Logs API (#1130)
Co-authored-by: codegen-sh[bot] <131295404+codegen-sh[bot]@users.noreply.github.com>
1 parent 649b1b0 commit 3761b6c

File tree

2 files changed

+291
-0
lines changed

2 files changed

+291
-0
lines changed

‎docs/api-reference/agent-run-logs.mdx

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
---
2+
title: "Agent Run Logs API"
3+
sidebarTitle: "Agent Run Logs"
4+
icon: "list"
5+
iconType: "solid"
6+
---
7+
8+
<Warning>
9+
This endpoint is currently in **ALPHA** status and is subject to change. We welcome your feedback to help improve this API.
10+
</Warning>
11+
12+
The Agent Run Logs API allows you to retrieve detailed execution logs for agent runs, providing insights into the agent's thought process, tool usage, and execution flow.
13+
14+
## Endpoint
15+
16+
```
17+
GET /v1/organizations/{org_id}/agent/run/{agent_run_id}/logs
18+
```
19+
20+
## Authentication
21+
22+
This endpoint requires API token authentication. Include your token in the Authorization header:
23+
24+
```bash
25+
Authorization: Bearer YOUR_API_TOKEN
26+
```
27+
28+
## Parameters
29+
30+
| Parameter | Type | Required | Description |
31+
|-----------|------|----------|-------------|
32+
| `org_id` | integer | Yes | Your organization ID |
33+
| `agent_run_id` | integer | Yes | The ID of the agent run to retrieve logs for |
34+
| `skip` | integer | No | Number of logs to skip for pagination (default: 0) |
35+
| `limit` | integer | No | Maximum number of logs to return (default: 100, max: 100) |
36+
37+
## Response Structure
38+
39+
The endpoint returns an `AgentRunWithLogsResponse` object containing the agent run details and paginated logs:
40+
41+
```json
42+
{
43+
"id": 12345,
44+
"organization_id": 67890,
45+
"status": "completed",
46+
"created_at": "2024-01-15T10:30:00Z",
47+
"web_url": "https://app.codegen.com/agent/trace/12345",
48+
"result": "Task completed successfully",
49+
"logs": [
50+
{
51+
"agent_run_id": 12345,
52+
"created_at": "2024-01-15T10:30:15Z",
53+
"tool_name": "ripgrep_search",
54+
"message_type": "ACTION",
55+
"thought": "I need to search for the user's function in the codebase",
56+
"observation": {
57+
"status": "success",
58+
"results": ["Found 3 matches..."]
59+
},
60+
"tool_input": {
61+
"query": "function getUserData",
62+
"file_extensions": [".js", ".ts"]
63+
},
64+
"tool_output": {
65+
"matches": 3,
66+
"files": ["src/user.js", "src/api.ts"]
67+
}
68+
}
69+
],
70+
"total_logs": 25,
71+
"page": 1,
72+
"size": 100,
73+
"pages": 1
74+
}
75+
```
76+
77+
## Agent Run Log Fields
78+
79+
Each log entry in the `logs` array contains the following fields:
80+
81+
### Core Fields
82+
83+
| Field | Type | Description |
84+
|-------|------|-------------|
85+
| `agent_run_id` | integer | The ID of the agent run this log belongs to |
86+
| `created_at` | string | ISO 8601 timestamp when the log entry was created |
87+
| `message_type` | string | The type of log entry (see [Log Types](#log-types) below) |
88+
89+
### Agent Reasoning Fields
90+
91+
| Field | Type | Description |
92+
|-------|------|-------------|
93+
| `thought` | string \| null | The agent's internal reasoning or thought process for this step |
94+
95+
### Tool Execution Fields
96+
97+
| Field | Type | Description |
98+
|-------|------|-------------|
99+
| `tool_name` | string \| null | Name of the tool being executed (e.g., "ripgrep_search", "file_write") |
100+
| `tool_input` | object \| null | JSON object containing the parameters passed to the tool |
101+
| `tool_output` | object \| null | JSON object containing the tool's execution results |
102+
| `observation` | object \| string \| null | The agent's observation of the tool execution results or other contextual data |
103+
104+
## Log Types
105+
106+
The `message_type` field indicates the type of log entry. Here are the possible values:
107+
108+
### Plan Agent Types
109+
110+
| Type | Description |
111+
|------|-------------|
112+
| `ACTION` | The agent is executing a tool or taking an action |
113+
| `PLAN_EVALUATION` | The agent is evaluating or updating its plan |
114+
| `FINAL_ANSWER` | The agent is providing its final response or conclusion |
115+
| `ERROR` | An error occurred during execution |
116+
| `USER_MESSAGE` | A message from the user (e.g., interruptions or additional context) |
117+
| `USER_GITHUB_ISSUE_COMMENT` | A comment from a GitHub issue that the agent is processing |
118+
119+
### PR Agent Types
120+
121+
| Type | Description |
122+
|------|-------------|
123+
| `INITIAL_PR_GENERATION` | The agent is generating the initial pull request |
124+
| `DETECT_PR_ERRORS` | The agent is detecting errors in a pull request |
125+
| `FIX_PR_ERRORS` | The agent is fixing errors found in a pull request |
126+
| `PR_CREATION_FAILED` | Pull request creation failed |
127+
| `PR_EVALUATION` | The agent is evaluating a pull request |
128+
129+
### Commit Agent Types
130+
131+
| Type | Description |
132+
|------|-------------|
133+
| `COMMIT_EVALUATION` | The agent is evaluating commits |
134+
135+
### Link Types
136+
137+
| Type | Description |
138+
|------|-------------|
139+
| `AGENT_RUN_LINK` | A link to another related agent run |
140+
141+
## Field Population Patterns
142+
143+
Different log types populate different fields:
144+
145+
### ACTION Logs
146+
- Always have: `tool_name`, `tool_input`, `tool_output`
147+
- Often have: `thought`, `observation`
148+
- Example: Tool executions like searching code, editing files, creating PRs
149+
150+
### PLAN_EVALUATION Logs
151+
- Always have: `thought`
152+
- May have: `observation`
153+
- Rarely have: Tool-related fields
154+
- Example: Agent reasoning about next steps
155+
156+
### ERROR Logs
157+
- Always have: `observation` (containing error details)
158+
- May have: `tool_name` (if error occurred during tool execution)
159+
- Example: Failed tool executions or system errors
160+
161+
### FINAL_ANSWER Logs
162+
- Always have: `observation` (containing the final response)
163+
- May have: `thought`
164+
- Example: Agent's final response to the user
165+
166+
## Usage Examples
167+
168+
### Basic Log Retrieval
169+
170+
```python
171+
import requests
172+
173+
url = "https://api.codegen.com/v1/organizations/67890/agent/run/12345/logs"
174+
headers = {"Authorization": "Bearer YOUR_API_TOKEN"}
175+
176+
response = requests.get(url, headers=headers)
177+
data = response.json()
178+
179+
print(f"Agent run status: {data['status']}")
180+
print(f"Total logs: {data['total_logs']}")
181+
182+
for log in data['logs']:
183+
print(f"[{log['created_at']}] {log['message_type']}: {log['thought']}")
184+
```
185+
186+
### Filtering by Log Type
187+
188+
```python
189+
# Get only ACTION logs to see tool executions
190+
action_logs = [log for log in data['logs'] if log['message_type'] == 'ACTION']
191+
192+
for log in action_logs:
193+
print(f"Tool: {log['tool_name']}")
194+
print(f"Input: {log['tool_input']}")
195+
print(f"Output: {log['tool_output']}")
196+
print("---")
197+
```
198+
199+
### Pagination Example
200+
201+
```python
202+
# Get logs in batches of 50
203+
skip = 0
204+
limit = 50
205+
all_logs = []
206+
207+
while True:
208+
url = f"https://api.codegen.com/v1/organizations/67890/agent/run/12345/logs?skip={skip}&limit={limit}"
209+
response = requests.get(url, headers=headers)
210+
data = response.json()
211+
212+
all_logs.extend(data['logs'])
213+
214+
if len(data['logs']) < limit:
215+
break # No more logs
216+
217+
skip += limit
218+
219+
print(f"Retrieved {len(all_logs)} total logs")
220+
```
221+
222+
### Debugging Failed Runs
223+
224+
```python
225+
# Find error logs to debug issues
226+
error_logs = [log for log in data['logs'] if log['message_type'] == 'ERROR']
227+
228+
for error_log in error_logs:
229+
print(f"Error at {error_log['created_at']}: {error_log['observation']}")
230+
if error_log['tool_name']:
231+
print(f"Failed tool: {error_log['tool_name']}")
232+
```
233+
234+
## Common Use Cases
235+
236+
### 1. Building Monitoring Dashboards
237+
Use the logs to create dashboards showing:
238+
- Agent execution progress
239+
- Tool usage patterns
240+
- Error rates and types
241+
- Execution timelines
242+
243+
### 2. Debugging Agent Behavior
244+
Analyze logs to understand:
245+
- Why an agent made certain decisions
246+
- Where errors occurred in the execution flow
247+
- What tools were used and their results
248+
249+
### 3. Audit and Compliance
250+
Track agent actions for:
251+
- Code change auditing
252+
- Compliance reporting
253+
- Security monitoring
254+
255+
### 4. Performance Analysis
256+
Monitor:
257+
- Tool execution times
258+
- Common failure patterns
259+
- Agent reasoning efficiency
260+
261+
## Rate Limits
262+
263+
- **60 requests per 60 seconds** per API token
264+
- Rate limits are shared across all API endpoints
265+
266+
## Error Responses
267+
268+
| Status Code | Description |
269+
|-------------|-------------|
270+
| 400 | Bad Request - Invalid parameters |
271+
| 401 | Unauthorized - Invalid or missing API token |
272+
| 403 | Forbidden - Insufficient permissions |
273+
| 404 | Not Found - Agent run not found |
274+
| 429 | Too Many Requests - Rate limit exceeded |
275+
276+
## Feedback and Support
277+
278+
Since this endpoint is in ALPHA, we'd love your feedback! Please reach out through:
279+
280+
- [Community Slack](https://join.slack.com/t/codegen-community/shared_invite/zt-2p4xjjzjx-1~3tTbJWZWQUYOLAhvG5rA)
281+
- [GitHub Issues](https://github.com/codegen-sh/codegen-sdk/issues)
282+
- Email: [email protected]
283+
284+
<Note>
285+
The structure and fields of this API may change as we gather feedback and improve the service. We'll provide advance notice of any breaking changes.
286+
</Note>
287+

‎docs/docs.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,10 @@
6767
"source": "/api-reference/openapi3.json",
6868
"directory": "api-reference"
6969
}
70+
},
71+
{
72+
"group": "Guides",
73+
"pages": ["api-reference/agent-run-logs"]
7074
}
7175
]
7276
},

0 commit comments

Comments
 (0)