> ## Documentation Index
> Fetch the complete documentation index at: https://docs.logfleet.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Log Streaming
> On-demand real-time log streaming from edge agents
LogFleet provides on-demand log streaming for troubleshooting. Instead of always shipping logs to the cloud, you stream them only when needed.
## How It Works
```
Normal Operation:
Edge Agent → Metrics Only → Cloud
Streaming Enabled:
Edge Agent → Metrics + Logs → Cloud (temporary)
```
Streaming sessions have automatic timeouts to prevent forgotten streams from running indefinitely.
## Starting a Stream
### Via API
```bash theme={null}
curl -X POST https://api.logfleet.io/api/v1/stream/start \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"timeout_seconds": 300,
"filters": {
"level": ["ERROR", "WARN"],
"source": "nginx"
}
}'
```
Response:
```json theme={null}
{
"session": {
"id": "sess_abc123",
"agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"status": "active",
"started_at": "2024-01-15T10:00:00Z",
"expires_at": "2024-01-15T10:05:00Z",
"websocket_url": "wss://api.logfleet.io/api/v1/stream/ws/sess_abc123"
}
}
```
### Connecting to WebSocket
```javascript theme={null}
const ws = new WebSocket(
`wss://api.logfleet.io/api/v1/stream/ws/sess_abc123?token=${token}`
);
ws.onmessage = (event) => {
const log = JSON.parse(event.data);
console.log(`[${log.level}] ${log.source}: ${log.message}`);
};
```
## Filtering Logs
Apply filters to reduce noise and focus on relevant logs:
### Level Filter
```json theme={null}
{
"filters": {
"level": ["ERROR", "WARN", "FATAL"]
}
}
```
### Source Filter
```json theme={null}
{
"filters": {
"source": "nginx"
}
}
```
### Pattern Filter
```json theme={null}
{
"filters": {
"pattern": "user_id=12345"
}
}
```
### Combined Filters
```json theme={null}
{
"filters": {
"level": ["ERROR"],
"source": "api-gateway",
"pattern": "timeout"
}
}
```
## Session Timeouts
All streaming sessions have mandatory timeouts:
| Setting | Value |
| ------------------ | ---------- |
| Default timeout | 5 minutes |
| Maximum timeout | 1 hour |
| Warning before end | 30 seconds |
### Timeout Warning
30 seconds before a session ends, you'll receive a control message:
```json theme={null}
{
"type": "control",
"event": "session_ending",
"reason": "timeout",
"seconds_remaining": 30
}
```
### Extending a Session
To continue streaming, start a new session before the current one ends:
```bash theme={null}
curl -X POST https://api.logfleet.io/api/v1/stream/start \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"timeout_seconds": 300
}'
```
## Stopping a Stream
Manually stop streaming when done:
```bash theme={null}
curl -X POST https://api.logfleet.io/api/v1/stream/stop \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"session_id": "sess_abc123"
}'
```
## Viewing Active Sessions
List all active streaming sessions:
```bash theme={null}
curl -X GET https://api.logfleet.io/api/v1/stream/sessions \
-H "Authorization: Bearer $TOKEN"
```
Response:
```json theme={null}
{
"sessions": [
{
"id": "sess_abc123",
"agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"agent_name": "edge-location-01",
"status": "active",
"started_at": "2024-01-15T10:00:00Z",
"expires_at": "2024-01-15T10:05:00Z"
}
],
"total": 1
}
```
## Log Message Format
Each log message contains:
```json theme={null}
{
"type": "log",
"timestamp": "2024-01-15T10:00:00.123Z",
"level": "ERROR",
"source": "nginx",
"message": "upstream timed out (110: Connection timed out)",
"metadata": {
"host": "edge-location-01",
"file": "/var/log/nginx/error.log",
"line": 12345
}
}
```
## Use Cases
Stream logs from affected agents during an outage to identify root cause.
Watch real-time logs while reproducing an issue in production.
Monitor logs during deployments to catch errors immediately.
Stream logs to investigate suspicious activity or security alerts.
## Best Practices
Always apply filters to reduce bandwidth and focus on relevant logs.
Streaming everything can overwhelm both the network and your analysis.
Use the shortest timeout that meets your needs. 5 minutes is usually
sufficient for quick debugging sessions.
Manually stop sessions when you're done troubleshooting. Don't rely
on timeouts for cleanup.
Keep an eye on active sessions across your organization to avoid
excessive bandwidth usage.
## Bandwidth Considerations
Streaming full logs consumes significantly more bandwidth than metrics:
| Mode | Bandwidth (est.) |
| -------------------- | ---------------- |
| Metrics only | \~10 KB/hour |
| Streaming (filtered) | \~1-10 MB/hour |
| Streaming (all logs) | \~100+ MB/hour |
Use streaming sparingly and with appropriate filters.
## Client Libraries
### JavaScript/TypeScript
```typescript theme={null}
import { LogFleetClient } from '@logfleet/client';
const client = new LogFleetClient({ token: 'your-jwt-token' });
const stream = await client.startStream({
agentId: 'agent-uuid',
timeout: 300,
filters: { level: ['ERROR'] }
});
stream.on('log', (log) => {
console.log(`[${log.level}] ${log.message}`);
});
stream.on('ending', (secondsRemaining) => {
console.log(`Stream ending in ${secondsRemaining}s`);
});
// When done
await stream.stop();
```
### Python
```python theme={null}
from logfleet import Client
client = Client(token="your-jwt-token")
with client.stream(
agent_id="agent-uuid",
timeout=300,
filters={"level": ["ERROR"]}
) as stream:
for log in stream:
print(f"[{log.level}] {log.message}")
```
Client libraries are coming soon. See the [WebSocket documentation](/api-reference/streaming/websocket)
for implementing streaming with raw WebSockets.