Skip to main content
LogFleet provides on-demand log streaming for troubleshooting. Instead of always shipping logs to the cloud, you stream them only when needed.

​
How It Works

Normal Operation:
Edge Agent → Metrics Only → Cloud

Streaming Enabled:
Edge Agent → Metrics + Logs → Cloud (temporary)
Streaming sessions have automatic timeouts to prevent forgotten streams from running indefinitely.

​
Starting a Stream

​
Via API

curl -X POST https://api.logfleet.io/api/v1/stream/start \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "timeout_seconds": 300,
    "filters": {
      "level": ["ERROR", "WARN"],
      "source": "nginx"
    }
  }'
Response: 
{
  "session": {
    "id": "sess_abc123",
    "agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "status": "active",
    "started_at": "2024-01-15T10:00:00Z",
    "expires_at": "2024-01-15T10:05:00Z",
    "websocket_url": "wss://api.logfleet.io/api/v1/stream/ws/sess_abc123"
  }
}

​
Connecting to WebSocket

const ws = new WebSocket(
  `wss://api.logfleet.io/api/v1/stream/ws/sess_abc123?token=${token}`
);

ws.onmessage = (event) => {
  const log = JSON.parse(event.data);
  console.log(`[${log.level}] ${log.source}: ${log.message}`);
};

​
Filtering Logs

Apply filters to reduce noise and focus on relevant logs:

​
Level Filter

{
  "filters": {
    "level": ["ERROR", "WARN", "FATAL"]
  }
}

​
Source Filter

{
  "filters": {
    "source": "nginx"
  }
}

​
Pattern Filter

{
  "filters": {
    "pattern": "user_id=12345"
  }
}

​
Combined Filters

{
  "filters": {
    "level": ["ERROR"],
    "source": "api-gateway",
    "pattern": "timeout"
  }
}

​
Session Timeouts

All streaming sessions have mandatory timeouts:
SettingValue
Default timeout5 minutes
Maximum timeout1 hour
Warning before end30 seconds

​
Timeout Warning

30 seconds before a session ends, you’ll receive a control message: 
{
  "type": "control",
  "event": "session_ending",
  "reason": "timeout",
  "seconds_remaining": 30
}

​
Extending a Session

To continue streaming, start a new session before the current one ends: 
curl -X POST https://api.logfleet.io/api/v1/stream/start \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "timeout_seconds": 300
  }'

​
Stopping a Stream

Manually stop streaming when done: 
curl -X POST https://api.logfleet.io/api/v1/stream/stop \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "session_id": "sess_abc123"
  }'

​
Viewing Active Sessions

List all active streaming sessions: 
curl -X GET https://api.logfleet.io/api/v1/stream/sessions \
  -H "Authorization: Bearer $TOKEN"
Response: 
{
  "sessions": [
    {
      "id": "sess_abc123",
      "agent_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "agent_name": "edge-location-01",
      "status": "active",
      "started_at": "2024-01-15T10:00:00Z",
      "expires_at": "2024-01-15T10:05:00Z"
    }
  ],
  "total": 1
}

​
Log Message Format

Each log message contains: 
{
  "type": "log",
  "timestamp": "2024-01-15T10:00:00.123Z",
  "level": "ERROR",
  "source": "nginx",
  "message": "upstream timed out (110: Connection timed out)",
  "metadata": {
    "host": "edge-location-01",
    "file": "/var/log/nginx/error.log",
    "line": 12345
  }
}

​
Use Cases

Incident Response

Stream logs from affected agents during an outage to identify root cause.

Debugging

Watch real-time logs while reproducing an issue in production.

Deployment Verification

Monitor logs during deployments to catch errors immediately.

Security Investigation

Stream logs to investigate suspicious activity or security alerts.

​
Best Practices

Always apply filters to reduce bandwidth and focus on relevant logs. Streaming everything can overwhelm both the network and your analysis.
Use the shortest timeout that meets your needs. 5 minutes is usually sufficient for quick debugging sessions.
Manually stop sessions when you’re done troubleshooting. Don’t rely on timeouts for cleanup.
Keep an eye on active sessions across your organization to avoid excessive bandwidth usage.

​
Bandwidth Considerations

Streaming full logs consumes significantly more bandwidth than metrics:
ModeBandwidth (est.)
Metrics only~10 KB/hour
Streaming (filtered)~1-10 MB/hour
Streaming (all logs)~100+ MB/hour
Use streaming sparingly and with appropriate filters.

​
Client Libraries

​
JavaScript/TypeScript

import { LogFleetClient } from '@logfleet/client';

const client = new LogFleetClient({ token: 'your-jwt-token' });

const stream = await client.startStream({
  agentId: 'agent-uuid',
  timeout: 300,
  filters: { level: ['ERROR'] }
});

stream.on('log', (log) => {
  console.log(`[${log.level}] ${log.message}`);
});

stream.on('ending', (secondsRemaining) => {
  console.log(`Stream ending in ${secondsRemaining}s`);
});

// When done
await stream.stop();

​
Python

from logfleet import Client

client = Client(token="your-jwt-token")

with client.stream(
    agent_id="agent-uuid",
    timeout=300,
    filters={"level": ["ERROR"]}
) as stream:
    for log in stream:
        print(f"[{log.level}] {log.message}")
Client libraries are coming soon. See the WebSocket documentation for implementing streaming with raw WebSockets.