Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.logfleet.io/llms.txt

Use this file to discover all available pages before exploring further.

This guide covers common issues and their solutions. For architecture-level failure handling, see Resilience.

​
Quick Diagnostics

Run this command on your edge device to get a quick health check: 
# Check all LogFleet containers
docker-compose ps

# View recent logs
docker-compose logs --tail=50

# Check Vector health
curl -s http://localhost:8686/health | jq

​
Common Issues

​
Agent Not Appearing in Dashboard

Symptoms: Edge agent deployed but not visible in LogFleet dashboard. Diagnostic steps: 
# 1. Verify agent is running
docker ps | grep logfleet

# 2. Check registration logs
docker logs logfleet-agent 2>&1 | grep -i "register"

# 3. Test API connectivity
curl -I https://api.logfleet.io/health
Common causes:
CauseSolution
Invalid API keyRegenerate key with edge:register permission
Network blockedEnsure outbound HTTPS (443) is allowed
DNS resolutionVerify api.logfleet.io resolves correctly
Clock skewSync NTP - JWT validation requires accurate time
Fix: 
# Verify API key is set
echo $LOGFLEET_API_KEY

# Test registration manually
curl -X POST https://api.logfleet.io/api/v1/edge/register \
  -H "X-API-Key: $LOGFLEET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "test-agent", "location": "test"}'

​
Logs Not Being Collected

Symptoms: Agent running but no logs appearing in local Loki. Diagnostic steps: 
# 1. Check Vector is receiving data
curl -s http://localhost:8686/metrics | grep "events_in_total"

# 2. Verify Vector config
docker exec logfleet-vector vector validate /etc/vector/vector.yaml

# 3. Check Loki ingestion
curl -s "http://localhost:3100/loki/api/v1/labels" | jq
Common causes:
CauseSolution
Wrong log pathUpdate file source paths in Vector config
Permission deniedRun Vector with appropriate user/group
Loki not readyWait for Loki to initialize (check loki_ready metric)
Log format mismatchVerify transform regex matches your log format
Fix for file permissions: 
# Check log file permissions
ls -la /var/log/your-app/

# Add Vector user to log group
usermod -aG syslog vector

​
Metrics Not Reaching Cloud

Symptoms: Local logs work but metrics don’t appear in cloud dashboards. Diagnostic steps: 
# 1. Check Vector sink status
curl -s http://localhost:8686/metrics | grep "sink"

# 2. Verify outbound connectivity
curl -I https://api.logfleet.io/api/v1/edge/metrics

# 3. Check for backpressure
docker logs logfleet-vector 2>&1 | grep -i "backpressure\|retry"
Common causes:
CauseSolution
Rate limitedCheck Rate Limits quotas
API key expiredRegenerate API key
Payload too largeReduce batch size in Vector config
Network intermittentMetrics buffer locally and retry
Fix rate limiting: 
# vector.yaml - reduce batch size
sinks:
  logfleet_metrics:
    type: http
    batch:
      max_events: 100  # Reduce from default
      timeout_secs: 5

​
High Memory Usage

Symptoms: Loki or Vector consuming excessive memory. Diagnostic steps: 
# Check container memory
docker stats --no-stream

# Loki memory breakdown
curl -s http://localhost:3100/metrics | grep "loki_ingester_memory"

# Vector memory
curl -s http://localhost:8686/metrics | grep "memory"
Common causes:
CauseSolution
High log volumeIncrease memory limits or add sampling
Retention too longReduce retention_period in Loki config
Too many streamsReduce label cardinality
Buffer overflowIncrease disk buffer, reduce memory buffer
Fix - reduce Loki memory: 
# loki-config.yaml
limits_config:
  ingestion_rate_mb: 4
  ingestion_burst_size_mb: 8
  max_streams_per_user: 1000

chunk_store_config:
  max_look_back_period: 168h  # 7 days instead of 30

​
Disk Filling Up

Symptoms: Edge device running out of storage. Diagnostic steps: 
# Check disk usage
df -h

# Find large Loki chunks
du -sh /var/lib/loki/*

# Check retention is working
curl -s http://localhost:3100/metrics | grep "compactor"
Common causes:
CauseSolution
Compactor not runningVerify Loki compactor is enabled
Retention too longReduce retention_period
High ingestion rateAdd sampling or filtering
WAL not flushingCheck Loki WAL health
Fix - force compaction: 
# Trigger manual compaction
curl -X POST http://localhost:3100/compactor/ring/forget

# Or restart with compaction
docker-compose restart loki

​
Log Streaming Not Working

Symptoms: Cannot stream logs on-demand from cloud. Diagnostic steps: 
# 1. Check streaming is enabled
echo $STREAM_ENABLED

# 2. Verify WebSocket connectivity
curl -I https://api.logfleet.io/api/v1/streaming/ws

# 3. Check agent streaming status
curl -s http://localhost:8080/status | jq '.streaming'
Common causes:
CauseSolution
Streaming disabledEnable via dashboard or API
Firewall blocking WSAllow WebSocket upgrade on port 443
Agent offlineCheck agent heartbeat status
Session expiredStreaming auto-disables after timeout
Fix - enable streaming: 
# Enable streaming via API
curl -X POST https://api.logfleet.io/api/v1/streaming/start \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"agent_id": "your-agent-id", "duration_minutes": 30}'

​
Debug Mode

Enable verbose logging for deeper diagnostics: 
# Vector debug mode
docker run -e VECTOR_LOG=debug logfleet/vector

# Loki debug mode
docker run -e LOKI_LOG_LEVEL=debug grafana/loki

​
Getting Help

If these steps don’t resolve your issue:
  1. Check the logs - Most issues are visible in container logs
  2. GitHub Issues - Search or file at github.com/sadhiappan/logfleet
  3. Community - Join our Discord for real-time help
When reporting issues, include:
  • LogFleet version (docker images | grep logfleet)
  • Edge device OS and architecture
  • Relevant container logs
  • Output from diagnostic commands above