Rate Limits

LogFleet uses rate limiting to ensure fair usage and protect the platform from abuse. This guide explains how rate limiting works and how to handle it in your applications.

Rate Limit Overview

Different endpoint groups have different rate limits:

Endpoint Group	Rate Limit	Window
Authentication (`/auth/*`)	10 requests	per minute
Dashboard (`/dashboard/*`)	60 requests	per minute
Management (`/agents`, `/api-keys`)	100 requests	per minute
Streaming (`/stream/*`)	10 requests	per minute
Billing (`/billing/*`)	30 requests	per minute
Metric Configs (`/metric-configs/*`)	100 requests	per minute
Edge API (`/edge/*`)	1000 requests	per minute

Rate Limit Headers

Every API response includes rate limit information:

HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1705312800

Header	Description
`X-RateLimit-Limit`	Maximum requests allowed in window
`X-RateLimit-Remaining`	Requests remaining in current window
`X-RateLimit-Reset`	Unix timestamp when the window resets

Handling Rate Limits

When you exceed the rate limit, you’ll receive a 429 Too Many Requests response:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 45

{
  "error": "rate_limit_exceeded",
  "message": "Rate limit exceeded. Try again in 45 seconds.",
  "retry_after": 45
}

Retry Strategy

Implement exponential backoff with jitter:

async function fetchWithRetry(url, options, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const response = await fetch(url, options);

    if (response.status !== 429) {
      return response;
    }

    const retryAfter = parseInt(response.headers.get('Retry-After') || '60');
    const jitter = Math.random() * 1000; // 0-1 second jitter
    const delay = (retryAfter * 1000) + jitter;

    console.log(`Rate limited. Retrying in ${delay}ms...`);
    await new Promise(resolve => setTimeout(resolve, delay));
  }

  throw new Error('Max retries exceeded');
}

import time
import random
import requests

def fetch_with_retry(url, headers, max_retries=3):
    for attempt in range(max_retries):
        response = requests.get(url, headers=headers)

        if response.status_code != 429:
            return response

        retry_after = int(response.headers.get('Retry-After', 60))
        jitter = random.uniform(0, 1)  # 0-1 second jitter
        delay = retry_after + jitter

        print(f"Rate limited. Retrying in {delay:.2f}s...")
        time.sleep(delay)

    raise Exception("Max retries exceeded")

func fetchWithRetry(url string, maxRetries int) (*http.Response, error) {
    for attempt := 0; attempt < maxRetries; attempt++ {
        resp, err := http.Get(url)
        if err != nil {
            return nil, err
        }

        if resp.StatusCode != 429 {
            return resp, nil
        }

        retryAfter, _ := strconv.Atoi(resp.Header.Get("Retry-After"))
        if retryAfter == 0 {
            retryAfter = 60
        }

        jitter := time.Duration(rand.Intn(1000)) * time.Millisecond
        delay := time.Duration(retryAfter)*time.Second + jitter

        log.Printf("Rate limited. Retrying in %v...", delay)
        time.Sleep(delay)
    }

    return nil, errors.New("max retries exceeded")
}

Best Practices

1. Monitor Rate Limit Headers

Track your remaining quota and slow down before hitting limits:

function checkRateLimit(response) {
  const remaining = parseInt(response.headers.get('X-RateLimit-Remaining'));
  const limit = parseInt(response.headers.get('X-RateLimit-Limit'));

  if (remaining < limit * 0.1) {
    console.warn(`Rate limit warning: ${remaining}/${limit} remaining`);
  }
}

2. Batch Requests

When possible, use batch endpoints instead of individual requests:

# Instead of multiple individual requests
GET /api/v1/agents/id1
GET /api/v1/agents/id2
GET /api/v1/agents/id3

# Use list with filters
GET /api/v1/agents?ids=id1,id2,id3

3. Cache Responses

Cache responses that don’t change frequently:

const cache = new Map();
const CACHE_TTL = 60000; // 1 minute

async function getCachedOrFetch(url, options) {
  const cached = cache.get(url);
  if (cached && Date.now() - cached.timestamp < CACHE_TTL) {
    return cached.data;
  }

  const response = await fetch(url, options);
  const data = await response.json();

  cache.set(url, { data, timestamp: Date.now() });
  return data;
}

4. Use Webhooks

For real-time updates, use webhooks instead of polling:

# Instead of polling every second
while true; do
  curl https://api.logfleet.io/api/v1/agents
  sleep 1
done

# Configure a webhook to receive updates
POST /api/v1/webhooks
{
  "url": "https://your-app.com/webhook",
  "events": ["agent.status_changed"]
}

Edge API Rate Limits

Edge agents have higher rate limits (1000 req/min) to accommodate:

Heartbeats every 30 seconds
Metrics every 60 seconds
Config syncs every 60 seconds

With multiple agents, ensure your total request rate stays within limits:

Agents	Heartbeats/min	Metrics/min	Config/min	Total/min
10	20	10	10	40
50	100	50	50	200
100	200	100	100	400
500	1000	500	500	2000 ⚠️

If you have more than ~300 agents, contact support to discuss rate limit increases.

Rate Limits by Plan

Higher-tier plans have increased rate limits:

Plan	Management	Dashboard	Edge API
Free	100/min	60/min	1000/min
Pro	500/min	300/min	5000/min
Enterprise	Custom	Custom	Custom

Troubleshooting

I'm hitting rate limits unexpectedly

Check if you have retry loops without proper backoff
Verify you’re not making duplicate requests
Review the X-RateLimit-Remaining header to see your usage
Consider caching frequently-accessed data

My edge agents are being rate limited

Verify heartbeat/metrics intervals aren’t too aggressive
Check total agent count vs. edge API limits
Ensure agents aren’t retrying failed requests too quickly
Contact support if you need higher limits

How do I request higher limits?

Enterprise customers can request custom rate limits. Contact support@logfleet.io with:

Your organization ID
Current usage patterns
Required limits and justification

Deployment

Operations

Support

Rate Limit Overview

Rate Limit Headers

Handling Rate Limits

Retry Strategy

Best Practices

1. Monitor Rate Limit Headers

2. Batch Requests

3. Cache Responses

4. Use Webhooks

Edge API Rate Limits

Rate Limits by Plan

Troubleshooting

​Rate Limit Overview

​Rate Limit Headers

​Handling Rate Limits

​Retry Strategy

​Best Practices

​1. Monitor Rate Limit Headers

​2. Batch Requests

​3. Cache Responses

​4. Use Webhooks

​Edge API Rate Limits

​Rate Limits by Plan

​Troubleshooting

Rate Limit Overview

Rate Limit Headers

Handling Rate Limits

Retry Strategy

Best Practices

1. Monitor Rate Limit Headers

2. Batch Requests

3. Cache Responses

4. Use Webhooks

Edge API Rate Limits

Rate Limits by Plan

Troubleshooting