> ## Documentation Index
> Fetch the complete documentation index at: https://docs.logfleet.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Vector Configuration

> Customize log collection, transforms, and sinks

## Overview

LogFleet uses [Vector](https://vector.dev) for log collection at the edge. The default configuration provides:

* **HTTP JSON input** on port 9880
* **Syslog UDP input** on port 514
* **Automatic enrichment** with edge metadata
* **Metric extraction** for counters and gauges
* **Local Loki storage** with configurable retention

## Default Sources

### HTTP JSON (Primary)

Send JSON logs via HTTP POST:

```bash theme={null}
curl -X POST http://localhost:9880 \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Order completed",
    "level": "info",
    "service": "pos",
    "order_id": "12345"
  }'
```

**Optional Headers:**

* `X-Device-ID`: Device identifier
* `X-Source-Type`: Device type (e.g., "pos", "kiosk")
* `X-Service`: Service name

### Syslog UDP

Send standard syslog messages:

```bash theme={null}
echo "<14>Jan 15 10:30:00 pos-terminal app: Order completed" | nc -u localhost 514
```

Syslog severity is automatically mapped to log levels:

* **0-3**: error
* **4**: warn
* **5-6**: info
* **7**: debug

## Automatic Enrichment

All logs are automatically enriched with:

| Field         | Source              | Example                |
| ------------- | ------------------- | ---------------------- |
| `source_type` | Input type          | `http`, `syslog`       |
| `edge_agent`  | `AGENT_NAME` env    | `store-001`            |
| `location_id` | `LOCATION_ID` env   | `nyc-01`               |
| `region`      | `REGION` env        | `us-east`              |
| `env`         | `ENVIRONMENT` env   | `production`           |
| `timestamp`   | Log or current time | `2024-01-15T10:30:00Z` |
| `level`       | Normalized          | `info`, `error`, etc.  |

## Metric Extraction

Vector extracts metrics from logs automatically, reducing bandwidth by 100-1000x.

### Automatic Metrics

| Metric             | Type    | Tags                | Trigger                       |
| ------------------ | ------- | ------------------- | ----------------------------- |
| `edge.logs`        | counter | level, location\_id | All logs                      |
| `edge.errors`      | counter | location\_id        | `level=error/fatal/critical`  |
| `edge.events`      | counter | event, location\_id | Logs with `.event` field      |
| `edge.latency_ms`  | gauge   | location\_id        | Logs with `.latency_ms` field |
| `edge.order_value` | gauge   | location\_id        | Logs with `.amount` field     |

### Emit Business Events

Include an `event` field in your logs to create business metrics:

```bash theme={null}
curl -X POST http://localhost:9880 \
  -H "Content-Type: application/json" \
  -d '{
    "event": "order_completed",
    "amount": 24.99,
    "latency_ms": 150,
    "service": "pos"
  }'
```

This creates three metrics:

* `edge.events{event="order_completed", location_id="..."}` (counter)
* `edge.order_value{location_id="..."}` (gauge: 24.99)
* `edge.latency_ms{location_id="..."}` (gauge: 150)

## Loki Labels

Logs are stored in Loki with these labels:

```
{source_type="http", edge_agent="store-001", location_id="nyc-01", level="info"}
```

<Warning>
  Keep label cardinality low. High cardinality labels (like user IDs) can overwhelm Loki.
</Warning>

## Customizing Vector Config

### Override with Volume Mount

Mount a custom config file:

```bash theme={null}
docker run -d \
  --name logfleet-agent \
  -e API_KEY=$API_KEY \
  -e CLOUD_URL=https://api.logfleet.io \
  -v /path/to/custom-vector.toml:/etc/vector/vector.toml:ro \
  ghcr.io/sadhiappan/logfleet-agent:latest
```

### Add File Tailing

To collect from log files:

```toml theme={null}
[sources.app_logs]
type = "file"
include = ["/var/log/app/*.log"]
read_from = "beginning"

[transforms.enrich_app_logs]
type = "remap"
inputs = ["app_logs"]
source = '''
.source_type = "file"
.edge_agent = get_env_var("AGENT_NAME") ?? "unknown"
.location_id = get_env_var("LOCATION_ID") ?? "unknown"
'''
```

### Add Custom Metrics

Extract additional metrics from specific log patterns:

```toml theme={null}
[transforms.custom_metrics]
type = "log_to_metric"
inputs = ["all_logs"]

[[transforms.custom_metrics.metrics]]
type = "counter"
field = "payment_method"
name = "payments"
namespace = "edge"
tags.method = "{{payment_method}}"
tags.location_id = "{{location_id}}"
```

## Datadog Integration

By default, metrics are sent to Datadog (if `DATADOG_API_KEY` is set):

```bash theme={null}
docker run -d \
  --name logfleet-agent \
  -e API_KEY=$API_KEY \
  -e CLOUD_URL=https://api.logfleet.io \
  -e DATADOG_API_KEY=your_datadog_api_key \
  ghcr.io/sadhiappan/logfleet-agent:latest
```

**Cost Optimization:**

* Metrics are aggregated over 5-minute windows (99%+ reduction)
* Only business metrics are sent (no internal Vector metrics)
* Counters preferred over histograms (12x fewer data points)

## Prometheus Metrics

A Prometheus exporter runs on port 9598 for local monitoring:

```bash theme={null}
curl http://localhost:9598/metrics
```

## Debugging

### Enable Console Output

Uncomment the console sink in your Vector config:

```toml theme={null}
[sinks.console]
type = "console"
inputs = ["all_logs"]
encoding.codec = "json"
```

### Validate Configuration

```bash theme={null}
vector validate --config /path/to/vector.toml
```

### Check Vector API

```bash theme={null}
curl http://localhost:8686/health
```

## Environment Variables

| Variable          | Default        | Purpose                  |
| ----------------- | -------------- | ------------------------ |
| `AGENT_NAME`      | `edge-agent-1` | Agent identifier in logs |
| `LOCATION_ID`     | `unknown`      | Location identifier      |
| `REGION`          | `unknown`      | Geographic region        |
| `ENVIRONMENT`     | `production`   | Environment tag          |
| `DATADOG_API_KEY` | (none)         | Enable Datadog metrics   |

## Full Default Configuration

The complete default Vector configuration is available at:
[github.com/sadhiappan/logfleet/blob/main/agent/configs/vector/vector.toml](https://github.com/sadhiappan/logfleet/blob/main/agent/configs/vector/vector.toml)