# What it sends

Every field the InfraNest agent sends, how often, and how long InfraNest keeps it. The tables below are built
from the push endpoint's own validation, so they list exactly what InfraNest accepts. The agent's
[PAYLOAD.md](https://github.com/InfraNest-Infrastructure-Organized/infranest-agent/blob/main/PAYLOAD.md) has
the full contract with every edge case.

## See it yourself first

On the server, before or after installing:

```sh
infranest-agent print
```

That takes one reading and prints the exact JSON the agent would send, and sends nothing. It needs no
`sudo`: it reads the machine, not the token.

## How often

- **Every interval** (60 seconds by default; `INFRANEST_INTERVAL` from 10 seconds to 5 minutes) the agent
  takes a reading and sends it with any it could not send before, in one `POST` to
  `https://ingest.infranest.io/api/metrics/push`.
- **Once an hour at most**, the push also says which directories fill the fullest disk.
- The agent sends at most 60 readings in one push. InfraNest accepts up to 180,
  and up to 120 pushes a minute from one server's token.
- If your plan stores readings less often than the agent takes them, InfraNest tells the agent and it slows
  down to match. It never speeds up because of an answer.

The token names one server. Nothing in the push says which machine it came from: InfraNest knows it from the
token.

## Never sent

The agent runs as its own unprivileged user and reads only what that user may read. On top of that:

- **No log output.** When a service fails, the agent sends systemd's one-word reason (`oom-kill`, `timeout`,
  `exit-code`) and the exit status. Never a line the service printed, which can contain credentials.
- **No command lines.** A process is sent by its executable's name. Its arguments are sent only if you set
  both `INFRANEST_PROCESSES=1` and `INFRANEST_PROCESS_ARGS=1`; the processes list itself is off by default.
- **No hook commands.** For certbot it sends which kinds of renewal hook exist (`deploy`, `post`), never what
  they run.
- **No private keys or certificate contents.** For a certificate it sends the names, the expiry and the file it
  read them from.
- **No sizes it could not measure.** A directory the agent may not enter is named, with its permissions, but
  never given a size.
- **Nothing from the Docker socket.** Containers are read from systemd, so they are sent by ID, without names.
- **Nothing about your Ubuntu Pro account.** Only whether Pro is attached, the state of Livepatch and ESM, and
  when the subscription ends. Not the account, the contract or the machine ID.

It reads some configuration files to answer the questions in the tables, such as certbot's renewal configs,
APT and dnf-automatic settings, and the Ubuntu Pro cache. It sends the answers listed below, not the files.

## Every field

A field is left out when the agent cannot tell, and that is different from zero: "no updates" is `0`,
"could not check" is absent. Lists are capped, and a string over its limit makes InfraNest refuse the whole
push, not only that field.

### Every reading

Sent with every reading, by default every 60 seconds.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `collected_at` | time (RFC 3339) |  | When the reading was taken. Defaults to when it arrived |
| `cpu_percent` | number | 0 to 100 | CPU in use: user plus system time, not counting steal or iowait |
| `memory_percent` | number | 0 to 100 | Memory in use |
| `memory_used_bytes` | integer | 0 or more | Memory in use: total minus what the kernel calls available |
| `memory_total_bytes` | integer | 0 or more | Total memory |
| `swap_used_bytes` | integer | 0 or more | Swap in use |
| `load_1` | number | 0 to 10,000 | Load average over 1 minute. Absent on Windows |
| `load_5` | number | 0 to 10,000 | Load average over 5 minutes |
| `load_15` | number | 0 to 10,000 | Load average over 15 minutes |
| `uptime_seconds` | integer | 0 or more | Seconds since boot |
| `network_in_bps` | integer | 0 or more | Bytes per second received since the previous reading, on interfaces backed by a device |
| `network_out_bps` | integer | 0 or more | Bytes per second sent since the previous reading |
| `disk_read_bps` | integer | 0 or more | Bytes per second read from whole disks since the previous reading |
| `disk_write_bps` | integer | 0 or more | Bytes per second written since the previous reading |
| `pressure_cpu` | number | 0 to 100 | Share of the last minute some work waited for CPU (Linux pressure stall information) |
| `pressure_memory` | number | 0 to 100 | The same for memory |
| `pressure_memory_full` | number | 0 to 100 | Share of the last minute all work waited for memory |
| `pressure_io` | number | 0 to 100 | Share of the last minute some work waited for disk |
| `pressure_io_full` | number | 0 to 100 | Share of the last minute all work waited for disk |

### Disk space, per mount

Sent with every reading. Only bytes and inodes: the server works out the percentage.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `mounts` | list | up to 64 | The mounted filesystems |
| `mounts[].mount_point` | string | up to 191 characters; required | Where it is mounted, such as `/var` |
| `mounts[].device` | string | up to 191 characters | The device, such as `/dev/sda1` |
| `mounts[].used_bytes` | integer | 0 or more; required | Bytes used |
| `mounts[].total_bytes` | integer | at least 1; required | Size in bytes |
| `mounts[].inodes_used` | integer | 0 or more | Inodes used. Absent where the filesystem has no fixed inode table |
| `mounts[].inodes_total` | integer | at least 1 | Inodes in total |

### The machine itself

A snapshot of the machine, applied from the newest reading in each push.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `system` | object |  | The machine |
| `system.kernel` | string | up to 128 characters | Kernel version |
| `system.os` | string | up to 128 characters | Operating system and version |
| `system.pending_updates` | integer | 0 to 100,000 | Updates waiting to be installed |
| `system.security_updates` | integer | 0 to 100,000 | Of those, security updates |
| `system.reboot_required` | boolean |  | A reboot is needed to finish updates |
| `system.reboot_reason` | string | up to 16 characters | What said so: `flag_file`, `newer_kernel`, … |
| `system.newest_kernel` | string | up to 128 characters | The newer installed kernel |
| `system.auto_update` | object |  | Whether the machine installs updates by itself |
| `system.auto_update.source` | string | up to 32 characters | `unattended-upgrades` or `dnf-automatic` |
| `system.auto_update.enabled` | boolean |  | It installs updates on its own |
| `system.auto_update.reboot` | boolean |  | It reboots on its own when an update needs it |
| `system.auto_update.reboot_time` | string | up to 32 characters | When it reboots, as configured |
| `system.auto_update.reboot_with_users` | boolean |  | It reboots while someone is logged in |
| `system.auto_update.utc_offset_minutes` | integer | -900 to 900 | The machine's offset from UTC |
| `system.oom_kills` | integer | 0 to 1,000,000,000 | Processes the kernel killed for lack of memory since boot |
| `system.boot_id` | string | up to 64 characters | An ID that changes on every boot |
| `system.cpus` | integer | 1 to 4,096 | CPUs the kernel can use |
| `system.raid` | list | up to 32 | Software RAID arrays, from `/proc/mdstat` |
| `system.raid[].name` | string | up to 32 characters; required | The array, such as `md0` |
| `system.raid[].level` | string | up to 16 characters | Its level, such as `raid1` |
| `system.raid[].state` | string | up to 16 characters; required | `active` or `inactive` |
| `system.raid[].devices` | integer | 0 to 1,024 | Members it is built for |
| `system.raid[].active` | integer | 0 to 1,024 | Members working |
| `system.raid[].failed` | integer | 0 to 1,024 | Members marked failed |
| `system.raid[].sync` | string | up to 16 characters | A resync, recovery or check in progress |
| `system.raid[].sync_percent` | number | 0 to 100 | How far it is |
| `system.time_synced` | boolean |  | The kernel considers its clock synchronised |
| `system.time_error_ms` | integer | 0 to 2,147,483,647 | No longer sent by the agent; still accepted |
| `system.pro` | object |  | Ubuntu Pro, from its client's world-readable cache. Never the account or contract |
| `system.pro.attached` | boolean |  | Attached to a subscription |
| `system.pro.livepatch` | string | up to 32 characters | Livepatch's state |
| `system.pro.esm_infra` | string | up to 32 characters | ESM Infra's state |
| `system.pro.expires` | time (RFC 3339) |  | When the subscription ends |

### Services

systemd units that are enabled or have failed (Windows: automatic-start services). On by default; `INFRANEST_SERVICES=0` turns it off.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `services` | list | up to 200 | The watched units |
| `services[].unit` | string | up to 255 characters; required | The unit's name, such as `nginx.service` |
| `services[].description` | string | up to 255 characters | Its description |
| `services[].active_state` | string | up to 32 characters; required | systemd's state: `active`, `failed`, … |
| `services[].sub_state` | string | up to 32 characters | systemd's sub-state: `running`, `exited`, … |
| `services[].state_changed_at` | time (RFC 3339) |  | When it entered that state |
| `services[].result` | string | up to 32 characters | Why a failed unit failed, one word from systemd: `exit-code`, `oom-kill`, `timeout`, … Never log output |
| `services[].exec_main_code` | integer | -16 to 16 | How its main process ended: exited, killed or dumped |
| `services[].exec_main_status` | integer | -2,147,483,648 to 2,147,483,647 | The exit status, or the signal |
| `services[].restarts` | integer | 0 to 100,000,000 | How many times systemd restarted it |
| `services[].memory_bytes` | integer | 0 to 1,099,511,627,776 | Memory its cgroup uses |

### Containers

Running Docker and Podman containers, read from the units systemd keeps for them — never from the Docker socket.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `containers` | list | up to 100 | The running containers |
| `containers[].id` | string | up to 64 characters; required | The short container ID. No name: that is only readable as root |
| `containers[].runtime` | string | up to 32 characters | `docker` or `podman` |
| `containers[].active_state` | string | up to 32 characters | The state of its systemd scope |
| `containers[].started_at` | time (RFC 3339) |  | When it last started |
| `containers[].restarts` | integer | 0 to 100,000,000 | Restarts the agent has seen |
| `containers[].memory_bytes` | integer | 0 to 1,099,511,627,776 | Memory it uses |

### Certificate renewers

What renews certificates here: certbot, Caddy, Traefik. Only world-readable files are read. On by default; `INFRANEST_RENEWERS=0` turns it off.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `renewers` | list | up to 50 | What renews certificates |
| `renewers[].tool` | string | one of `certbot`, `caddy`, `traefik`; required | Which tool |
| `renewers[].name` | string | up to 255 characters | certbot's name for the certificate, or its main name |
| `renewers[].domains` | list | up to 100 | The names it renews |
| `renewers[].domains[]` | string | up to 253 characters | One name |
| `renewers[].domains_guessed` | boolean |  | The names are guessed from the certificate's name rather than read |
| `renewers[].not_after` | time (RFC 3339) |  | When the certificate expires, only if its file is readable |
| `renewers[].source` | string | up to 512 characters | The file this was read from |
| `renewers[].authenticator` | string | up to 64 characters | certbot's authenticator: `webroot`, `nginx`, `dns-…` |
| `renewers[].installer` | string | up to 64 characters | certbot's installer: `nginx` or `apache` |
| `renewers[].hooks` | list | up to 8 | Which kinds of hook run after a renewal: `deploy`, `post`. Never the commands |
| `renewers[].hooks[]` | string | up to 32 characters | One kind |
| `renewers[].unit` | string | up to 255 characters | The systemd timer that runs it |
| `renewers[].last_run_at` | time (RFC 3339) |  | When the timer last fired |
| `renewers[].last_result` | string | up to 32 characters | How that run finished, systemd's word |
| `renewers[].next_run_at` | time (RFC 3339) |  | When it fires next |
| `renewers[].unreadable` | boolean |  | The store exists and the agent was not allowed to read it |
| `renewers[].mode` | string | up to 8 characters | Its permissions |
| `renewers[].root_owned` | boolean |  | Owned by root |

### Processes

**Off by default.** Sent only when `INFRANEST_PROCESSES=1`.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `processes` | list | up to 32 | The largest processes by memory |
| `processes[].command` | string | up to 255 characters; required | The executable's name. The full command line only if you also set `INFRANEST_PROCESS_ARGS=1` |
| `processes[].cpu_percent` | number | 0 to 100 | Share of one core since the previous reading |
| `processes[].memory_bytes` | integer | 0 or more | Resident memory |
| `processes[].pid` | integer | 0 or more | Process ID |
| `processes[].started_at` | time (RFC 3339) |  | When it started |

### Once per push

Beside the readings, once for the whole push.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `failed` | object | up to 32 | Collectors that could not read what they were asked for |
| `failed.<collector>` | string | up to 255 characters | Why, per collector name |
| `agent_version` | string | up to 32 characters | The agent's version |
| `collectors` | object |  | Which optional collectors are switched on, sent even when they are off |
| `collectors.processes` | boolean |  | Processes are collected |
| `collectors.process_args` | boolean |  | Full command lines are collected |
| `collectors.services` | boolean |  | Services are collected |
| `collectors.renewers` | boolean |  | Certificate renewers are collected |

### What is filling a disk

At most once an hour, for the fullest mount only: the largest directories, and the ones the agent was not allowed to read.

| Field | Type | Limits | What it is |
|---|---|---|---|
| `disk_usage` | object |  | One mount's largest directories |
| `disk_usage.mount_point` | string | up to 191 characters; required | Which mount was looked at |
| `disk_usage.scanned_at` | time (RFC 3339) |  | When |
| `disk_usage.partial` | boolean |  | The look ran out of time; there may be more |
| `disk_usage.dirs` | list | up to 16 | The largest directories, four levels deep at most |
| `disk_usage.dirs[].path` | string | up to 512 characters; required | The directory |
| `disk_usage.dirs[].bytes` | integer | 0 or more; required | The size of the files in it |
| `disk_usage.dirs[].kind` | string | up to 64 characters | A plain name where the agent recognises it, such as "docker images and containers" |
| `disk_usage.accounted_bytes` | integer | 0 or more | Everything the look counted, listed or not |
| `disk_usage.unreadable` | list | up to 16 | Directories the agent was not allowed to read. Named, never sized |
| `disk_usage.unreadable[].path` | string | up to 512 characters; required | The directory |
| `disk_usage.unreadable[].mode` | string | up to 8 characters | Its permissions, as `ls -l` shows them |
| `disk_usage.unreadable[].root_owned` | boolean |  | Owned by root |
| `disk_usage.more_unreadable` | integer | 0 or more | How many more refusals were not named |

## How long it is kept

InfraNest keeps every reading for 48 hours, for the detailed charts.
Hourly averages are kept for as long as your plan keeps monitoring history:

| Plan | Every reading | Hourly averages |
|---|---|---|
| Free | 48 hours | 30 days |
| Pro | 48 hours | 90 days |
| Business | 48 hours | 365 days |
| Enterprise | 48 hours | 3,650 days |

An organization on a plan agreed separately may have other numbers; **Settings → Subscription** in the app
shows yours. Turning off monitoring for a server makes InfraNest refuse its pushes and store nothing, and the
agent then tries again only every 15 minutes.

## Related

- [Network requirements](network-requirements.md): what it connects to, and what happens offline.
- [Install the agent on many servers](install-on-many-servers.md).