One query for hosts with ZFS and one for those without.

Since ZoL frees ARC under memory pressure, let's not
count it as "used" but instead as "free" memory.

node_memory_MemAvailable_bytes is a better estimator than the sum I used before
says some Prometheus documentation.  It is also almost the same, but reads nicer.

This is only semi correct. We keep logs up to what's specified in the privacy
policy.  Currently, that is implemented as keeping them for up to 30 days on
the individual nodes as well as on the central Loki server.

This should fix the current alerts for our RAID arrays.
It's only "should" because I can't test it proper without said
RAID arrays in the dev or staging machines.

Tom Prince authored 3 years ago and Florian Sesser committed 3 years ago

This includes the host-based metrics collector, and the VPN client setup (including key deployment).

The newer "Time Series" panel does not support two axes.

... and fix that option the other commit introduced

Nix sometimes seems peculiar about merging sets?

- Also alert on "negative" (== receiving) errors
- Use 'rate' so we can get out of the reporting if situation normalizes

groups based on cfg.introducers are already set up above and this expression
is clearly meant to define groups for the nodes

Grafana 8 recommends these for added performance and capabilities.

- (tryfix) Switch to Loki datasource;
- Filter requests to metrics instead of all GET;
- Show the last week by default
- Show log times

Interesting and not quickly fixable as it looks like: the
GET ... line comes *after*  the result.  We might be logging
this wrong to begin with, but probably this is some next
gen AI line ordering intelligence

Or rather, let it listen only on localhost.
I thought Grafana needed it, or Promtail needed it, but I don't remember
clearly, the web doesn't say clearly, and on my local dev stack
Promtail/Loki seems to still work just fine without GRPC on the network.

OMG look at teh genius of it

Do some name resolution for those poor computers

Improve factoring by removing those `/etc/hosts` entries

And avoid the infinite loop @tomprince was encountering in !258