Ensure the swap is correctly configured on every HRO Cloud servers
Following this discussion, and some recent reboot of the HRO Cloud servers, it seems like the swap is configure quite differently on some hosts.
To start with, the vm.swappiness
should be assess and defined at least similarly on type of each host.
find . -type f -exec grep --color=auto -nH --null -e 'swappiness' \{\} +
./morph/lib/hardware-vagrant.nix23: boot.kernel.sysctl = { "vm.swappiness" = 0; };
./morph/lib/issuer-aws.nix9: boot.kernel.sysctl = { "vm.swappiness" = 1; };
./morph/grid/production/storage001-hardware.nix15: boot.kernel.sysctl = { "vm.swappiness" = 1; };
./morph/grid/production/storage002-hardware.nix15: boot.kernel.sysctl = { "vm.swappiness" = 1; };
./morph/grid/production/storage003-hardware.nix16: boot.kernel.sysctl = { "vm.swappiness" = 1; };
./morph/grid/production/storage004-hardware.nix15: boot.kernel.sysctl = { "vm.swappiness" = 1; };
./morph/grid/production/storage005-hardware.nix15: boot.kernel.sysctl = { "vm.swappiness" = 1; };
./morph/grid/hro-cloud/storage002-hardware.nix15: boot.kernel.sysctl = { "vm.swappiness" = 0; };
Assuming 1
is the way to go:
- What about the issuer in
staging
andhro-cloud
? - What about all the monitoring instances?
- What about the storage001 in
staging
and 1 and 2 inhro-cloud
?
Secondly, some hosts use a dedicated swap partition, some use a swap file.
But at least one has a swap partition and still try to use a swap file (e.g.: payments.deerfield.leastauthority.com
)!
And I've see at least 2 hosts which fail to enable the swap file at boot time, likely because the parent FS is not yet mounted (e.g.: payments.deerfield.leastauthority.com
and monitoring.deerfield.leastauthority.com
).
Also, some hosts have a very different memory/swap size ratio, but this is unlikely to lead to any problem (there is always enough swap):
-
storage001.deerfield.leastauthority.com
has 16GB of memory and 32GB of swap (partition) -
storage002.deerfield.leastauthority.com
has 32GB of memory and 32GB of swap (partition) -
storage003.deerfield.leastauthority.com
has 16GB of memory and 8GB of swap (partition) -
payments.deerfield.leastauthority.com
has 8GB of memory and 4GB of swap (file) and 8GB of swap (partition) -
monitoring.deerfield.leastauthority.com
has 2GB of memory and 4GB of swap (file - but not enabled!)