diff --git a/docs/source/ops/generating-keys.rst b/docs/source/ops/generating-keys.rst
index afe2ece4009f761aea56acd24fcbf627b985cadb..47a1f4e91a876ac1919252c099654886f0bd128a 100644
--- a/docs/source/ops/generating-keys.rst
+++ b/docs/source/ops/generating-keys.rst
@@ -1,7 +1,28 @@
Generating keys
===============
-``config.json`` has the paths for the Ristretto and the Stripe secret key files.
+There's an example ``secrets`` repo in ``morph/grid/local/secrets``.
+``<grid>/config.json`` has the paths for the key files for the respective grid.
+Create a symlink named ``secrets`` to your secret key repository for the deployment you are working on.
+
+
+Stripe
+``````
+
+For the Stripe key any random bytes with a little light formatting "work" - at least to make our software happy - but if you want to be able to interact with Stripe and have payments (even pretend payments) move all the way through the system you should get a Stripe account and generate a key w/ them.
+Lauri can get you added to our "dev" Stripe account, too, though I forget how important that is for ad hoc dev/testing.
+
+I think this will work for generating random Stripe secret keys (that our software will load, I think, but Stripe will reject)::
+
+ >>> import base64, os
+ >>> print((b"sk_test_" + base64.b64encode(os.urandom(25)).strip(b"=")).decode("ascii"))
+ sk_test_Dr+XLVjkC0oO3Zw8Ws0yWtDLqR1sM+/fmw
+
+Public keys are the same but "pk_test" instead of "sk_test" ("test" is for "test mode" key that can only process pretend txns; for real txns there are keys with "live" embedded).
+
+
+ZKAP-Issuer Ristretto
+`````````````````````
Here is a Ristretto key you can use, randomly generated just now::
@@ -19,16 +40,9 @@ For example::
echo -n "SILOWzbnkBjxC1hGde9d5Q3Ir/4yLosCLEnEQGAxEQE=" > ristretto.signing-key
-For the Stripe key any random bytes with a little light formatting "work" - at least to make our software happy - but if you want to be able to interact with Stripe and have payments (even pretend payments) move all the way through the system you should get a Stripe account and generate a key w/ them.
-Lauri can get you added to our "dev" Stripe account, too, though I forget how important that is for ad hoc dev/testing.
-
-I think this will work for generating random Stripe secret keys (that our software will load, I think, but Stripe will reject)::
-
- >>> import base64, os
- >>> print((b"sk_test_" + base64.b64encode(os.urandom(25)).strip(b"=")).decode("ascii"))
- sk_test_Dr+XLVjkC0oO3Zw8Ws0yWtDLqR1sM+/fmw
-Public keys are the same but "pk_test" instead of "sk_test" ("test" is for "test mode" key that can only process pretend txns; for real txns there are keys with "live" embedded).
+ZKAP-Issuer TLS
+```````````````
The ZKAPIssuer.service needs a working TLS certificate and expects it in the certbot directory for the domain you configured, in my case::
@@ -37,14 +51,26 @@ The ZKAPIssuer.service needs a working TLS certificate and expects it in the cer
Move the three .pem files into the payment's server ``/var/lib/letsencrypt/live/payments.localdev/`` directory and issue a ``sudo systemctl restart zkapissuer.service``.
-Create Wireguard VPN key pairs in ``PrivateStorageSecrets/monitoringvpn/`` or where you have them::
- for i in "172.23.23.11" "172.23.23.12" "172.23.23.13" "server"; do
- wg genkey | tee ${i}.key | wg pubkey > ${i}.pub
+Monitoring VPN
+``````````````
+
+Create Wireguard VPN key pairs in ``secrets/monitoringvpn/`` or where you have them.
+
+``tools/create-vpn-keys.sh`` holds a script to rotate all VPN keys at once::
+
+ ./tools/create-vpn-keys.sh morph/grid/testing/grid.nix
+
+Or do it manually::
+
+ cd secrets/monitoringvpn
+ for i in 1 11 12 13 ; do
+ wg genkey | tee 172.23.23.${i}.key | wg pubkey > 172.23.23.${i}.pub
done
+ ln -s 172.23.23.1.key server.key
+ ln -s 172.23.23.1.pub server.pub
+
And a shared VPN key for "post-quantum resistance"::
wg genpsk > preshared.key
-
-
diff --git a/morph/grid/local/README.rst b/morph/grid/local/README.rst
index 59f032a0fad5bf2fd328955db7649b3cd8b288e4..345547244635734278aa76cb5cd59946f2afd37f 100644
--- a/morph/grid/local/README.rst
+++ b/morph/grid/local/README.rst
@@ -14,7 +14,7 @@ Use the local development environment
0. Add VirtualBox to your NixOs system configuration at ``/etc/nixos/configuration.nix``::
virtualisation.virtualbox.host.enable = true;
- # Use VirtualBox installation without GUI and Qt dependency:
+ # Save bytes and build time, optional but recommended:
virtualisation.virtualbox.host.headless = true;
1. Enter the morph local grid directory::
@@ -35,7 +35,7 @@ Use the local development environment
5. Edit the generated configuration: Add the ``publicIP`` addresses from ``grid.nix`` to ssh config **Host** match blocks (**not** HostName) so the ``Host`` lines all read like::
- Host payments1 192.168.67.21
+ Host payments 192.168.67.21
HostName 127.0.0.1
User vagrant
[...]
diff --git a/morph/grid/local/Vagrantfile b/morph/grid/local/Vagrantfile
index 82bbef1063b108829261670fdceb2e27af8d6764..7ad95ca872a72e5da6c11b3269e2a824cf8a55f9 100644
--- a/morph/grid/local/Vagrantfile
+++ b/morph/grid/local/Vagrantfile
@@ -8,8 +8,8 @@ Vagrant.configure("2") do |config|
# For a complete reference, please see the online documentation at
# https://docs.vagrantup.com.
- config.vm.define "payments1" do |config|
- config.vm.hostname = "payments1"
+ config.vm.define "payments" do |config|
+ config.vm.hostname = "payments"
config.vm.box = "esselius/nixos"
config.vm.box_version = "20.09"
config.vm.box_check_update = false
@@ -36,8 +36,8 @@ Vagrant.configure("2") do |config|
config.vm.network "private_network", ip: "192.168.67.23"
end
- config.vm.define "monitoring1" do |config|
- config.vm.hostname = "monitoring1"
+ config.vm.define "monitoring" do |config|
+ config.vm.hostname = "monitoring"
config.vm.box = "esselius/nixos"
config.vm.box_version = "20.09"
config.vm.box_check_update = false
diff --git a/morph/grid/local/grid.nix b/morph/grid/local/grid.nix
index 0c114ccf11b9ddb963d5e2e6297b6a7c83792aba..1729425d91bc02f56fa895d04731ffcc74971cbe 100644
--- a/morph/grid/local/grid.nix
+++ b/morph/grid/local/grid.nix
@@ -1,64 +1,75 @@
-# Load the helper function and call it with arguments tailored for the local
-# grid. It will make the morph configuration for us. We share this function
-# with the production grid and have one fewer possible point of divergence.
-import ../../lib/make-grid.nix {
- name = "LocalDev";
- config = ./config.json;
- nodes = cfg:
- let
+let
+ pkgs = import <nixpkgs> { };
+
+ gridlib = import ../../lib;
+ rawConfig = pkgs.lib.trivial.importJSON ./config.json;
+ config = rawConfig // {
sshUsers = import ./secrets/users.nix;
# Get absolute vpn key directory path, as a string:
- monitoringvpnKeyDir = toString ./. + "/${cfg.monitoringvpnKeyDir}";
+ monitoringvpnKeyDir = toString ./. + "/${rawConfig.monitoringvpnKeyDir}";
+ };
+
+ payments = let publicIPv4 = "192.168.67.21"; in {
+ imports = [
+ gridlib.issuer
+ (gridlib.hardware-virtual ({ inherit publicIPv4; }))
+ (gridlib.customize-issuer (config // {
+ monitoringvpnIPv4 = "172.23.23.11";
+ }))
+ ];
+ };
- # TBD: derive these automatically:
- hostsMap = {
- "172.23.23.1" = [ "monitoring1" "monitoring1.monitoringvpn" ];
- "172.23.23.11" = [ "payments1" "payments1.monitoringvpn" ];
- "172.23.23.12" = [ "storage1" "storage1.monitoringvpn" ];
- "172.23.23.13" = [ "storage2" "storage2.monitoringvpn" ];
- };
- vpnClientIPs = [ "172.23.23.11" "172.23.23.12" "172.23.23.13" ];
- nodeExporterTargets = [ "monitoring1" "payments1" "storage1" "storage2" ];
+ storage1 = let publicIPv4 = "192.168.67.22"; in {
+ imports = [
+ gridlib.storage
+ (gridlib.hardware-virtual ({ inherit publicIPv4; }))
+ (gridlib.customize-storage (config // {
+ inherit publicIPv4;
+ monitoringvpnIPv4 = "172.23.23.12";
+ stateVersion = "19.09";
+ }))
+ ];
+ };
- in {
- "payments1" = import ../../lib/make-issuer.nix (cfg // rec {
- publicIPv4 = "192.168.67.21";
- monitoringvpnIPv4 = "172.23.23.11";
- hardware = import ./virtual-hardware.nix ({ inherit publicIPv4; });
- stateVersion = "19.03";
- inherit monitoringvpnKeyDir;
- inherit sshUsers;
- });
+ storage2 = let publicIPv4 = "192.168.67.23"; in {
+ imports = [
+ gridlib.storage
+ (gridlib.hardware-virtual ({ inherit publicIPv4; }))
+ (gridlib.customize-storage (config // {
+ inherit publicIPv4;
+ monitoringvpnIPv4 = "172.23.23.13";
+ stateVersion = "19.09";
+ }))
+ ];
+ };
- "storage1" = import ../../lib/make-testing.nix (cfg // rec {
- publicIPv4 = "192.168.67.22";
- monitoringvpnIPv4 = "172.23.23.12";
- hardware = import ./virtual-hardware.nix ({ inherit publicIPv4; });
- stateVersion = "19.09";
- inherit monitoringvpnKeyDir;
- inherit sshUsers;
- });
+ monitoring = let publicIPv4 = "192.168.67.24"; in {
+ imports = [
+ gridlib.monitoring
+ (gridlib.hardware-virtual ({ inherit publicIPv4; }))
+ (gridlib.customize-monitoring {
+ inherit hostsMap publicIPv4 vpnClientIPs nodeExporterTargets;
+ inherit (config) monitoringvpnKeyDir;
+ monitoringvpnIPv4 = "172.23.23.1";
+ stateVersion = "19.09";
+ })
+ ];
+ };
- "storage2" = import ../../lib/make-testing.nix (cfg // rec {
- publicIPv4 = "192.168.67.23";
- monitoringvpnIPv4 = "172.23.23.13";
- hardware = import ./virtual-hardware.nix ({ inherit publicIPv4; });
- stateVersion = "19.09";
- inherit monitoringvpnKeyDir;
- inherit sshUsers;
- });
+ # TBD: derive these automatically:
+ hostsMap = {
+ "172.23.23.1" = [ "monitoring" "monitoring.monitoringvpn" ];
+ "172.23.23.11" = [ "payments" "payments.monitoringvpn" ];
+ "172.23.23.12" = [ "storage1" "storage1.monitoringvpn" ];
+ "172.23.23.13" = [ "storage2" "storage2.monitoringvpn" ];
+ };
+ vpnClientIPs = [ "172.23.23.11" "172.23.23.12" "172.23.23.13" ];
+ nodeExporterTargets = [ "monitoring" "payments" "storage1" "storage2" ];
- "monitoring1" = import ../../lib/make-monitoring.nix (cfg // rec {
- publicIPv4 = "192.168.67.24";
- monitoringvpnIPv4 = "172.23.23.1";
- inherit vpnClientIPs;
- inherit hostsMap;
- inherit nodeExporterTargets;
- hardware = import ./virtual-hardware.nix ({ inherit publicIPv4; });
- stateVersion = "19.09";
- inherit monitoringvpnKeyDir;
- inherit sshUsers;
- });
+in {
+ network = {
+ description = "PrivateStorage.io LocalDev Grid";
};
+ inherit payments monitoring storage1 storage2;
}
diff --git a/morph/grid/production/config.json b/morph/grid/production/config.json
index e71cb8b4b5f999e3059f0669c2bc3f92f29242a6..ef7dc53649febcd7beb7901bb3608204df197059 100644
--- a/morph/grid/production/config.json
+++ b/morph/grid/production/config.json
@@ -1,6 +1,8 @@
{ "publicStoragePort": 8898
, "ristrettoSigningKeyPath": "./secrets/ristretto.signing-key"
, "stripeSecretKeyPath": "./secrets/stripe.secret"
+, "monitoringvpnKeyDir": "./secrets/monitoringvpn"
+, "monitoringvpnEndpoint": "monitoring.private.storage:51820"
, "passValue": 1000000
, "issuerDomains": [
"payments.privatestorage.io"
diff --git a/morph/grid/production/grid.nix b/morph/grid/production/grid.nix
index f5735d259dbff27f1d9cabbbca512af81d4550bb..7a8e29bb58da584d07660663912a1c993146787b 100644
--- a/morph/grid/production/grid.nix
+++ b/morph/grid/production/grid.nix
@@ -1,61 +1,108 @@
-# Load the helper function and call it with arguments tailored for the testing
-# grid. It will make the morph configuration for us. We share this function
-# with the testing grid and have one fewer possible point of divergence.
-import ../../lib/make-grid.nix {
- name = "Production";
- config = ./config.json;
- nodes = cfg:
- let
- sshUsers = import ./secrets/users.nix;
- in {
- # Here are the hosts that are in this morph network. This is sort of like
- # a server manifest. We try to keep as many of the specific details as
- # possible out of *this* file so that this file only grows as server count
- # grows. If it grows too much, we can load servers by listing contents of
- # a directory or reading from another JSON file or some such. For now,
- # I'm just manually maintaining these entries.
- #
- # The name on the left of the `=` is mostly irrelevant but it does provide
- # a default hostname for the server if the configuration on the right side
- # doesn't specify one.
- #
- # The names must be unique!
- "payments.privatestorage.io" = import ../../lib/make-issuer.nix ({
- publicIPv4 = "18.184.142.208";
- inherit sshUsers;
- hardware = ../../lib/issuer-aws.nix;
- stateVersion = "19.03";
- } // cfg);
-
- "storage001" = import ../../lib/make-storage.nix ({
- cfg = import ./storage001-config.nix;
- inherit sshUsers;
- hardware = ./storage001-hardware.nix;
- stateVersion = "19.09";
- } // cfg);
- "storage002" = import ../../lib/make-storage.nix ({
- cfg = import ./storage002-config.nix;
- inherit sshUsers;
- hardware = ./storage002-hardware.nix;
- stateVersion = "19.09";
- } // cfg);
- "storage003" = import ../../lib/make-storage.nix ({
- cfg = import ./storage003-config.nix;
- inherit sshUsers;
- hardware = ./storage003-hardware.nix;
- stateVersion = "19.09";
- } // cfg);
- "storage004" = import ../../lib/make-storage.nix ({
- cfg = import ./storage004-config.nix;
- inherit sshUsers;
- hardware = ./storage004-hardware.nix;
+# See morph/grid/local/grid.nix for additional commentary.
+let
+ pkgs = import <nixpkgs> { };
+
+ gridlib = import ../../lib;
+ rawConfig = pkgs.lib.trivial.importJSON ./config.json;
+ config = rawConfig // {
+ sshUsers = import ./secrets/users.nix;
+
+ # Get absolute vpn key directory path, as a string:
+ monitoringvpnKeyDir = toString ./. + "/${rawConfig.monitoringvpnKeyDir}";
+ };
+
+ payments = {
+ imports = [
+ gridlib.issuer
+ gridlib.hardware-aws
+ (gridlib.customize-issuer (config // {
+ monitoringvpnIPv4 = "172.23.23.11";
+ }))
+ ];
+ };
+
+ monitoring = let publicIPv4 = "monitoring.private.storage"; in {
+ imports = [
+ gridlib.monitoring
+ gridlib.hardware-aws
+ (gridlib.customize-monitoring {
+ inherit hostsMap publicIPv4 vpnClientIPs nodeExporterTargets;
+ inherit (config) monitoringvpnKeyDir;
+ monitoringvpnIPv4 = "172.23.23.1";
stateVersion = "19.09";
- } // cfg);
- "storage005" = import ../../lib/make-storage.nix ({
- cfg = import ./storage005-config.nix;
- inherit sshUsers;
- hardware = ./storage005-hardware.nix;
- stateVersion = "19.03";
- } // cfg);
+ })
+ ];
+ };
+
+ defineStorageNode = name: { vpnIP, stateVersion }: let nodecfg = import "${./.}/${name}-config.nix"; in {
+ imports = [
+ # Get some of the very lowest-level system configuration for this
+ # node. This isn't all *completely* hardware related. Maybe some
+ # more factoring is in order, someday.
+ "${./.}/${name}-hardware.nix"
+
+ # Slightly awkwardly, enable some of our hardware / network / bootloader options.
+ ../../../nixos/modules/100tb.nix
+
+ # Get all of the configuration that is common across all storage nodes.
+ gridlib.storage
+
+ # Then customize the storage system a little bit based on this node's particulars.
+ (gridlib.customize-storage (config // nodecfg // {
+ monitoringvpnIPv4 = vpnIP;
+ inherit stateVersion;
+ }))
+ ];
+
+ # And supply configuration for those hardware / network / bootloader
+ # options. See the 100tb module for handling of this value. The module
+ # name is quoted because `1` makes `100tb` look an awful lot like a
+ # number.
+ "100tb".config = nodecfg;
+ };
+
+ # Define all of the storage nodes for this grid.
+ storageNodes = builtins.mapAttrs defineStorageNode {
+ storage001 = { vpnIP = "172.23.23.21"; stateVersion = "19.09"; };
+ storage002 = { vpnIP = "172.23.23.22"; stateVersion = "19.09"; };
+ storage003 = { vpnIP = "172.23.23.23"; stateVersion = "19.09"; };
+ storage004 = { vpnIP = "172.23.23.24"; stateVersion = "19.09"; };
+ storage005 = { vpnIP = "172.23.23.25"; stateVersion = "19.03"; };
};
-}
+
+ # TBD: derive these automatically:
+ hostsMap = {
+ "172.23.23.1" = [ "monitoring" "monitoring.monitoringvpn" ];
+ "172.23.23.11" = [ "payments" "payments.monitoringvpn" ];
+ "172.23.23.21" = [ "storage001" "storage001.monitoringvpn" ];
+ "172.23.23.22" = [ "storage002" "storage002.monitoringvpn" ];
+ "172.23.23.23" = [ "storage003" "storage003.monitoringvpn" ];
+ "172.23.23.24" = [ "storage004" "storage004.monitoringvpn" ];
+ "172.23.23.25" = [ "storage005" "storage005.monitoringvpn" ];
+ };
+ vpnClientIPs = [
+ "172.23.23.11"
+ "172.23.23.21"
+ "172.23.23.22"
+ "172.23.23.23"
+ "172.23.23.24"
+ "172.23.23.25"
+ ];
+ nodeExporterTargets = [
+ "monitoring"
+ "payments"
+ "storage001"
+ "storage002"
+ "storage003"
+ "storage004"
+ "storage005"
+ ];
+
+in {
+ network = {
+ description = "PrivateStorage.io Production Grid";
+ };
+
+ "payments.privatestorage.io" = payments;
+ inherit monitoring;
+} // storageNodes
diff --git a/morph/grid/production/storage003-config.nix b/morph/grid/production/storage003-config.nix
index e83546adbcdab2fd35d990a13550dd3907d7226b..5b3f5adf969317322b2c39014e6500294b5f3c02 100644
--- a/morph/grid/production/storage003-config.nix
+++ b/morph/grid/production/storage003-config.nix
@@ -4,5 +4,5 @@
"prefixLength" = 30;
"gateway" = "45.83.89.185";
"gatewayInterface" = "eno1";
- "grubDeviceID" = "wwn-0x5000cca248c31469";
+ "grubDeviceID" = "wwn-0x5000039a8bc00766";
}
diff --git a/morph/grid/production/storage003-hardware.nix b/morph/grid/production/storage003-hardware.nix
index 83a37d9e164d9c8eb6f4123e96d53ea65ae4dfbe..d8ffe5d59fb39ba4a9c6b1b73313f199a2ed980b 100644
--- a/morph/grid/production/storage003-hardware.nix
+++ b/morph/grid/production/storage003-hardware.nix
@@ -1,31 +1,32 @@
# Do not modify this file! It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations. Please make changes
# to /etc/nixos/configuration.nix instead.
-{ config, lib, pkgs, ... }:
+{ config, lib, pkgs, modulesPath, ... }:
{
imports =
- [ <nixpkgs/nixos/modules/installer/scan/not-detected.nix>
+ [ (modulesPath + "/installer/scan/not-detected.nix")
];
- boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "ehci_pci" "megaraid_sas" "usbhid" "sd_mod" ];
+ boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "ehci_pci" "megaraid_sas" "usbhid" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ "kvm-intel" ];
boot.extraModulePackages = [ ];
+ boot.supportedFilesystems = [ "zfs" ];
boot.kernel.sysctl = { "vm.swappiness" = 0; };
fileSystems."/" =
- { device = "/dev/disk/by-uuid/daf0b345-97da-46bc-b9df-500d771ec375";
+ { device = "/dev/disk/by-uuid/240fc1f6-cd55-48a3-ac80-5b3550a32ef5";
fsType = "ext4";
};
fileSystems."/boot" =
- { device = "/dev/disk/by-uuid/a1843705-f4e9-4805-924c-19f464d23da7";
+ { device = "/dev/disk/by-label/boot";
fsType = "ext4";
};
# Manually created using:
- # zpool create -f -m legacy -o ashift=12 root raidz /dev/disk/by-id/{wwn-0x5000cca249d43969,wwn-0x5000cca248dd1f83,wwn-0x5000cca249d44a67,wwn-0x5000cca249d46730,wwn-0x5000cca25dcc719c,wwn-0x5000cca25dcc0241,wwn-0x5000cca24ac2b2df}
+ # zpool create -f -m legacy -o ashift=12 root raidz /dev/disk/by-id/{wwn-0x5000cca249d43969,wwn-0x5000039a8bc0075e,wwn-0x5000cca249d44a67,wwn-0x5000cca249d46730,wwn-0x5000cca25dcc719c,wwn-0x5000cca25dcc0241,wwn-0x5000039a8bc00765}
fileSystems."/storage" =
{ device = "root";
fsType = "zfs";
diff --git a/morph/grid/testing/config.json b/morph/grid/testing/config.json
index ec28840a2857c621a22658efc14368e4c07aa5db..a44b465f7f293f9d70c369a076c30b6cf810924f 100644
--- a/morph/grid/testing/config.json
+++ b/morph/grid/testing/config.json
@@ -1,6 +1,8 @@
{ "publicStoragePort": 8898
, "ristrettoSigningKeyPath": "./secrets/ristretto.signing-key"
, "stripeSecretKeyPath": "./secrets/stripe.secret"
+, "monitoringvpnKeyDir": "./secrets/monitoringvpn"
+, "monitoringvpnEndpoint": "monitoring.privatestorage-staging.com:51820"
, "passValue": 1000000
, "issuerDomains": [
"payments.privatestorage-staging.com"
diff --git a/morph/grid/testing/grid.nix b/morph/grid/testing/grid.nix
index 065cd5faa5a5e90a657d1fd1a38e79266e6b6475..8e68558a13c750eebac48c40dc0822d7f24db1bf 100644
--- a/morph/grid/testing/grid.nix
+++ b/morph/grid/testing/grid.nix
@@ -1,25 +1,63 @@
-# Load the helper function and call it with arguments tailored for the testing
-# grid. It will make the morph configuration for us. We share this function
-# with the production grid and have one fewer possible point of divergence.
-import ../../lib/make-grid.nix {
- name = "Testing";
- config = ./config.json;
- nodes = cfg:
- let
+# See morph/grid/local/grid.nix for additional commentary.
+let
+ pkgs = import <nixpkgs> { };
+
+ gridlib = import ../../lib;
+ rawConfig = pkgs.lib.trivial.importJSON ./config.json;
+ config = rawConfig // {
sshUsers = import ./secrets/users.nix;
- in {
- "payments" = import ../../lib/make-issuer.nix ({
- publicIPv4 = "18.194.183.13";
- inherit sshUsers;
- hardware = ../../lib/issuer-aws.nix;
- stateVersion = "19.03";
- } // cfg);
- "storage001" = import ../../lib/make-testing.nix (cfg // {
- publicIPv4 = "3.120.26.190";
- inherit sshUsers;
- hardware = ./testing001-hardware.nix;
- stateVersion = "19.03";
- });
+ # Get absolute vpn key directory path, as a string:
+ monitoringvpnKeyDir = toString ./. + "/${rawConfig.monitoringvpnKeyDir}";
+ };
+
+ payments = {
+ imports = [
+ gridlib.issuer
+ gridlib.hardware-aws
+ (gridlib.customize-issuer (config // {
+ monitoringvpnIPv4 = "172.23.23.11";
+ }))
+ ];
+ };
+
+ storage001 = let publicIPv4 = "3.120.26.190"; in {
+ imports = [
+ gridlib.storage
+ ./testing001-hardware.nix
+ (gridlib.customize-storage (config // {
+ inherit publicIPv4;
+ monitoringvpnIPv4 = "172.23.23.12";
+ stateVersion = "19.03";
+ }))
+ ];
+ };
+
+ monitoring = let publicIPv4 = "18.156.171.217"; in {
+ imports = [
+ gridlib.monitoring
+ gridlib.hardware-aws
+ (gridlib.customize-monitoring {
+ inherit hostsMap publicIPv4 vpnClientIPs nodeExporterTargets;
+ inherit (config) monitoringvpnKeyDir;
+ monitoringvpnIPv4 = "172.23.23.1";
+ stateVersion = "19.09";
+ })
+ ];
+ };
+
+ # TBD: derive these automatically:
+ hostsMap = {
+ "172.23.23.1" = [ "monitoring" "monitoring.monitoringvpn" ];
+ "172.23.23.11" = [ "payments" "payments.monitoringvpn" ];
+ "172.23.23.12" = [ "storage001" "storage001.monitoringvpn" ];
+ };
+ vpnClientIPs = [ "172.23.23.11" "172.23.23.12" ];
+ nodeExporterTargets = [ "monitoring" "payments" "storage001" ];
+
+in {
+ network = {
+ description = "PrivateStorage.io Testing Grid";
};
+ inherit payments monitoring storage001;
}
diff --git a/morph/lib/customize-issuer.nix b/morph/lib/customize-issuer.nix
new file mode 100644
index 0000000000000000000000000000000000000000..410bce47db83381f42949a4b8dd3f552f1b0bc5c
--- /dev/null
+++ b/morph/lib/customize-issuer.nix
@@ -0,0 +1,68 @@
+# Define a function which returns a value which fills in all the holes left by
+# ``issuer.nix``.
+{
+ # A path on the deployment system to a file containing the Ristretto signing
+ # key. This is used as the source of the Ristretto signing key morph
+ # secret.
+ ristrettoSigningKeyPath
+
+ # A path on the deployment system to a file containing the Stripe secret
+ # key. This is used as the source of the Stripe secret key morph secret.
+, stripeSecretKeyPath
+
+ # A path on the deployment system to a directory containing a number of
+ # VPN-related secrets. This is expected to contain a number of files named
+ # like ``<VPN IPv4 address>.key`` containing the VPN private key for the
+ # corresponding host. It must also contain ``server.pub`` and
+ # ``preshared.key`` holding the VPN server's public key and the pre-shared
+ # key, respectively. All of these things are used as the sources of various
+ # VPN-related morph secrets.
+, monitoringvpnKeyDir
+
+ # A string giving the IP address and port number (":"-separated) of the VPN
+ # server.
+, monitoringvpnEndpoint
+
+ # A string giving the VPN IPv4 address for this system.
+, monitoringvpnIPv4
+
+ # A set mapping usernames as strings to SSH public keys as strings. For
+ # each element of the site, the indicated user is configured on the system
+ # with the indicated SSH key as an authorized key.
+, sshUsers
+
+ # A string giving an email address to use for Let's Encrypt registration and
+ # certificate issuance.
+, letsEncryptAdminEmail
+
+ # A list of strings giving the domain names that point at this issuer
+ # system. These will all be included in Let's Encrypt certificate.
+, issuerDomains
+
+ # A list of strings giving CORS Origins will the issuer will be configured
+ # to allow.
+, allowedChargeOrigins
+, ...
+}: {
+ deployment.secrets = {
+ "ristretto-signing-key".source = ristrettoSigningKeyPath;
+ "stripe-secret-key".source = stripeSecretKeyPath;
+ "monitoringvpn-secret-key".source = "${monitoringvpnKeyDir}/${monitoringvpnIPv4}.key";
+ "monitoringvpn-preshared-key".source = "${monitoringvpnKeyDir}/preshared.key";
+ };
+
+ services.private-storage.sshUsers = sshUsers;
+ services.private-storage.monitoring.vpn.client = {
+ enable = true;
+ ip = monitoringvpnIPv4;
+ endpoint = monitoringvpnEndpoint;
+ endpointPublicKeyFile = "${monitoringvpnKeyDir}/server.pub";
+ };
+
+ services.private-storage-issuer = {
+ inherit letsEncryptAdminEmail allowedChargeOrigins;
+ domains = issuerDomains;
+ };
+
+ system.stateVersion = "19.03";
+}
diff --git a/morph/lib/customize-monitoring.nix b/morph/lib/customize-monitoring.nix
new file mode 100644
index 0000000000000000000000000000000000000000..331a2dbd7c9875acbf72ae22da786f4bc868cfc0
--- /dev/null
+++ b/morph/lib/customize-monitoring.nix
@@ -0,0 +1,54 @@
+# Define a function which returns a value which fills in all the holes left by
+# ``monitoring.nix``.
+{
+ # A set mapping VPN IP addresses as strings to lists of hostnames as
+ # strings. The system's ``/etc/hosts`` will be populated with this
+ # information. Apart from helping with normal forward resolution, this
+ # *also* gives us reverse resolution from the VPN IPs to hostnames which
+ # allows Grafana to show us hostnames instead of VPN IP addresses.
+ hostsMap
+
+ # See ``customize-issuer.nix``.
+, monitoringvpnKeyDir
+, monitoringvpnIPv4
+
+ # XXX To be removed
+, publicIPv4
+
+ # A list of VPN IP addresses as strings indicating which clients will be
+ # allowed onto the VPN.
+, vpnClientIPs
+
+ # A list of VPN clients (IP addresses or hostnames) as strings indicating
+ # which nodes to scrape "nodeExporter" metrics from.
+, nodeExporterTargets
+
+ # A list of VPN clients (IP addresses or hostnames) as strings indicating
+ # which nodes to scrape "nginxExporter" metrics from.
+, nginxExporterTargets ? []
+
+ # A string giving the NixOS state version for the system.
+, stateVersion
+, ...
+}: {
+ deployment.targetHost = publicIPv4;
+ deployment.secrets = {
+ "monitoringvpn-private-key".source = "${monitoringvpnKeyDir}/server.key";
+ "monitoringvpn-preshared-key".source = "${monitoringvpnKeyDir}/preshared.key";
+ };
+ networking.hosts = hostsMap;
+
+ services.private-storage.monitoring.vpn.server = {
+ enable = true;
+ ip = monitoringvpnIPv4;
+ inherit vpnClientIPs;
+ pubKeysPath = monitoringvpnKeyDir;
+ };
+
+ services.private-storage.monitoring.prometheus = {
+ inherit nodeExporterTargets;
+ inherit nginxExporterTargets;
+ };
+
+ system.stateVersion = stateVersion;
+}
diff --git a/morph/lib/customize-storage.nix b/morph/lib/customize-storage.nix
new file mode 100644
index 0000000000000000000000000000000000000000..b9d25e985977bc77d08c21fd869644ab170c3b64
--- /dev/null
+++ b/morph/lib/customize-storage.nix
@@ -0,0 +1,43 @@
+# Define a function which returns a value which fills in all the holes left by
+# ``storage.nix``.
+{
+ # See ``customize-issuer.nix``
+ ristrettoSigningKeyPath
+, monitoringvpnKeyDir
+, monitoringvpnEndpoint
+, monitoringvpnIPv4
+, sshUsers
+
+ # An integer giving the value of a single pass in byte×months.
+, passValue
+
+ # An integer giving the port number to include in Tahoe storage service
+ # advertisements and on which to listen for storage connections.
+, publicStoragePort
+
+ # XXX To be removed
+, publicIPv4
+
+ # A string giving the NixOS state version for the system.
+, stateVersion
+, ...
+}: {
+ deployment.secrets = {
+ "ristretto-signing-key".source = ristrettoSigningKeyPath;
+ "monitoringvpn-secret-key".source = "${monitoringvpnKeyDir}/${monitoringvpnIPv4}.key";
+ "monitoringvpn-preshared-key".source = "${monitoringvpnKeyDir}/preshared.key";
+ };
+
+ services.private-storage = {
+ inherit sshUsers publicIPv4 passValue publicStoragePort;
+ };
+
+ services.private-storage.monitoring.vpn.client = {
+ enable = true;
+ ip = monitoringvpnIPv4;
+ endpoint = monitoringvpnEndpoint;
+ endpointPublicKeyFile = "${monitoringvpnKeyDir}/server.pub";
+ };
+
+ system.stateVersion = stateVersion;
+}
diff --git a/morph/lib/default.nix b/morph/lib/default.nix
new file mode 100644
index 0000000000000000000000000000000000000000..bdd92f4bfe52eba2e19df3ac73a087a4af4a53dc
--- /dev/null
+++ b/morph/lib/default.nix
@@ -0,0 +1,16 @@
+# Gather up the grid library functionality and present it in a (somewhat)
+# coherent public interface. Application code should prefer these names over
+# directly importing the source files in this directory.
+{
+ hardware-aws = import ./issuer-aws.nix;
+ hardware-virtual = import ./hardware-virtual.nix;
+
+ issuer = import ./issuer.nix;
+ customize-issuer = import ./customize-issuer.nix;
+
+ storage = import ./storage.nix;
+ customize-storage = import ./customize-storage.nix;
+
+ monitoring = import ./monitoring.nix;
+ customize-monitoring = import ./customize-monitoring.nix;
+}
diff --git a/morph/grid/local/virtual-hardware.nix b/morph/lib/hardware-virtual.nix
similarity index 100%
rename from morph/grid/local/virtual-hardware.nix
rename to morph/lib/hardware-virtual.nix
diff --git a/morph/lib/issuer.nix b/morph/lib/issuer.nix
new file mode 100644
index 0000000000000000000000000000000000000000..417ef7965ea0120322995059fcca7a5a9afe2543
--- /dev/null
+++ b/morph/lib/issuer.nix
@@ -0,0 +1,56 @@
+# This is all of the static NixOS system configuration necessary to specify an
+# "issuer"-type system. The configuration has various holes in it which must
+# be filled somehow. These holes correspond to configuration which is not
+# statically known. This value is suitable for use as a module to be imported
+# into a more complete system configuration. It is expected that the holes
+# will be filled by a sibling module created by ``customize-issuer.nix``.
+rec {
+ deployment = {
+ secrets = {
+ "ristretto-signing-key" = {
+ destination = "/run/keys/ristretto.signing-key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "zkapissuer.service"];
+ };
+ "stripe-secret-key" = {
+ destination = "/run/keys/stripe.secret-key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "zkapissuer.service"];
+ };
+
+ "monitoringvpn-secret-key" = {
+ destination = "/run/keys/monitoringvpn/client.key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
+ };
+ "monitoringvpn-preshared-key" = {
+ destination = "/run/keys/monitoringvpn/preshared.key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
+ };
+ };
+ };
+
+ imports = [
+ ../../nixos/modules/issuer.nix
+ ../../nixos/modules/monitoring/vpn/client.nix
+ ../../nixos/modules/monitoring/exporters/node.nix
+ ];
+
+ services.private-storage-issuer = {
+ enable = true;
+ tls = true;
+ ristrettoSigningKeyPath = deployment.secrets.ristretto-signing-key.destination;
+ stripeSecretKeyPath = deployment.secrets.stripe-secret-key.destination;
+ database = "SQLite3";
+ databasePath = "/var/db/vouchers.sqlite3";
+ };
+}
diff --git a/morph/lib/make-grid.nix b/morph/lib/make-grid.nix
deleted file mode 100644
index de10df1e9a62ee0ac7fde98070743ee4a9cf484b..0000000000000000000000000000000000000000
--- a/morph/lib/make-grid.nix
+++ /dev/null
@@ -1,19 +0,0 @@
-# Define a function for making a morph configuration for a storage grid. It
-# takes two arguments. A string like "Production" giving the name of the grid
-# and a function that takes the grid configuration as an argument and returns
-# a set of nodes specifying the addresses and NixOS configurations for each
-# server in the morph network.
-{ name, config, nodes }:
-let
- pkgs = import <nixpkgs> { };
- # Load our JSON configuration for later use.
- cfg = pkgs.lib.trivial.importJSON config;
-in
-{
- network = {
- # Make all of the hosts in this network use the nixpkgs we pinned above.
- inherit pkgs;
- # This is just for human consumption as far as I can tell.
- description = "PrivateStorage.io ${name} Grid";
- };
-} // (nodes cfg)
diff --git a/morph/lib/make-issuer.nix b/morph/lib/make-issuer.nix
deleted file mode 100644
index bbdf0cebbf770738e9ccb997daec75e58df021b5..0000000000000000000000000000000000000000
--- a/morph/lib/make-issuer.nix
+++ /dev/null
@@ -1,91 +0,0 @@
-{ hardware
-, ristrettoSigningKeyPath
-, stripeSecretKeyPath
-, issuerDomains
-, letsEncryptAdminEmail
-, allowedChargeOrigins
-, sshUsers
-, stateVersion
-, publicIPv4
-, monitoringvpnKeyDir ? null
-, monitoringvpnIPv4 ? null
-, monitoringvpnEndpoint ? null
-, ...
-}: let
-
- enableVpn = monitoringvpnKeyDir != null &&
- monitoringvpnIPv4 != null &&
- monitoringvpnEndpoint != null;
-
- vpnSecrets = if !enableVpn then {} else {
- "monitoringvpn-secret-key" = {
- source = monitoringvpnKeyDir + "/${monitoringvpnIPv4}.key";
- destination = "/run/keys/monitoringvpn/client.key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
- };
- "monitoringvpn-preshared-key" = {
- source = monitoringvpnKeyDir + "/preshared.key";
- destination = "/run/keys/monitoringvpn/preshared.key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
- };
- };
-
-in rec {
- deployment = {
- targetHost = publicIPv4;
-
- secrets = {
- "ristretto-signing-key" = {
- source = ristrettoSigningKeyPath;
- destination = "/run/keys/ristretto.signing-key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "zkapissuer.service"];
- };
- "stripe-secret-key" = {
- source = stripeSecretKeyPath;
- destination = "/run/keys/stripe.secret-key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "zkapissuer.service"];
- };
- } // vpnSecrets;
- };
-
- imports = [
- hardware
- ../../nixos/modules/issuer.nix
- ../../nixos/modules/monitoring/vpn/client.nix
- ../../nixos/modules/monitoring/exporters/node.nix
- ];
-
- services.private-storage.sshUsers = sshUsers;
- services.private-storage-issuer = {
- enable = true;
- tls = true;
- ristrettoSigningKeyPath = deployment.secrets.ristretto-signing-key.destination;
- stripeSecretKeyPath = deployment.secrets.stripe-secret-key.destination;
- database = "SQLite3";
- databasePath = "/var/db/vouchers.sqlite3";
- inherit letsEncryptAdminEmail;
- domains = issuerDomains;
- inherit allowedChargeOrigins;
- };
-
- system.stateVersion = stateVersion;
-
- services.private-storage.monitoring.vpn.client = if !enableVpn then {} else {
- enable = true;
- ip = monitoringvpnIPv4;
- endpoint = monitoringvpnEndpoint;
- endpointPublicKeyFile = monitoringvpnKeyDir + "/server.pub";
- };
-}
diff --git a/morph/lib/make-monitoring.nix b/morph/lib/make-monitoring.nix
deleted file mode 100644
index 8eb53c6db9552e65a84e5d3e5564449db437e902..0000000000000000000000000000000000000000
--- a/morph/lib/make-monitoring.nix
+++ /dev/null
@@ -1,77 +0,0 @@
-{ publicIPv4
-, hardware
-, publicStoragePort
-, ristrettoSigningKeyPath
-, passValue
-, sshUsers
-, stateVersion
-, monitoringvpnIPv4 ? null
-, monitoringvpnKeyDir ? null
-, vpnClientIPs ? null
-, nodeExporterTargets ? []
-, nginxExporterTargets ? []
-, hostsMap ? {}
-, ... }: let
-
- enableVpn = monitoringvpnKeyDir != null &&
- monitoringvpnIPv4 != null &&
- vpnClientIPs != null;
-
- vpnSecrets = if !enableVpn then {} else {
- "monitoringvpn-private-key" = {
- source = monitoringvpnKeyDir + "/server.key";
- destination = "/run/keys/monitoringvpn/server.key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
- };
- "monitoringvpn-preshared-key" = {
- source = monitoringvpnKeyDir + "/preshared.key";
- destination = "/run/keys/monitoringvpn/preshared.key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
- };
- };
-
-in rec {
-
- deployment = {
- targetHost = publicIPv4;
- secrets = vpnSecrets;
- };
-
- imports = [
- hardware
- ../../nixos/modules/monitoring/vpn/server.nix
- ../../nixos/modules/monitoring/server/grafana.nix
- ../../nixos/modules/monitoring/server/prometheus.nix
- ../../nixos/modules/monitoring/exporters/node.nix
- # Loki 0.3.0 from Nixpkgs 19.09 is too old and does not work:
- # ../../nixos/modules/monitoring/server/loki.nix
- ];
-
- services.private-storage.monitoring.vpn.server = if !enableVpn then {} else {
- enable = true;
- ip = monitoringvpnIPv4;
- inherit vpnClientIPs;
- pubKeysPath = monitoringvpnKeyDir;
- };
-
- services.private-storage.monitoring.grafana = {
- domain = "grafana.grid.private.storage";
- prometheusUrl = "http://localhost:9090/";
- lokiUrl = "http://localhost:3100/";
- };
-
- services.private-storage.monitoring.prometheus = {
- inherit nodeExporterTargets;
- inherit nginxExporterTargets;
- };
-
- system.stateVersion = stateVersion;
-
- networking.hosts = hostsMap;
-}
diff --git a/morph/lib/make-storage.nix b/morph/lib/make-storage.nix
deleted file mode 100644
index af0867c8b8342e31393f19a76a7cbfc4c95f86c9..0000000000000000000000000000000000000000
--- a/morph/lib/make-storage.nix
+++ /dev/null
@@ -1,70 +0,0 @@
-# Define the function that defines the node.
-{ cfg # Get the configuration that's specific to this node.
-, hardware # The path to the hardware configuration for this node.
-, publicStoragePort # The storage port number on which to accept connections.
-, ristrettoSigningKeyPath # The *local* path to the Ristretto signing key file.
-, passValue # Bytes component of size×time value of passes.
-, sshUsers # Users for which to configure SSH access to this node.
-, stateVersion # The value for system.stateVersion on this node.
- # This value determines the NixOS release with
- # which your system is to be compatible, in order
- # to avoid breaking some software such as
- # database servers. You should change this only
- # after NixOS release notes say you should.
-, ...
-}: rec {
- deployment = {
- targetHost = cfg.publicIPv4;
-
- secrets = {
- "ristretto-signing-key" = {
- source = ristrettoSigningKeyPath;
- destination = "/run/keys/ristretto.signing-key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- # Service name here matches the name defined by our tahoe-lafs nixos
- # module. It would be nice to not have to hard-code it here. Can we
- # extract it from the tahoe-lafs nixos module somehow?
- action = ["sudo" "systemctl" "restart" "tahoe.storage.service"];
- };
- };
- };
-
- # Any extra NixOS modules to load on this server.
- imports = [
- # Include the results of the hardware scan.
- hardware
- # Configure it as a system operated by 100TB.
- ../../nixos/modules/100tb.nix
- # Bring in our module for configuring the Tahoe-LAFS service and other
- # Private Storage-specific things.
- ../../nixos/modules/private-storage.nix
- ];
-
- # Pass the configuration specific to this host to the 100TB module to be
- # expanded into a complete system configuration. See the 100tb module for
- # handling of this value.
- #
- # The module name is quoted because `1` makes `100tb` look an awful lot like
- # it should be a number.
- "100tb".config = cfg;
-
- # Turn on the Private Storage (Tahoe-LAFS) service.
- services.private-storage = {
- # Yep. Turn it on.
- enable = true;
- # Get the public IPv4 address from the node configuration.
- inherit (cfg) publicIPv4;
- # And the port to operate on is specified via parameter.
- inherit publicStoragePort;
- # Give it the Ristretto signing key, too, to support authorization.
- ristrettoSigningKeyPath = deployment.secrets.ristretto-signing-key.destination;
- # Assign the configured pass value.
- inherit passValue;
- # It gets the users, too.
- inherit sshUsers;
- };
-
- system.stateVersion = stateVersion;
-}
diff --git a/morph/lib/make-testing.nix b/morph/lib/make-testing.nix
deleted file mode 100644
index 3f6e767db5ee734a8ca2314b216d4fa602c01907..0000000000000000000000000000000000000000
--- a/morph/lib/make-testing.nix
+++ /dev/null
@@ -1,80 +0,0 @@
-{ publicIPv4
-, hardware
-, publicStoragePort
-, ristrettoSigningKeyPath
-, passValue
-, sshUsers
-, stateVersion
-, monitoringvpnKeyDir ? null
-, monitoringvpnIPv4 ? null
-, monitoringvpnEndpoint ? null
-, ... }: let
-
- enableVpn = monitoringvpnKeyDir != null &&
- monitoringvpnIPv4 != null &&
- monitoringvpnEndpoint != null;
-
- vpnSecrets = if !enableVpn then {} else {
- "monitoringvpn-secret-key" = {
- source = monitoringvpnKeyDir + "/${monitoringvpnIPv4}.key";
- destination = "/run/keys/monitoringvpn/client.key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
- };
- "monitoringvpn-preshared-key" = {
- source = monitoringvpnKeyDir + "/preshared.key";
- destination = "/run/keys/monitoringvpn/preshared.key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
- };
- };
-
-in rec {
-
- deployment = {
- targetHost = publicIPv4;
-
- secrets = {
- "ristretto-signing-key" = {
- source = ristrettoSigningKeyPath;
- destination = "/run/keys/ristretto.signing-key";
- owner.user = "root";
- owner.group = "root";
- permissions = "0400";
- # Service name here matches the name defined by our tahoe-lafs nixos
- # module. It would be nice to not have to hard-code it here. Can we
- # extract it from the tahoe-lafs nixos module somehow?
- action = ["sudo" "systemctl" "restart" "tahoe.storage.service"];
- };
- } // vpnSecrets;
- };
-
- imports = [
- hardware
- ../../nixos/modules/private-storage.nix
- ../../nixos/modules/monitoring/vpn/client.nix
- ../../nixos/modules/monitoring/exporters/node.nix
- ];
-
- services.private-storage =
- { enable = true;
- inherit publicIPv4;
- inherit publicStoragePort;
- ristrettoSigningKeyPath = deployment.secrets.ristretto-signing-key.destination;
- inherit passValue;
- inherit sshUsers;
- };
-
- system.stateVersion = stateVersion;
-
- services.private-storage.monitoring.vpn.client = if !enableVpn then {} else {
- enable = true;
- ip = monitoringvpnIPv4;
- endpoint = monitoringvpnEndpoint;
- endpointPublicKeyFile = monitoringvpnKeyDir + "/server.pub";
- };
-}
diff --git a/morph/lib/monitoring.nix b/morph/lib/monitoring.nix
new file mode 100644
index 0000000000000000000000000000000000000000..b48820f0941694869fdda06e724ba1ae714b5993
--- /dev/null
+++ b/morph/lib/monitoring.nix
@@ -0,0 +1,37 @@
+# Similar to ``issuer.nix`` but for a "monitoring"-type system. Holes are
+# filled by ``customize-monitoring.nix``.
+rec {
+ deployment = {
+ secrets = {
+ "monitoringvpn-private-key" = {
+ destination = "/run/keys/monitoringvpn/server.key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
+ };
+ "monitoringvpn-preshared-key" = {
+ destination = "/run/keys/monitoringvpn/preshared.key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
+ };
+ };
+ };
+
+ imports = [
+ ../../nixos/modules/monitoring/vpn/server.nix
+ ../../nixos/modules/monitoring/server/grafana.nix
+ ../../nixos/modules/monitoring/server/prometheus.nix
+ ../../nixos/modules/monitoring/exporters/node.nix
+ # Loki 0.3.0 from Nixpkgs 19.09 is too old and does not work:
+ # ../../nixos/modules/monitoring/server/loki.nix
+ ];
+
+ services.private-storage.monitoring.grafana = {
+ domain = "monitoring.private.storage";
+ prometheusUrl = "http://localhost:9090/";
+ lokiUrl = "http://localhost:3100/";
+ };
+}
diff --git a/morph/lib/storage.nix b/morph/lib/storage.nix
new file mode 100644
index 0000000000000000000000000000000000000000..1cac51b43aa38fb90a535fd34ba53363fc0cdbaa
--- /dev/null
+++ b/morph/lib/storage.nix
@@ -0,0 +1,51 @@
+# Similar to ``issuer.nix`` but for a "storage"-type system. Holes are filled
+# by ``customize-storage.nix``.
+rec {
+ deployment = {
+ secrets = {
+ "ristretto-signing-key" = {
+ destination = "/run/keys/ristretto.signing-key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ # Service name here matches the name defined by our tahoe-lafs nixos
+ # module. It would be nice to not have to hard-code it here. Can we
+ # extract it from the tahoe-lafs nixos module somehow?
+ action = ["sudo" "systemctl" "restart" "tahoe.storage.service"];
+ };
+ "monitoringvpn-secret-key" = {
+ destination = "/run/keys/monitoringvpn/client.key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
+ };
+ "monitoringvpn-preshared-key" = {
+ destination = "/run/keys/monitoringvpn/preshared.key";
+ owner.user = "root";
+ owner.group = "root";
+ permissions = "0400";
+ action = ["sudo" "systemctl" "restart" "wireguard-monitoringvpn.service"];
+ };
+ };
+ };
+
+ # Any extra NixOS modules to load on this server.
+ imports = [
+ # Bring in our module for configuring the Tahoe-LAFS service and other
+ # Private Storage-specific things.
+ ../../nixos/modules/private-storage.nix
+ # Connect to the monitoringvpn.
+ ../../nixos/modules/monitoring/vpn/client.nix
+ # Expose base system metrics over the monitoringvpn.
+ ../../nixos/modules/monitoring/exporters/node.nix
+ ];
+
+ # Turn on the Private Storage (Tahoe-LAFS) service.
+ services.private-storage = {
+ # Yep. Turn it on.
+ enable = true;
+ # Give it the Ristretto signing key to support authorization.
+ ristrettoSigningKeyPath = deployment.secrets.ristretto-signing-key.destination;
+ };
+}
diff --git a/morph/grid/local/vagrant-guest.nix b/morph/lib/vagrant-guest.nix
similarity index 100%
rename from morph/grid/local/vagrant-guest.nix
rename to morph/lib/vagrant-guest.nix
diff --git a/shell.nix b/shell.nix
index 6e46c9ca0feaa3ab6fbd22c1228ec786a49e79b6..2c1c5123da656d34fafe0883b50ef49c578c6c8b 100644
--- a/shell.nix
+++ b/shell.nix
@@ -8,5 +8,6 @@ pkgs.mkShell {
buildInputs = [
pkgs.morph
stable2105.vagrant
+ pkgs.jp
];
}
diff --git a/tools/create-vpn-keys.sh b/tools/create-vpn-keys.sh
new file mode 100755
index 0000000000000000000000000000000000000000..e092a8ced698bd3a3bb2d4acc3ca07a3a8e6032d
--- /dev/null
+++ b/tools/create-vpn-keys.sh
@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+
+# Scope: Create wireguard keys for all monitoringVPN hosts
+# Parameters:
+# file: path to grid.nix of morph deployment
+#
+# Output: Key files for all monitoring VPN hosts in secrets/monitoringvpn
+# relative to the grid.nix
+#
+# The server key will also be symlinked to server.{key,pub}.
+
+set -euxo pipefail
+
+umask 077
+
+if [[ $# -ne 1 ]]; then
+ echo "Illegal number of parameters. Expected: file (path of grid.nix)"
+ exit 2
+fi
+
+SRC=$(dirname $0)
+VPN_SECRETS=$(dirname $1)/secrets/monitoringvpn
+
+CONFIG=$(nix-instantiate --strict --json --eval "${SRC}"/get-vpn-config.nix --arg pathToGrid "${1}")
+
+MONITORING_IPS=$(echo $CONFIG | jp --unquoted "join(' ', clientIPs)")
+VPNSERVER_IP=$(echo $CONFIG | jp --unquoted "serverIP")
+
+mkdir -p "${VPN_SECRETS}"
+
+for i in $MONITORING_IPS $VPNSERVER_IP; do
+ wg genkey | tee "${VPN_SECRETS}"/${i}.key | wg pubkey > "${VPN_SECRETS}"/${i}.pub
+done
+
+wg genpsk > "${VPN_SECRETS}"/preshared.key
+
+ln -fs $VPNSERVER_IP.key "${VPN_SECRETS}"/server.key
+ln -fs $VPNSERVER_IP.pub "${VPN_SECRETS}"/server.pub
+
+# EOF
diff --git a/tools/get-vpn-config.nix b/tools/get-vpn-config.nix
new file mode 100644
index 0000000000000000000000000000000000000000..7753292aa83c4b63be7457228de0cd84e6eeefa2
--- /dev/null
+++ b/tools/get-vpn-config.nix
@@ -0,0 +1,19 @@
+# A function that accepts a path to a grid.nix-style file and returns a set
+# with two attributes:
+#
+# * serverIP - a string giving the VPN IP address of the grid's VPN server.
+#
+# * clientIPs - a list of strings giving the VPN IP addresses of all of the
+# grid's VPN clients.
+#
+{ pathToGrid }:
+let
+ grid = import pathToGrid;
+ vpnConfig = node: node.services.private-storage.monitoring.vpn or null;
+ vpnClientIP = node: (vpnConfig node).client.ip or null;
+ vpnServerIP = node: (vpnConfig node).server.ip or null;
+in
+{
+ "serverIP" = vpnServerIP grid.monitoring;
+ "clientIPs" = builtins.filter (x: x != null) (map vpnClientIP (builtins.attrValues grid));
+}