Developer documentation
=======================

Building
--------

The build system uses `Nix`_ which must be installed before anything can be built.
Start by setting up the development/operations environment::

  $ nix-shell

Testing
-------

The test system uses `Nix`_ which must be installed before any tests can be run.

Unit tests are run using this command::

  $ nix-build --attr unit-tests

Unit tests are also run on CI.

The system tests are run using this command::

  $ nix-build --attr system-tests

The system tests boot QEMU VMs which prevents them from running on CI at this time.
The build requires > 10 GB of disk space,
and the VMs might be timing out on slow or busy machines.
If you run into timeouts,
try `raising the number of retries <https://whetstone.privatestorage.io/privatestorage/PrivateStorageio/-/blob/e8233d2/nixos/modules/tests/run-introducer.py#L55-62>`_.

It is also possible go through the testing script interactively - useful for debugging::

  $ nix-build --attr system-tests.private-storage.driver

This will give you a result symlink in the current directory.
Inside that is bin/nixos-test-driver which gives you a kind of REPL for interacting with the VMs.
The kind of `Python in this testScript <https://whetstone.privatestorage.io/privatestorage/PrivateStorageio/-/blob/78881a3/nixos/modules/tests/private-storage.nix#L180>`_ is what you can enter into this REPL.
Consult the `official documentation on NixOS Tests <https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests>`_ for more information.

Updatings Pins
--------------

Nixpkgs
```````

To update the version of NixOS we deploy with, run:

.. code: shell

   nix-shell --run 'update-nixpkgs'

That will update ``nixpkgs-2015.json`` to the latest release on the nixos-21.05 channel.

To update the channel, the script will need to be updated,
along with the filenames that have the channel in them.

Gitlab Repositories
```````````````````
To update the version of packages we import from gitlab, run:

.. code: shell

   nix-shell --command 'tools/update-gitlab nixos/pkgs/<package>/repo.json'

That will update the package to point at the latest version of the project.\
The command uses branch and repository owner specified in the ``repo.json`` file,
but you can override them by passing the ``--branch`` or ``-owner`` arguments to the command.
A specific revision can also be pinned, by passing ``-rev``.

Interactions
------------

Storage-Time Purchase (ie Payment)
``````````````````````````````````

.. uml::

   actor User as User
   participant GridSync
   participant ZKAPAuthorizer
   database    ZKAPAuthzDB as "ZKAPAuthorizer"
   participant Browser
   participant PaymentServer as "Payment Server"
   database    PaymentServerDB as "Payment Server"
   participant WebServer as "Web Server"
   participant Stripe

   User           -> GridSync       : buy storage-time
   activate User
   GridSync       -> GridSync       : generate voucher
   GridSync       -> ZKAPAuthorizer : redeem voucher
   activate ZKAPAuthorizer
   ZKAPAuthorizer -> ZKAPAuthzDB    : store voucher
   ZKAPAuthorizer -> GridSync       : acknowledge
   GridSync       -> Browser        : open payment page

   loop until redeemed
       GridSync       -> ZKAPAuthorizer : query voucher state
       ZKAPAuthorizer -> GridSync       : not paid
   end

   Browser       -> WebServer  : request payment form
   WebServer     -> Browser    : payment form
   Browser       -> User       : Payment form displayed
   activate User
   User          -> Browser    : Submit payment details
   Browser       -> Stripe     : Submit payment details

   alt payment details accepted
       Stripe         -> Browser         : details okay, return card token
       Browser        -> PaymentServer   : create charge using card token
       PaymentServer  -> Stripe          : charge card using token
       note left: the user has now paid for the service
       Stripe         -> PaymentServer   : acknowledge
       PaymentServer  -> PaymentServerDB : store voucher paid state
   else payment details rejected
       Stripe    -> Browser        : payment failure
   end

   Browser       -> User       : payment processing results displayed
   deactivate User

   group repeat for each redemption group
       ZKAPAuthorizer -> ZKAPAuthzDB    : generate and store random tokens
       ZKAPAuthorizer -> PaymentServer  : redeem voucher with blinded tokens
       PaymentServer  -> ZKAPAuthorizer : return signatures for blinded tokens
       ZKAPAuthorizer -> ZKAPAuthzDB    : store unblinded signatures for tokens
       note right: the user has now been authorized to use the service
   end
   deactivate ZKAPAuthorizer

   loop until redeemed
       GridSync       -> ZKAPAuthorizer : query voucher state
       ZKAPAuthorizer -> GridSync       : fully redeemed
   end

   GridSync           -> User           : storage-time available displayed
   deactivate User

Storage-Time Spending (ie Use)
``````````````````````````````

.. uml::

   participant MagicFolder
   participant TahoeLAFS as "Tahoe-LAFS"
   participant ZKAPAuthorizer
   database    ZKAPAuthzDB as "ZKAPAuthorizer"
   participant StorageNode as "Storage Node"
   participant SpendingService as "Spending Service"

   [-> MagicFolder: upload triggered
   activate MagicFolder

   MagicFolder -> TahoeLAFS : store some data
   activate TahoeLAFS

   TahoeLAFS -> ZKAPAuthorizer : store some data
   activate ZKAPAuthorizer

   loop until tokens accepted
       ZKAPAuthorizer <- ZKAPAuthzDB : load some tokens
       ZKAPAuthorizer -> StorageNode : store some data using these tokens
       StorageNode -> SpendingService : spend these tokens

       alt spent tokens
           SpendingService -> StorageNode: already spent, rejected
           StorageNode -> ZKAPAuthorizer: already spent, rejected
       else fresh tokens
           SpendingService -> StorageNode: accepted
       end
   end

   StorageNode -> ZKAPAuthorizer: data stored
   deactivate ZKAPAuthorizer
   ZKAPAuthorizer -> ZKAPAuthzDB: discard spent tokens
   ZKAPAuthorizer -> TahoeLAFS: data stored
   deactivate TahoeLAFS
   TahoeLAFS -> MagicFolder: data stored
   deactivate MagicFolder

.. include::
      ../../morph/grid/local/README.rst

.. _Nix: https://nixos.org/nix