README.rst

Developer documentation
=======================

Building
--------

The build system uses `Nix`_ which must be installed before anything can be built.
Start by setting up the development/operations environment::

  $ nix-shell

Testing
-------

The test system uses `Nix`_ which must be installed before any tests can be run.

Unit tests are run using this command::

  $ nix-build nixos/unit-tests.nix

Unit tests are also run on CI.

The system tests are run using this command::

  $ nix-build nixos/system-tests.nix

The system tests boot QEMU VMs which prevents them from running on CI at this time.
The build requires > 10 GB of disk space,
and the VMs might be timing out on slow or busy machines.
If you run into timeouts,
try `raising the number of retries <https://whetstone.privatestorage.io/privatestorage/PrivateStorageio/-/blob/e8233d2/nixos/modules/tests/run-introducer.py#L55-62>`_.

It is also possible go through the testing script interactively - useful for debugging::

  $ nix-build -A private-storage.driver nixos/system-tests.nix

This will give you a result symlink in the current directory.
Inside that is bin/nixos-test-driver which gives you a kind of REPL for interacting with the VMs.
The kind of `Python in this testScript <https://whetstone.privatestorage.io/privatestorage/PrivateStorageio/-/blob/78881a3/nixos/modules/tests/private-storage.nix#L180>`_ is what you can enter into this REPL.
Consult the `official documentation on NixOS Tests <https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests>`_ for more information.

Updatings Pins
--------------

Nixpkgs
```````

To update the version of NixOS we deploy with, run:

.. code: shell

   nix-shell --run 'update-nixpkgs'

That will update ``nixpkgs-2015.json`` to the latest release on the nixos-21.05 channel.

To update the channel, the script will need to be updated,
along with the filenames that have the channel in them.


Interactions
------------

Storage-Time Purchase (ie Payment)
``````````````````````````````````

.. uml::

   actor User as User
   participant GridSync
   participant ZKAPAuthorizer
   database    ZKAPAuthzDB as "ZKAPAuthorizer"
   participant Browser
   participant PaymentServer as "Payment Server"
   database    PaymentServerDB as "Payment Server"
   participant WebServer as "Web Server"
   participant Stripe

   User           -> GridSync       : buy storage-time
   activate User
   GridSync       -> GridSync       : generate voucher
   GridSync       -> ZKAPAuthorizer : redeem voucher
   activate ZKAPAuthorizer
   ZKAPAuthorizer -> ZKAPAuthzDB    : store voucher
   ZKAPAuthorizer -> GridSync       : acknowledge
   GridSync       -> Browser        : open payment page

   loop until redeemed
       GridSync       -> ZKAPAuthorizer : query voucher state
       ZKAPAuthorizer -> GridSync       : not paid
   end

   Browser       -> WebServer  : request payment form
   WebServer     -> Browser    : payment form
   Browser       -> User       : Payment form displayed
   activate User
   User          -> Browser    : Submit payment details
   Browser       -> Stripe     : Submit payment details

   alt payment details accepted
       Stripe         -> Browser         : details okay, return card token
       Browser        -> PaymentServer   : create charge using card token
       PaymentServer  -> Stripe          : charge card using token
       note left: the user has now paid for the service
       Stripe         -> PaymentServer   : acknowledge
       PaymentServer  -> PaymentServerDB : store voucher paid state
   else payment details rejected
       Stripe    -> Browser        : payment failure
   end

   Browser       -> User       : payment processing results displayed
   deactivate User

   group repeat for each redemption group
       ZKAPAuthorizer -> ZKAPAuthzDB    : generate and store random tokens
       ZKAPAuthorizer -> PaymentServer  : redeem voucher with blinded tokens
       PaymentServer  -> ZKAPAuthorizer : return signatures for blinded tokens
       ZKAPAuthorizer -> ZKAPAuthzDB    : store unblinded signatures for tokens
       note right: the user has now been authorized to use the service
   end
   deactivate ZKAPAuthorizer

   loop until redeemed
       GridSync       -> ZKAPAuthorizer : query voucher state
       ZKAPAuthorizer -> GridSync       : fully redeemed
   end

   GridSync           -> User           : storage-time available displayed
   deactivate User

Storage-Time Spending (ie Use)
``````````````````````````````

.. uml::

   participant MagicFolder
   participant TahoeLAFS as "Tahoe-LAFS"
   participant ZKAPAuthorizer
   database    ZKAPAuthzDB as "ZKAPAuthorizer"
   participant StorageNode as "Storage Node"
   participant SpendingService as "Spending Service"

   MagicFolder -> TahoeLAFS : store some data
   activate MagicFolder

   TahoeLAFS -> ZKAPAuthorizer : store some data
   activate TahoeLAFS

   loop until tokens accepted
       ZKAPAuthorizer <- ZKAPAuthzDB : load some tokens
       ZKAPAuthorizer -> StorageNode : store some data using these tokens
       StorageNode -> SpendingService : spend these tokens

       alt spent tokens
           SpendingService -> StorageNode: already spent, rejected
           StorageNode -> ZKAPAuthorizer: already spent, rejected
       else fresh tokens
           SpendingService -> StorageNode: spent, allowed
           StorageNode -> ZKAPAuthorizer: data stored
           ZKAPAuthorizer -> ZKAPAuthzDB: discard spent tokens
       end

       ZKAPAuthorizer -> TahoeLAFS: data stored
       deactivate TahoeLAFS
       TahoeLAFS -> MagicFolder: data stored
       deactivate MagicFolder
   end

.. include::
      ../../../morph/grid/local/README.rst

.. _Nix: https://nixos.org/nix