Switch to using mach-nix for python packaging.

Created by: tomprince

Used in https://whetstone.privatestorage.io/privatestorage/PrivateStorageio/-/merge_requests/166.

What constraints does this name need to satisfy?

Created by: tomprince

See https://packaging.python.org/guides/distributing-packages-using-setuptools/#name and https://www.python.org/dev/peps/pep-0508/#names

Can you put those in a comment above the name?

developer/maintainer oriented docs pointing at Niv docs and/or giving brief summary of how to update dependencies with Niv would probably be helpful

Created by: tomprince

Done.

Review: Changes requested

Thanks. Some comments inline. In addition to those, while I see how the new mach-nix Tahoe packaging works in principle, in practice I must not really understand because I cannot reproduce a successful mach-nix-based Tahoe build independently. Using PrivateStorageio/mach-nix master@HEAD and a mach nix pypi db from this morning gives me errors either about cryptography == 35.0.0 being unsatisfiable or if I loosen that version constraint, errors about setuptools_rust being missing.

I'd like to have a more solid grasp of mach-nix before switching ZKAPAuthorizer to it to avoid nix packaging for this project becoming an obstacle for me.

Created by: tomprince

Using PrivateStorageio/mach-nix master@HEAD

The correct branch to use is the merged branch.

Created by: tomprince

I used this, instead of directly using the environment, to control which executables are included in the derivation. This is mainly because this package included in environment.systemPackages.

So, to avoid polluting the system PATH with unnecessary extra executables? That seems reasonable. Maybe worth converting into a comment in the code here.

Created by: tomprince

I added this derivation, and turned default.nix into an attrset rather than a derivation to get the right binaries into the derivation used by PrivateStorageio and also expose the zkaupauthorizer derivation to tests.nix. It'd also be possible to expose the later as an attribute on this derivation; I'm not sure which approach is better (if either).

Created by: tomprince

This is exposing the python environment with test dependencies, for use in shell.nix.

I want to like this. I gave up on Nix-supplied development environment a long time ago (due to the many bugs in and/or poor documentation about this features of nixpkgs Python support works). I would quite enjoy not using virtualenv anymore.

But this doesn't actually provide a development environment, does it? The _zkapauthorizer package in the resulting Python environment comes from the Nix store - ie, this is not like a "pip install -e ." situation. So I wonder what this is for.

The only thing I can think of is doing some very complex interactive debugging. For anything simple enough, the 1m setup time seems like it would be too much of a barrier.

Created by: tomprince

If you aren't currently using shell.nix, then I won't worry more about getting it working in this PR.

It would definitely be possible to get the equivalent of pip install -e . here. I did something similar for zkap-spending-service, though it might be more complex here to get the right dependencies.

Or, maybe something like

let
  sources = import nix/sources.nix;
in
{ pkgs ? import sources.release2015 {}
, tahoe-lafs-source ? "tahoe-lafs"
}:
  let
    privatesotrage = pkgs.callPackage ./default.nix {
      inherit tahoe-lafs-source;
    };
  in
    pkgs.mkShell {
      inputsFrom = [
        privatestorage.zkapauthorizer
      ];
      packages = [
        pkgs.niv
      ];
    }

might work after this PR.

Created by: tomprince

While I definitely appreciate the benifits of sphix/restructuredtext of markdown, I originally wrote all the link in here with markdown syntax. I wonder if there is enough benefit of restructuredtext over markdown, given that both github and gitlab use markdown as the syntax for all the places you can write text in them.

My main argument for rst is that I am more familiar with it than I am with markdown. That's not much of an argument, I know. Preferences of GitHub and GitLab are somewhat compelling, I guess (although can't GitLab also do rst if you ask it nicely enough?)

I don't know anything about what happens if you use Sphinx and markdown together. Do you lose functionality (I'm thinking about linking from rst to md and other similar interactions)?

If it all just works, I guess I don't really object (but, of course, don't convert it in this PR).

Created by: tomprince

I don't have particularly strong feelings either way, and probably won't spend a lot of time thinking about it. This was mostly idle musing prompted by the combination of renaming this file to match its actual file-type and the my "muscle memory" for writing links in markdown format led to the incorrect formatting initially.

I do wonder a bit about the value of sphix (as opposed to rst vs md), given that we don't have the built docs available anywhere. Without that, I am much more like to go look at the source (either locally or in the VCS host), and neither github nor gitlab handle any of the sphinx extensions.

Since niv doesn't support floating versions (I guess?) then I suppose the CI workflow for "test against master" would be something like:

• run niv update tahoe-lafs-master --branch master
• Run build w/ tahoe-lafs-source set to tahoe-lafs-master

(after initially creating tahoe-lafs-master with niv, pointed at some arbitrary master revision)

?

Hm. Or I guess, assuming that CI doesn't try to commit its pin change, you don't actually need tahoe-lafs-master - you can just update tahoe-lafs and then you don't even have to tell the Nix expression to use a different version...

maybe worth having a ticket filed for switching back to master/a release - w/ links to tickets/prs that need to be resolved in mach-nix for this to be possible.

Do you know if this expression can eventually go into Tahoe itself?

Created by: tomprince

Maybe? I'd probably a little bit ambivalent about putting there in its current state, as a public interface. In particular, all the "just-so" choices of providers for various dependencies to get things to work. And mach-nix doesn't really have a good story around composing mach-nix.buildPythonPackage built packages, so I'm not sure how easy it would be for us to then re-use it.