notes
=====
* Build a PrivateStorage-related Android application
* Build an app that can interact with magic-folders somehow
* Build an app that can read magic-folders and download linked files
* Build libchallenge_bypass_ristretto for armv7a and aarch64 Android
* nixpkgs *used to* use "crates-io.nix" which was quite friendly to cross-compilation!
But it got thrown in the trash and now you are directed to naersk or crate2nix.
* cross-compilation with crate2nix expressions is not straightforward
Does it work at all, for anything?
Who knows.
There's a ticket open.
I haven't yet figured out how to make it go.
* naersk looks super easy to use!
But does it support cross-compilation?
It is not clear.
(Naive?) Attempts to do so on nixpkgs master@HEAD fail with infinite recursion errors.
On nixos-22.05 with some patches applied ... maybe it will work?
(As of 2022-12-05 16:04, build is still running)
* Either way we need to preserve our postInstall step to get the .h and .pc files
* Build an app that runs a Haskell Tahoe-LAFS implementation on Android
* Reflex-FRP / Obsidian Systems Obelisk?
https://reflex-frp.org/tutorial
https://github.com/obsidiansystems/obelisk
* The Obelisk readme basically works.
* Adding a native extension seems tricky.
* Obelisk pins a reflex-platform
* reflex-platform pins a nixpkgs
* reflex-platform various arguments to control nixpkgs
* nixpkgsFunc to completely replace the nixpkgs used
* nixpkgsOverlays to contribute overlays to nixpkgs
* config to pass other config to nixpkgs
* Obelisk does not accept nixpkgsFunc or nixpkgsOverlays
* But it does accept reflex-platform-func
* So override this with a version that passes an overlay to reflex, I guess
* Build an app that runs the Python Tahoe-LAFS implementation on Android
* Build Tahoe-LAFS & ZKAPAuthorizer for Android
* Build Python for Android
* Terms
* "Build system" is the system where you compile a package.
* "Host system" is the system where you run the package.
* Try using nix (nixpkgs 3d58e3a31ee55554fe26650bb59db7d6d0e97625 / https://github.com/NixOS/nixpkgs/pull/201734 rebased onto staging) ::
nix build .#pkgsCross.aarch64-android.python3Minimal
* Or, Chaquopy has a build
* Build TahoeLAFS/ZKAPAuthorizer dependencies for Android
* zfec
* ::
nix build .#pkgsCross.aarch64-android.python3Minimal.pkgs.zfec
However, this fails.
Building an aarch64-android wheel package fails::
running bdist_wheel
Traceback (most recent call last):
File "/build/wheel-0.37.1-source/nix_run_setup", line 8, in <module>
exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
File "setup.py", line 4, in <module>
setup(maintainer=u'Alex Grönholm')
File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
.
.
.
File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
.
.
.
File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 26, in <module>
get_config_var("LDSHARED")
File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/_distutils/sysconfig.py", line 574, in get_config_var
return get_config_vars().get(name)
File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/_distutils/sysconfig.py", line 553, in get_config_vars
_config_vars = sysconfig.get_config_vars().copy()
File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/sysconfig.py", line 615, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/sysconfig.py", line 477, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata__linux_aarch64-linux-android'
``_sysconfigdata__linux_aarch64-linux-android`` is meant to hold information about the host platform for the software to be built.
CPython doesn't have a good cross-compilation system.
It will guess a sysconfidata module name to import or use a value given by _PYTHON_SYSCONFIGDATA_NAME.
nixpkgs defines a value for _PYTHON_SYSCONFIGDATA_NAME.
In this case it sets it to ``_sysconfigdata__linux_aarch64-linux-android``.
The CPython that is running is the "Python for build" interpreter -
the interpreter that is able to run on the build system.
It is searching for the sysconfigdata for the "Python for host" interpreter -
the interpreter that is able to run on the system where the package being built will be used.
The "Python for build" interpreter has ``site-packages/_sysconfigdata__linux_x86_64-linux-gnu.py``
which is a symlink to ``_sysconfigdata__linux_x86_64-linux-gnu.py`
which exists
and contains information about the build system.
The pkgsCross.aarch64-android.python3Minimal.pkgs.zfec derivation has PYTHONPATH set in its environment.
The value (split up for readability) is
* /nix/store/53afd480zidx2p9j9c7dsgckjgfk66h1-python3-minimal-aarch64-unknown-linux-android-3.10.8/lib/python3.10/site-packages
* /nix/store/azdg3l16q2d03g8drgr94bg1069mfzpq-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages
* /nix/store/spk2md1jgwrg6id10j1rl9448s4naa94-python3.10-setuptools-65.3.0/lib/python3.10/site-packages
* /nix/store/537cvh8mqdaglc0an8ndmzszllh1lwac-python3-minimal-3.10.8/lib/python3.10/site-packages
The first entry is a directory which contains ``_sysconfigdata__linux_.py``
which is a symlink to ``../_sysconfigdata__linux_.py``
which exists
and contains information about the host system.
The ``_sysconfigdata_*`` file is
* named by ``sysconfig._get_sysconfigdata_name``
* and written by ``sysconfig._generate_posix_vars``
* which is called by ``sysconfig._main``
* which is invoked by the ``pybuilddir.txt`` Makefile target
* which is depended on by the ``sharedmods`` Makefile target
* which is depended on by the ``build_all`` and ``sharedinstall`` Makefile targets
* ``build_all`` is the default Make target
* nixpkgs python3Minimal expression defines a postFixup hook
* ``sysconfigdataHook`` ::
sysconfigdataHook() {
if [ "$1" = '${placeholder "out"}' ]; then
export _PYTHON_HOST_PLATFORM='${pythonHostPlatform}'
export _PYTHON_SYSCONFIGDATA_NAME='${pythonSysconfigdataName}'
fi
}
addEnvHooks "$hostOffset" sysconfigdataHook
* In postFixup (after build, before install) sysconfigdataHook is added as a $hostOffset env hook.
This means ... ??????????????????????
* Some observations
* On a native build
* buildPhase started
* lots of core compiling
* ``LD_LIBRARY_PATH=/build/Python-3.10.8 ./python -E -S -m sysconfig --generate-posix-vars``
* lots of extension module compiling
* ``_sysconfigdata__linux_x86_64-linux-gnu.py`` generated/written
* On an aarch64-multiplatform build
* buildPhase started
* lots of core compiling
* ``_PYTHON_PROJECT_BASE=/build/Python-3.10.8 _PYTHON_HOST_PLATFORM=linux-aarch64 PYTHONPATH=./Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__linux_aarch64-linux-gnu python -S -m sysconfig --generate-posix-vars``
* lots of extension module compiling
* ``_sysconfigdata__linux_aarch64-linux-gnu.py'`` generated/written
* On an aarch64-android build
* buildPhase started
* lots of core compiling
* ``_PYTHON_PROJECT_BASE=/build/Python-3.10.8 _PYTHON_HOST_PLATFORM=linux-aarch64 PYTHONPATH=./Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__linux_ python -S -m sysconfig --generate-posix-vars``
* Uh oh.
``_sysconfigdata__linux_``
Why?
* We expect to see ``_sysconfigdata__linux_aarch64-linux-android``
* This means the "abi flags" are ""
The "platform" is "linux" (questionable...)
The "multiarch" is "aarch64-linux-android" (is that coherent?)
* The exact string might not really matter, we just need it to agree.
Where is the multiarch suffix going?
Or who is setting this unrelated value?
* overrideAttrs ``NIX_DEBUG = "6"``; into the derivation for extremely verbose output ...
* The nixpkgs expression does not result in this string
(by inspection with builtins.trace)
* python-cryptography
* cffi-based projects seem to be tricky!
* python-challenge-bypass-ristretto
* crossenv can do it! Mostly!
See crossenv subdirectory.
These work:
* python-challenge-bypass-ristretto
Note that you at least need to install cffi in the
build-python environment. Maybe some other deps require
this as well..
* bcrypt
* cbor2
* netifaces
* pycddl
* zope.interface
* pyyaml
These fail:
* pynacl
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
* cryptography
ld: error: unable to find library -lpthread
* Integrate a Python Tahoe-LAFS runtime with a GUI
* Build a GUI