Skip to content
Snippets Groups Projects
Notes.rst 8.14 KiB
Newer Older
  • Learn to ignore specific revisions
  • * Build a PrivateStorage-related Android application
    
    Jean-Paul Calderone's avatar
    Jean-Paul Calderone committed
    
    
      * Build an app that can interact with magic-folders somehow
    
    Jean-Paul Calderone's avatar
    Jean-Paul Calderone committed
    
    
        * Build an app that can read magic-folders and download linked files
    
    Jean-Paul Calderone's avatar
    Jean-Paul Calderone committed
    
    
          * Build an app that runs the Python Tahoe-LAFS implementation on Android
    
    Jean-Paul Calderone's avatar
    Jean-Paul Calderone committed
    
    
    	* Build Tahoe-LAFS & ZKAPAuthorizer for Android
    
    Jean-Paul Calderone's avatar
    Jean-Paul Calderone committed
    
    
    	  * Build Python for Android
    
    	    * Terms
    
    Jean-Paul Calderone's avatar
    Jean-Paul Calderone committed
    
    
    	      * "Build system" is the system where you compile a package.
    	      * "Host system" is the system where you run the package.
    
    	    * Try using nix (nixpkgs 3d58e3a31ee55554fe26650bb59db7d6d0e97625 / https://github.com/NixOS/nixpkgs/pull/201734 rebased onto staging) ::
    
    		nix build .#pkgsCross.aarch64-android.python3Minimal
    
    	    * Or, Chaquopy has a build
    
    	  * Build TahoeLAFS/ZKAPAuthorizer dependencies for Android
    
    	    * zfec
    
    	    * ::
    
    		nix build .#pkgsCross.aarch64-android.python3Minimal.pkgs.zfec
    
    	      However, this fails.
    	      Building an aarch64-android wheel package fails::
    
                      running bdist_wheel
                      Traceback (most recent call last):
                        File "/build/wheel-0.37.1-source/nix_run_setup", line 8, in <module>
                          exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
                        File "setup.py", line 4, in <module>
                          setup(maintainer=u'Alex Grönholm')
                        File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
                          return distutils.core.setup(**attrs)
                      .
                      .
                      .
                        File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
                          module = import_module(match.group('module'))
                        File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/importlib/__init__.py", line 126, in import_module
                          return _bootstrap._gcd_import(name[level:], package, level)
                      .
                      .
                      .
                        File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 26, in <module>
                          get_config_var("LDSHARED")
                        File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/_distutils/sysconfig.py", line 574, in get_config_var
                          return get_config_vars().get(name)
                        File "/nix/store/87qbq45psc97az9h5qmn6csfvrj9gl6g-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages/setuptools/_distutils/sysconfig.py", line 553, in get_config_vars
                          _config_vars = sysconfig.get_config_vars().copy()
                        File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/sysconfig.py", line 615, in get_config_vars
                          _init_posix(_CONFIG_VARS)
                        File "/nix/store/lygssilaq8p1a85pmbnfd7k2yw76kf2k-python3-minimal-3.10.8/lib/python3.10/sysconfig.py", line 477, in _init_posix
                          _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
                      ModuleNotFoundError: No module named '_sysconfigdata__linux_aarch64-linux-android'
    
    	      ``_sysconfigdata__linux_aarch64-linux-android`` is meant to hold information about the host platform for the software to be built.
    	      CPython doesn't have a good cross-compilation system.
    	      It will guess a sysconfidata module name to import or use a value given by _PYTHON_SYSCONFIGDATA_NAME.
    	      nixpkgs defines a value for _PYTHON_SYSCONFIGDATA_NAME.
    	      In this case it sets it to ``_sysconfigdata__linux_aarch64-linux-android``.
    	      The CPython that is running is the "Python for build" interpreter -
    	      the interpreter that is able to run on the build system.
    	      It is searching for the sysconfigdata for the "Python for host" interpreter -
    	      the interpreter that is able to run on the system where the package being built will be used.
    	      The "Python for build" interpreter has ``site-packages/_sysconfigdata__linux_x86_64-linux-gnu.py``
    	      which is a symlink to ``_sysconfigdata__linux_x86_64-linux-gnu.py`
    	      which exists
    	      and contains information about the build system.
    	      The pkgsCross.aarch64-android.python3Minimal.pkgs.zfec derivation has PYTHONPATH set in its environment.
    	      The value (split up for readability) is
    
    
    Jean-Paul Calderone's avatar
    Jean-Paul Calderone committed
    	      * /nix/store/53afd480zidx2p9j9c7dsgckjgfk66h1-python3-minimal-aarch64-unknown-linux-android-3.10.8/lib/python3.10/site-packages
    	      * /nix/store/azdg3l16q2d03g8drgr94bg1069mfzpq-python3.10-bootstrapped-pip-22.2.2/lib/python3.10/site-packages
    	      * /nix/store/spk2md1jgwrg6id10j1rl9448s4naa94-python3.10-setuptools-65.3.0/lib/python3.10/site-packages
    	      * /nix/store/537cvh8mqdaglc0an8ndmzszllh1lwac-python3-minimal-3.10.8/lib/python3.10/site-packages
    
    
    	      The first entry is a directory which contains ``_sysconfigdata__linux_.py``
    	      which is a symlink to ``../_sysconfigdata__linux_.py``
    	      which exists
    	      and contains information about the host system.
    
    	      The ``_sysconfigdata_*`` file is
    
    	      * named by ``sysconfig._get_sysconfigdata_name``
    	      * and written by ``sysconfig._generate_posix_vars``
    	      * which is called by ``sysconfig._main``
    	      * which is invoked by the ``pybuilddir.txt`` Makefile target
    	      * which is depended on by the ``sharedmods`` Makefile target
    	      * which is depended on by the ``build_all`` and ``sharedinstall`` Makefile targets
    	      * ``build_all`` is the default Make target
    	      * nixpkgs python3Minimal expression defines a postFixup hook
    		* ``sysconfigdataHook`` ::
    
                        sysconfigdataHook() {
                          if [ "$1" = '${placeholder "out"}' ]; then
                            export _PYTHON_HOST_PLATFORM='${pythonHostPlatform}'
                            export _PYTHON_SYSCONFIGDATA_NAME='${pythonSysconfigdataName}'
                          fi
                        }
    
                        addEnvHooks "$hostOffset" sysconfigdataHook
    
    
    		* In postFixup (after build, before install) sysconfigdataHook is added as a $hostOffset env hook.
    		  This means ... ??????????????????????
    
    		* Some observations
    
    		* On a native build
    		  * buildPhase started
    		  * lots of core compiling
    		  * ``LD_LIBRARY_PATH=/build/Python-3.10.8 ./python -E -S -m sysconfig --generate-posix-vars``
    		  * lots of extension module compiling
    		  * ``_sysconfigdata__linux_x86_64-linux-gnu.py`` generated/written
    
    		* On an aarch64-multiplatform build
    		  * buildPhase started
    		  * lots of core compiling
    		  * ``_PYTHON_PROJECT_BASE=/build/Python-3.10.8 _PYTHON_HOST_PLATFORM=linux-aarch64 PYTHONPATH=./Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__linux_aarch64-linux-gnu python -S -m sysconfig --generate-posix-vars``
    		  * lots of extension module compiling
    		  * ``_sysconfigdata__linux_aarch64-linux-gnu.py'`` generated/written
    
    		* On an aarch64-android build
    		  * buildPhase started
    		  * lots of core compiling
    		  * ``_PYTHON_PROJECT_BASE=/build/Python-3.10.8 _PYTHON_HOST_PLATFORM=linux-aarch64 PYTHONPATH=./Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__linux_ python -S -m sysconfig --generate-posix-vars``
    
    		    * Uh oh.
    		      ``_sysconfigdata__linux_``
    		      Why?
    
    		      * We expect to see ``_sysconfigdata__linux_aarch64-linux-android``
    
    			* This means the "abi flags" are ""
    			  The "platform" is "linux" (questionable...)
    			  The "multiarch" is "aarch64-linux-android" (is that coherent?)
    
    			* The exact string might not really matter, we just need it to agree.
    			  Where is the multiarch suffix going?
    
    			  Or who is setting this unrelated value?
    
    			* overrideAttrs ``NIX_DEBUG = "6"``; into the derivation for extremely verbose output ...
    
    		    * The nixpkgs expression does not result in this string
    		      (by inspection with builtins.trace)
    
    
    	    * python-cryptography
    	    * python-challenge-bypass-ristretto
    
    	* Integrate a Python Tahoe-LAFS runtime with a GUI
    
    	  * Build a GUI