项目场景:

python3.7下载pyspark

问题描述:

C:\Users\e123>pip install pyspark==2.3.3 -i https://pypi.douban.com/simple
Looking in indexes: https://pypi.douban.com/simple
Collecting pyspark==2.3.3
  Using cached https://pypi.doubanio.com/packages/af/7a/fa556cd533f106747e115c74c2f6d2fc864dc1dad25fdf991b06ab2c9403/pyspark-2.3.3.tar.gz (211.9 MB)
    ERROR: Command errored out with exit status 1:
     command: 'd:\python\python37\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\KiloYe\\AppData\\Local\\Temp\\pip-install-jn9lxybr\\pyspark_3e7f6123657a46a89187bc62803de0d6\\setup.py'"'"'; __file__='"'"'C:\\Users\\KiloYe\\AppData\\Local\\Temp\\pip-install-jn9lxybr\\pyspark_3e7f6123657a46a89187bc62803de0d6\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\KiloYe\AppData\Local\Temp\pip-pip-egg-info-g3ci27t5'
         cwd: C:\Users\KiloYe\AppData\Local\Temp\pip-install-jn9lxybr\pyspark_3e7f6123657a46a89187bc62803de0d6\
    Complete output (47 lines):
    Could not import pypandoc - required to package PySpark
    WARNING: The wheel package is not available.
      ERROR: Command errored out with exit status 1:
       command: 'd:\python\python37\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\KiloYe\\AppData\\Local\\Temp\\pip-wheel-t9kotk1b\\pypandoc_59519f743b2540cca0333c4d7e5a44d9\\setup.py'"'"'; __file__='"'"'C:\\Users\\KiloYe\\AppData\\Local\\Temp\\pip-wheel-t9kotk1b\\pypandoc_59519f743b2540cca0333c4d7e5a44d9\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\KiloYe\AppData\Local\Temp\pip-wheel-s47kz_dh'
           cwd: C:\Users\KiloYe\AppData\Local\Temp\pip-wheel-t9kotk1b\pypandoc_59519f743b2540cca0333c4d7e5a44d9\
      Complete output (8 lines):
      no pandoc found, building platform unspecific wheel...
      use 'python setup.py download_pandoc' to download pandoc.
      usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: setup.py --help [cmd1 cmd2 ...]
         or: setup.py --help-commands
         or: setup.py cmd --help

      error: invalid command 'bdist_wheel'
      ----------------------------------------
      ERROR: Failed building wheel for pypandoc
    ERROR: Failed to build one or more wheels
    Traceback (most recent call last):
      File "d:\python\python37\lib\site-packages\setuptools\installer.py", line 128, in fetch_build_egg
        subprocess.check_call(cmd)
      File "d:\python\python37\lib\subprocess.py", line 363, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['d:\\python\\python37\\python.exe', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', 'C:\\Users\\KiloYe\\AppData\\Local\\Temp\\tmpgxftys98', '--quiet', 'pypandoc']' returned non-zero exit status 1.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\KiloYe\AppData\Local\Temp\pip-install-jn9lxybr\pyspark_3e7f6123657a46a89187bc62803de0d6\setup.py", line 224, in <module>
        'Programming Language :: Python :: Implementation :: PyPy']
      File "d:\python\python37\lib\site-packages\setuptools\__init__.py", line 143, in setup
        _install_setup_requires(attrs)
      File "d:\python\python37\lib\site-packages\setuptools\__init__.py", line 138, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "d:\python\python37\lib\site-packages\setuptools\dist.py", line 698, in fetch_build_eggs
        replace_conflicting=True,
      File "d:\python\python37\lib\site-packages\pkg_resources\__init__.py", line 783, in resolve
        replace_conflicting=replace_conflicting
      File "d:\python\python37\lib\site-packages\pkg_resources\__init__.py", line 1066, in best_match
        return self.obtain(req, installer)
      File "d:\python\python37\lib\site-packages\pkg_resources\__init__.py", line 1078, in obtain
        return installer(requirement)
      File "d:\python\python37\lib\site-packages\setuptools\dist.py", line 754, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "d:\python\python37\lib\site-packages\setuptools\installer.py", line 130, in fetch_build_egg
        raise DistutilsError(str(e))
    distutils.errors.DistutilsError: Command '['d:\\python\\python37\\python.exe', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', 'C:\\Users\\KiloYe\\AppData\\Local\\Temp\\tmpgxftys98', '--quiet', 'pypandoc']' returned non-zero exit status 1.
    ----------------------------------------
WARNING: Discarding https://pypi.doubanio.com/packages/af/7a/fa556cd533f106747e115c74c2f6d2fc864dc1dad25fdf991b06ab2c9403/pyspark-2.3.3.tar.gz#sha256=8c6cc6b489de5ccea2e8e0f93cc907c1c155ef8ee363c8e4929c7dd0d4038cae (from https://pypi.doubanio.com/simple/pyspark/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pyspark==2.3.3 (from versions: 2.1.2, 2.1.3, 2.2.0.post0, 2.2.1, 2.2.2, 2.2.3, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.4.5, 2.4.6, 2.4.7, 2.4.8, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.1.1, 3.1.2)
ERROR: No matching distribution found for pyspark==2.3.3

原因分析:

注意到报错的代码

Could not import pypandoc - required to package PySpark

说明缺少了依赖

解决方案:

手动安装缺少的依赖

pip install pypandoc

再次安装就成功了

Logo

华为开发者空间,是为全球开发者打造的专属开发空间,汇聚了华为优质开发资源及工具,致力于让每一位开发者拥有一台云主机,基于华为根生态开发、创新。

更多推荐