yoshiislandblog.net
元営業の駆け出しアラサーSEが、休日にMACと戯れた際の殴り書きメモ。日々勉強。日々進歩。

pipでインストールできない問題(Can’t connect to HTTPS URL because the SSL module is not available)に直面した

2021-02-25

requestsをpipでインストールできない問題に直面した

結果として、修正不可能で「pyenvを入れ直す」という対応をしたが、
一応どんな対応をしたかログ残しする

一刻も早く解決したい人は、入れ直しがおすすめです
解決編:pyenvとvirtualenvをきれいに消してから入れ直す


  1. 問題
  2. Mac OSとXCodeの最新化したが解決しない
  3. 「brew update」と「openssl」の最新化
  4. pythonとopensslをインストールし直す
  5. pipを入れ直そうとしたができない
  6. openssl周りもう一回確認する
  7. opensslのパスを修正

問題

pipでrequestsがインストールできない
前はできてたはずなのにきっかけも思いつかない
pyenvで環境を変えても同じエラー

% pip install requests
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting requests
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/requests/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/requests/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/requests/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/requests/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/requests/
  Could not fetch URL https://pypi.org/simple/requests/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/requests/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping
  Could not find a version that satisfies the requirement requests (from versions: )
No matching distribution found for requests
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping

pipをアップグレードしてみるができない

% pip install -U pip
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping
Requirement already up-to-date: pip in /Users/hogehoge/.pyenv/versions/3.6.6/envs/python_scraping_work/lib/python3.6/site-packages (10.0.1)
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping

Mac OSとXCodeの最新化したが解決しない

XCodeが古いかもという思いつきから、
XCodeをApp Storeからアップデートしようとしたら、MacOSが古かったからXCodeがなんか無くなっていた

MacOSを最新のBig Sirにアップデートした

その後、XCodeを最新化した

「brew update」と「openssl」の最新化

しかし状況変わらない、、、
以下の記事を参考に、「brew update」と「openssl」の最新化をしてみる

参考:ImportError: cannot import name HTTPSHandler using PIP

ライセンス承諾しろと言われる

% brew update
Error: You have not agreed to the Xcode license. Please resolve this by running:
  sudo xcodebuild -license accept

言われるがままにコマンドを打つ

% sudo xcodebuild -license accept
Password:

晴れて「brew update」できた

% brew update

「openssl」を最新化しようとするとエラー
しかももうインストールできてると書いてある

% brew install openssl
Error:
  homebrew-core is a shallow clone.
  homebrew-cask is a shallow clone.
To `brew update`, first run:
  git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
  git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask fetch --unshallow
These commands may take a few minutes to run due to the large size of the repositories.
This restriction has been made on GitHub's request because updating shallow
clones is an extremely expensive operation due to the tree layout and traffic of
Homebrew/homebrew-core and Homebrew/homebrew-cask. We don't do this for you
automatically to avoid repeatedly performing an expensive unshallow operation in
CI systems (which should instead be fixed to not use shallow clones). Sorry for
the inconvenience!
Error: openssl@1.1 1.1.1d is already installed.
To upgrade to 1.1.1j, run:
  brew upgrade openssl@1.1

よくわからないが、一応エラー文に書いてあるコマンドを打っておく

% git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
% git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask fetch --unshallow

さっきできてたっぽいけど、一応もう一回「brew update」しておく

% brew update
Updated 1 tap (homebrew/core).
==> Updated Formulae
mpw                           nvc                           tfsec

先程の「brew install openssl」のエラー文に「やるならupgrade」と書いてあったのでupgradeする

またよくわからないエラー

% brew upgrade openssl
...
Error: Could not symlink Frameworks/Python.framework/Headers
Target /usr/local/Frameworks/Python.framework/Headers
is a symlink belonging to python@3.9. You can unlink it:
  brew unlink python@3.9

To force the link and overwrite all conflicting files:
  brew link --overwrite python@2

To list all files that would be deleted:
  brew link --overwrite --dry-run python@2

言われるがままエラー文に書いてあったコマンドを打つ

% brew unlink python@3.9
Unlinking /usr/local/Cellar/python@3.9/3.9.2... 25 symlinks removed.

今度はエラーを吐かなくなった

% brew upgrade openssl
Updating Homebrew...
Warning: openssl 1.1.1j already installed

pythonとopensslをインストールし直す

opensslは大丈夫っぽいので、pipを実行した時のエラーをもう一度よく見る

% pip
...
    from urllib.request import (urlopen, urlretrieve, Request, url2pathname,
ImportError: cannot import name 'HTTPSHandler'

以下のリンクを参考にpythonとopensslをインストールし直す
参考:python/pip error on osx

pythonをインストールし直してみたが、後からよく考えたらpyenv使っているのだからシステムのPythonをインストールし直しても意味なかった、、

% brew reinstall python
Warning: Building python@3.9 from source:
  The bottle needs the Apple Command Line Tools to be installed.
  You can install them, if desired, with:
    xcode-select --install
...

言われるがままこちらも打っておく

% xcode-select --install

opensslもインストールしなおそうとするがエラー

% brew reinstall openssl
...
Error: An exception occurred within a child process:
  FormulaUnavailableError: No available formula with the name "/usr/local/opt/python@2/.brew/python@2.rb".

以下のリンクを参考にpythonをアンインストール
hash md5 error on Mac #540

% brew uninstall python@2
Uninstalling /usr/local/Cellar/python@2/2.7.15_1... (4,741 files, 84MB)
% brew reinstall openssl

今度はopenssl再インストールできた
あれ、デジャブ

pipを入れ直そうとしたができない

pipの状況が変わらない

% pip
...
    from urllib.request import (urlopen, urlretrieve, Request, url2pathname,
ImportError: cannot import name 'HTTPSHandler'

よし、pipを入れ直そう、以下リンクを参考に再インストール
参考:1、pipの再インストール
参考:Installation

入れ直せない、、、

% python -m pip uninstall pip
/Users/hogehoge/.pyenv/versions/3.5.0/bin/python: Error while finding spec for 'pip.__main__' (<class 'ImportError'>: cannot import name 'HTTPSHandler'); 'pip' is a package and cannot be directly executed

openssl周りもう一回確認する

以下リンクを参考にopenssl周りもう一回確認する
ImportError: cannot import name HTTPSHandler using PIP

% openssl version -a
LibreSSL 2.8.3
built on: date not available
platform: information not available
options:  bn(64,64) rc4(16x,int) des(idx,cisc,16,int) blowfish(idx)
compiler: information not available
OPENSSLDIR: "/private/etc/ssl"
% which openssl
/usr/bin/openssl
% brew update
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Casks
nextcloud
% brew install openssl
Warning: openssl@1.1 1.1.1j is already installed and up-to-date.
To reinstall 1.1.1j, run:
  brew reinstall openssl@1.1
% brew link --overwrite --dry-run openssl
Would remove:


If you need to have this software first in your PATH instead consider running:
  echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc

言われるがまま実行

% echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc
% brew link openssl --overwrite
Warning: Refusing to link macOS provided/shadowed software: openssl@1.1
If you need to have openssl@1.1 first in your PATH, run:
  echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc

For compilers to find openssl@1.1 you may need to set:
  export LDFLAGS="-L/usr/local/opt/openssl@1.1/lib"
  export CPPFLAGS="-I/usr/local/opt/openssl@1.1/include"

For pkg-config to find openssl@1.1 you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig"
% export LDFLAGS="-L/usr/local/opt/openssl@1.1/lib"
% export CPPFLAGS="-I/usr/local/opt/openssl@1.1/include"
% brew uninstall python
Error: Refusing to uninstall /usr/local/Cellar/python@3.9/3.9.2
because it is required by ansible and sphinx-doc, which are currently installed.
You can override this and force removal with:
  brew uninstall --ignore-dependencies python

警告が出たが、またインストールすれば良いのでuninstallしてしまう

% brew uninstall --ignore-dependencies python
Uninstalling /usr/local/Cellar/python@3.9/3.9.2... (8,680 files, 129.8MB)

これはできなかった

% brew install python --with-brewed-openssl
...
Error: invalid option: --with-brewed-openssl

これはできた

% brew install python
...
==> Summary
🍺  /usr/local/Cellar/python@3.9/3.9.2: 3,935 files, 66.0MB

しかし、状況変わらず、、

opensslのパスを修正

opensslのパスがおかしいかも?

参考:homebrewでinstlalしたOpenSSLが反映されず、OSX標準のOpenSSLが使用されてしまう場合の対処方法

% which openssl
/usr/bin/openssl
% brew info openssl
openssl@1.1: stable 1.1.1j (bottled) [keg-only]
Cryptography and SSL/TLS Toolkit
https://openssl.org/
/usr/local/Cellar/openssl@1.1/1.1.1j (8,071 files, 18.5MB)
  Poured from bottle on 2021-02-21 at 22:53:51
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/openssl@1.1.rb
License: OpenSSL
==> Caveats
A CA file has been bootstrapped using certificates from the system
keychain. To add additional certificates, place .pem files in
  /usr/local/etc/openssl@1.1/certs

and run
  /usr/local/opt/openssl@1.1/bin/c_rehash

openssl@1.1 is keg-only, which means it was not symlinked into /usr/local,
because macOS provides LibreSSL.

If you need to have openssl@1.1 first in your PATH, run:
  echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc

For compilers to find openssl@1.1 you may need to set:
  export LDFLAGS="-L/usr/local/opt/openssl@1.1/lib"
  export CPPFLAGS="-I/usr/local/opt/openssl@1.1/include"

For pkg-config to find openssl@1.1 you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig"

==> Analytics
install: 709,314 (30 days), 2,262,160 (90 days), 8,340,730 (365 days)
install-on-request: 110,277 (30 days), 361,164 (90 days), 1,214,446 (365 days)
build-error: 0 (30 days)
% echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc

% tail -1 ~/.zshrc
export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"


% source ~/.zshrc

パスが反映された

% which openssl
/usr/local/opt/openssl@1.1/bin/openssl

ここまで、状況変わらない
何だか同じことを何度もしている気がする、、

解決編に続く。
解決編:pyenvとvirtualenvをきれいに消してから入れ直す