Introduction

When Python's requests library makes HTTPS connections, it verifies the server's SSL certificate against a bundle of trusted Certificate Authorities. If verification fails, the connection is rejected with an SSL error. This commonly occurs with corporate proxies, self-signed certificates, outdated CA bundles, or misconfigured servers.

Symptoms

  • requests.exceptions.SSLError: HTTPSConnectionPool(host='api.example.com', port=443): Max retries exceeded with url: /data (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)')))
  • SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain
  • curl works but Python requests fails on the same URL

Common Causes

  • Corporate MITM proxy replacing SSL certificates with internal CA
  • Outdated certifi CA bundle missing newer certificate authorities
  • Self-signed certificates in development/staging environments
  • Server not sending intermediate certificate chain
  • OpenSSL version mismatch between Python and system libraries

Step-by-Step Fix

  1. 1.Diagnose the certificate issue:
  2. 2.```bash
  3. 3.# Test with openssl
  4. 4.openssl s_client -connect api.example.com:443 -showcerts

# Test with Python's ssl module directly python -c " import ssl, socket ctx = ssl.create_default_context() try: with ctx.wrap_socket(socket.socket(), server_hostname='api.example.com') as s: s.connect(('api.example.com', 443)) print('Certificate OK') except ssl.SSLError as e: print(f'SSL Error: {e}') " ```

  1. 1.Update the certifi CA bundle:
  2. 2.```bash
  3. 3.pip install --upgrade certifi
  4. 4.python -c "import certifi; print(certifi.where())"
  5. 5.`
  6. 6.Use a custom CA bundle for corporate certificates:
  7. 7.```python
  8. 8.import requests

response = requests.get( 'https://api.example.com/data', verify='/path/to/corporate-ca-bundle.pem' ) ```

  1. 1.Combine system CA with custom CA:
  2. 2.```python
  3. 3.import certifi
  4. 4.import os

# Append corporate CA to certifi bundle ca_bundle = certifi.where() os.environ['REQUESTS_CA_BUNDLE'] = ca_bundle os.environ['SSL_CERT_FILE'] = ca_bundle

# Or specify explicitly import requests response = requests.get('https://api.example.com/data', verify='/path/to/combined-ca-bundle.pem') ```

  1. 1.Download and install the server certificate chain:
  2. 2.```bash
  3. 3.# Download the certificate chain
  4. 4.echo | openssl s_client -connect api.example.com:443 -showcerts 2>/dev/null | \
  5. 5.sed -n '/BEGIN CERTIFICATE/,/END CERTIFICATE/p' > server-chain.pem

# Use in requests import requests response = requests.get('https://api.example.com/data', verify='server-chain.pem') ```

  1. 1.Disable verification for development only (NEVER in production):
  2. 2.```python
  3. 3.import urllib3
  4. 4.urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

import requests response = requests.get('https://api.example.com/data', verify=False) # Warning: InsecureRequestWarning: Unverified HTTPS request ```

Prevention

  • Keep certifi updated: add certifi>=2024.0.0 to requirements
  • Set REQUESTS_CA_BUNDLE environment variable in deployment configs
  • Use pip install --trusted-host only for package indexes, not API calls
  • For corporate environments, distribute a CA bundle via configuration management
  • Test SSL connections in CI with the same CA bundle used in production