Requests debugging

Tested on version: 2.7.*

Especially when starting to communicate with a new API or other remote service, you might want to debug the raw API-calls you are making to this service.

Meaning you want to see what is inside these objects you are creating, the actual raw traffic send and received to and from the remote server. This will give you a good insight on what your are doing, and why it might or might not work as designed.

Actually this is fairly easy with a snippet i found on the internet long time ago and have been using frequently. (sorry, source got lost over time, contact me if it’s yours and you want the credit)

You can drop the following just before you make the actual request, and enjoy the output in the commandline terminal.

# debug stuff starts here
    import requests
    import logging

    logging.basicConfig()
    logging.getLogger().setLevel(logging.DEBUG)
    requests_log = logging.getLogger("requests.packages.urllib3")
    requests_log.setLevel(logging.DEBUG)
    requests_log.propagate = True

    try:
        import http.client as http_client
    except ImportError:
        # Python 2
        import httplib as http_client
    http_client.HTTPConnection.debuglevel = 1
  #end of debug

    _response = requests.patch(_strRequest, headers=_headers, data=_data)

Expected output will be something like:

DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): api.statuspage.io

send: 'PATCH /v1/pages/REDACTED/components/pxb9hvz82zkl.json HTTP/1.1\r\nHost: api.statuspage.io\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: PPRO_status_updater_V1.0\r\nContent-Type: application/x-www-form-urlencoded\r\nAuthorization: OAuth REDACTED\r\nContent-Length: 30\r\n\r\ncomponent[status]=major_outage'
reply: 'HTTP/1.1 301 Moved Permanently\r\n'
header: Content-Type: text/html
header: Date: Tue, 30 Jan 2018 10:56:16 GMT
header: Location: https://api.statuspage.io/v1/pages/REDACTED/components/pxb9hvz82zkl.json
header: Connection: Keep-Alive
header: Content-Length: 0

DEBUG:requests.packages.urllib3.connectionpool:http://api.statuspage.io:80 "PATCH /v1/pages/REDACTED/components/pxb9hvz82zkl.json HTTP/1.1" 301 0
DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): api.statuspage.io

send: 'PATCH /v1/pages/REDACTED/components/pxb9hvz82zkl.json HTTP/1.1\r\nHost: api.statuspage.io\r\nContent-Length: 0\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: PPRO_status_updater_V1.0\r\nAuthorization: OAuth REDACTED\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: Content-Type: application/json
header: Vary: Accept-Encoding, Origin
header: Content-Encoding: gzip
header: Cache-Control: no-cache
header: X-Request-Id: 72f0c2c3-2477-4ed5-816b-393863b061e2
header: X-Runtime: 0.036190
header: Transfer-Encoding: chunked

DEBUG:requests.packages.urllib3.connectionpool:https://api.statuspage.io:443 "PATCH /v1/pages/REDACTED/components/pxb9hvz82zkl.json HTTP/1.1" 403 None