Proxy Auth in Python

Some times Our crawler programs needs to satify Proxy Authentication, Generelly we get this situation when we work under Proxy Server. So we can authenticate the Proxy Server through Program.

In Python we can handle the proxy with urllib2 package, so if we look deep into the program urllib2.ProxyHandler and urllib2.HTTPBasicAuthHandler functions helps for authentication.

Code:

import urllib2
from lxml import html

proxy_handler = urllib2.ProxyHandler({’http’: ‘http://www.ragalahari.com/wallpapers.asp’})
proxy_auth_handler = urllib2.HTTPBasicAuthHandler()

Now we need to append our User Name,Password and Proxy Name to the realm.

Code:

proxy_auth_handler.add_password(’realm’, ‘Proxy Name’, ‘User Name’, ‘Password’)

opener = urllib2.build_opener(proxy_handler, proxy_auth_handler)

opener.addheaders = [('User-agent', 'Mozilla/5.0')]