Using the urllib2 Request class

The urlopen function of urllib2 can also take a Request object as a parameter, instead of the URL and the data to send. The Request class defines objects that encapsulate all the information related to a request. Through this object, we can make more complex requests, adding our own headers, such as the User-Agent.

The simplest constructor for the Request object only takes one string as an argument, indicating the URL to connect to, so using this object as a parameter of urlopen would be equivalent to using a string with the URL directly.

However, the Request constructor also has as optional parameters a data string for sending data by POST and a dictionary of headers.

Get Mastering Python for Networking and Security now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.