Getting ready

To perform the following example, you will need an AWS account and have access to secret keys for use in your Python code. They will be unique to your account.  We will use the boto3 library for S3 access. You can install this using pip install boto3.  Also, you will need to have environment variables set to authenticate.  These will look like the following:

AWS_ACCESS_KEY_ID=AKIAIDCQ5PH3UMWKZEWA AWS_SECRET_ACCESS_KEY=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

These are available in the AWS portal under IAM (Identity Access Management) portion of the portal.

It's a good practice to put these keys in environment variables.  Having them in code can lead to their theft.  During the writing of this book, I had this hard coded and ...

Get Python Web Scraping Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.