How to do it - reading and processing messages

To process the messages, run the 03/process_messages.py program:

import boto3import botocoreimport requestsfrom bs4 import BeautifulSoupprint("Starting")# declare our keys (normally, don't hard code this)access_key = "AKIAIXFTCYO7FEL55TCQ"access_secret_key = "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"# create sqs clientsqs = boto3.client('sqs', "us-west-2",           aws_access_key_id = access_key,           aws_secret_access_key = access_secret_key)print("Created client")# create / open the SQS queuequeue = sqs.create_queue(QueueName="PlanetMoreInfo")queue_url = queue["QueueUrl"]print ("Opened queue: %s" % queue_url)while True:  print ("Attempting to receive messages") response = sqs.receive_message(QueueUrl=queue_url, ...

Get Python Web Scraping Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.