How to do it

We proceed with the recipe as follows:

  1. The following code will read the planets data and write it to the database (code in 03/save_in_postgres.py):
import psycopg2from get_planet_data import get_planet_datatry:  # connect to PostgreSQL  conn = psycopg2.connect("dbname='scraping' host='localhost' user='postgres' password='mypassword'")  # the SQL INSERT statement we will use  insert_sql = ('INSERT INTO public."Planets"(name, mass, radius, description, moreinfo) ' +          'VALUES (%(Name)s, %(Mass)s, %(Radius)s, %(Description)s, %(MoreInfo)s);')  # open a cursor to access data  cur = conn.cursor()  # get the planets data and loop through each  planet_data = get_planet_data()  for planet in planet_data:    # write each record cur.execute(insert_sql, ...

Get Python Web Scraping Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.