O'Reilly logo

Python Web Scraping Cookbook by Michael Heydt

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

How to do it

  1. The following code (found in 03/store_in_mysql.py) will read the planets data and write it to MySQL:
import mysql.connectorimport get_planet_datafrom mysql.connector import errorcodefrom get_planet_data import get_planet_datatry:    # open the database connection    cnx = mysql.connector.connect(user='root', password='mypassword',                                  host="127.0.0.1", database="scraping")    insert_sql = ("INSERT INTO Planets (Name, Mass, Radius, Description) " +                  "VALUES (%(Name)s, %(Mass)s, %(Radius)s, %(Description)s)")    # get the planet data    planet_data = get_planet_data()    # loop through all planets executing INSERT for each with the cursor    cursor = cnx.cursor()    for planet in planet_data:        print("Storing data for %s" % (planet["Name"])) cursor.execute(insert_sql, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required