You are previewing Head First Python.

Head First Python

Cover of Head First Python by Paul Barry Published by O'Reilly Media, Inc.
  1. Head First Python
  2. Dedication
  3. SPECIAL OFFER: Upgrade this ebook with O’Reilly
  4. A Note Regarding Supplemental Files
  5. Advance Praise for Head First Python
  6. Praise for other Head First books
  7. Author of Head First Python
  8. How to use This Book: Intro
    1. Who is this book for?
      1. Who should probably back away from this book?
    2. We know what you’re thinking
    3. We know what your brain is thinking
    4. Metacognition: thinking about thinking
    5. Here’s what WE did
    6. Here’s what YOU can do to bend your brain into submission
    7. Read Me
    8. The technical review team
    9. Acknowledgments
    10. Safari® Books Online
  9. 1. Meet Python: Everyone loves lists
    1. What’s to like about Python?
    2. Install Python 3
    3. Use IDLE to help learn Python
    4. Work effectively with IDLE
      1. TAB completion
      2. Recall code statements
      3. Edit recalled code
      4. Adjust IDLE’s preferences
    5. Deal with complex data
    6. Create simple Python lists
    7. Lists are like arrays
      1. Access list data using the square bracket notation
    8. Add more data to your list
    9. Work with your list data
      1. It’s time to iterate
    10. For loops work with lists of any size
    11. Store lists within lists
    12. Check a list for a list
    13. Complex data is hard to process
    14. Handle many levels of nested lists
    15. Don’t repeat code; create a function
    16. Create a function in Python
      1. What does your function need to do?
    17. Recursion to the rescue!
      1. What a great start!
    18. Your Python Toolbox
  10. 2. Sharing your Code: Modules of functions
    1. It’s too good not to share
    2. Turn your function into a module
    3. Modules are everywhere
    4. Comment your code
    5. Prepare your distribution
    6. Build your distribution
    7. A quick review of your distribution
    8. Import a module to use it
    9. Python’s modules implement namespaces
    10. Register with the PyPI website
    11. Upload your code to PyPI
    12. Welcome to the PyPI community
    13. With success comes responsibility
      1. Requests for change are inevitable
    14. Life’s full of choices
    15. Control behavior with an extra argument
      1. Take your function to the next level
    16. Before your write new code, think BIF
      1. The range() BIF iterates a fixed number of times
    17. Python tries its best to run your code
    18. Trace your code
    19. Work out what’s wrong
    20. Update PyPI with your new code
    21. You’ve changed your API
    22. Use optional arguments
    23. Your module supports both APIs
    24. Your API is still not right
    25. Your module’s reputation is restored
      1. Your Python skills are starting to build
    26. Your Python Toolbox
  11. 3. Files and Exceptions: Dealing with errors
    1. Data is external to your program
    2. It’s all lines of text
    3. Take a closer look at the data
    4. Know your data
    5. Know your methods and ask for help
    6. Know your data (better)
      1. The case of the missing colon
    7. Two very different approaches
    8. Add extra logic
    9. Handle exceptions
    10. Try first, then recover
      1. The try/except mechanism
    11. Identify the code to protect
    12. Take a pass on the error
    13. What about other errors?
      1. Handling missing files
    14. Add more error-checking code...
    15. ...Or add another level of exception handling
    16. So, which approach is best?
      1. Complexity is rarely a good thing
    17. You’re done...except for one small thing
    18. Be specific with your exceptions
    19. Your Python Toolbox
  12. 4. Persistence: Saving data to files
    1. Programs produce data
    2. Open your file in write mode
    3. Files are left open after an exception!
    4. Extend try with finally
    5. Knowing the type of error is not enough
    6. Use with to work with files
    7. Default formats are unsuitable for files
    8. Why not modify print_lol()?
    9. Pickle your data
    10. Save with dump and restore with load
      1. What if something goes wrong?
    11. Generic file I/O with pickle is the way to go!
    12. Your Python Toolbox
  13. 5. Comprehending data: Work that data!
    1. Coach Kelly needs your help
    2. Sort in one of two ways
    3. The trouble with time
    4. Comprehending lists
      1. The beauty of list comprehensions
    5. Iterate to remove duplicates
    6. Remove duplicates with sets
    7. Your Python Toolbox
  14. 6. Custom data Objects: Bundling code with data
    1. Coach Kelly is back (with a new file format)
    2. Use a dictionary to associate data
    3. Bundle your code and its data in a class
    4. Define a class
    5. Use class to define classes
      1. Creating object instances
    6. The importance of self
    7. Every method’s first argument is self
    8. Inherit from Python’s built-in list
    9. Coach Kelly is impressed
    10. Your Python Toolbox
  15. 7. Web Development: Putting it all together
    1. It’s good to share
      1. You’re a victim of your own success
    2. You can put your program on the Web
      1. A “webapp” is what you want
    3. What does your webapp need to do?
    4. Design your webapp with MVC
    5. Model your data
    6. View your interface
      1. YATE: Yet Another Template Engine
    7. Control your code
    8. CGI lets your web server run programs
    9. Display the list of athletes
    10. The dreaded 404 error!
    11. Create another CGI script
      1. But how do you know which athlete is selected?
    12. Enable CGI tracking to help with errors
    13. A small change can make all the difference
      1. It’s a small change, but it’s an important one
    14. Your webapp’s a hit!
    15. Your Python Toolbox
  16. 8. Mobile app Development: Small devices
    1. The world is getting smaller
      1. There’s more than just desktop computers out there
    2. Coach Kelly is on Android
      1. Run Python on the coach’s smartphone
    3. Don’t worry about Python 2
    4. Set up your development environment
      1. Download the Software Development Kit (SDK)
    5. Configure the SDK and emulator
      1. Add an Android platform
      2. Create a new Android Virtual Device (AVD)
    6. Install and configure Android Scripting
    7. Add Python to your SL4A installation
    8. Test Python on Android
      1. Take your Android emulator for a spin
    9. Define your app’s requirements
    10. The SL4A Android API
    11. Select from a list on Android
    12. The athlete’s data CGI script
      1. The complete Android app, so far
    13. The data appears to have changed type
    14. JSON can’t handle your custom datatypes
    15. Run your app on a real phone
      1. Step 1: Prepare your computer
      2. Step 2: Install AndFTP on your Android phone
    16. Configure AndFTP
    17. The coach is thrilled with his app
      1. Welcome to the future!
    18. Your Python Toolbox
  17. 9. Manage Your data: Handling input
    1. Your athlete times app has gone national
    2. Use a form or dialog to accept input
    3. Create an HTML form template
    4. The data is delivered to your CGI script
    5. Ask for input on your Android phone
    6. It’s time to update your server data
    7. Avoid race conditions
    8. You need a better data storage mechanism
    9. Use a database management system
    10. Python includes SQLite
    11. Exploit Python’s database API
    12. The database API as Python code
    13. A little database design goes a long way
    14. Define your database schema
    15. What does the data look like?
    16. Transfer the data from your pickle to SQLite
    17. What ID is assigned to which athlete?
    18. Insert your timing data
    19. SQLite data management tools
    20. Integrate SQLite with your existing webapp
    21. You still need the list of names
    22. Get an athlete’s details based on ID
    23. You need to amend your Android app, too
    24. Update your SQLite-based athlete data
    25. The NUAC is over the moon!
    26. Your Python Toolbox
  18. 10. Scaling your Webapp: Getting real
    1. There are whale sightings everywhere
    2. The HFWWG needs to automate
    3. Build your webapp with Google App Engine
    4. Download and install App Engine
      1. GAE uses Python 2.5
    5. Make sure App Engine is working
    6. App Engine uses the MVC pattern
    7. Model your data with App Engine
    8. What good is a model without a view?
      1. App Engine templates in an instant
    9. Use templates in App Engine
    10. Django’s form validation framework
    11. Check your form
    12. Controlling your App Engine webapp
      1. Improve the look of your form
    13. Restrict input by providing options
    14. Meet the “blank screen of death”
    15. Process the POST within your webapp
      1. App Engine handles requests as well as responses
    16. Put your data in the datastore
    17. Don’t break the “robustness principle”
    18. Accept almost any date and time
      1. There is a third option
      2. Use “db.StringProperty()” for dates and times
    19. It looks like you’re not quite done yet
    20. Sometimes, the tiniest change can make all the difference...
    21. Capture your user’s Google ID, too
    22. Deploy your webapp to Google’s cloud
    23. Your HFWWG webapp is deployed!
    24. Your Python Toolbox
  19. 11. Dealing with Complexity: Data wrangling
    1. What’s a good time goal for the next race?
    2. So... what’s the problem?
    3. Start with the data
      1. Take another look at the data
    4. Store each time as a dictionary
    5. Dissect the prediction code
    6. Get input from your user
      1. Use input() for input
    7. Getting input raises an issue...
    8. If it’s not in the dictionary, it can’t be found
    9. Search for the closest match
    10. The trouble is with time
    11. The time-to-seconds-to-time module
    12. The trouble is still with time...
    13. Port to Android
    14. Your Android app is a bunch of dialogs
      1. Get your Android app code ready
    15. Put your app together...
    16. Your app’s a wrap!
      1. And there’s no stopping you!
    17. Your Python Toolbox
    18. It’s time to go...
      1. This is just the beginning
  20. A. Leftovers: The Top Ten Things (we didn’t cover)
    1. #1: Using a “professional” IDE
    2. #2: Coping with scoping
    3. #3: Testing
    4. #4: Advanced language features
    5. #5: Regular expressions
    6. #6: More on web frameworks
    7. #7: Object relational mappers and NoSQL
      1. There’s NoSQL, too
    8. #8: Programming GUIs
    9. #9: Stuff to avoid
    10. #10: Other books
  21. Index
  22. About the Author
  23. SPECIAL OFFER: Upgrade this ebook with O’Reilly
  24. Copyright
O'Reilly logo

Chapter 4. Persistence: Saving data to files

image with no caption

It is truly great to be able to process your file-based data. But what happens to your data when you’re done? Of course, it’s best to save your data to a disk file, which allows you to use it again at some later date and time. Taking your memory-based data and storing it to disk is what persistence is all about. Python supports all the usual tools for writing to files and also provides some cool facilities for efficiently storing Python data. So...flip the page and let’s get started learning them.

Programs produce data

It’s a rare program that reads data from a disk file, processes the data, and then throws away the processed data. Typically, programs save the data they process, display their output on screen, or transfer data over a network.

image with no caption

Before you learn what’s involved in writing data to disk, let’s process the data from the previous chapter to work out who said what to whom.

When that’s done, you’ll have something worth saving.

Open your file in write mode

When you use the open() BIF to work with a disk file, you can specify an access mode to use. By default, open() uses mode r for reading, so you don’t need to specify it. To open a file for writing, use mode w:

image with no caption

By default, the print() BIF uses standard output (usually the screen) when displaying data. To write data to a file instead, use the file argument to specify the data file object to use:

image with no caption

When you’re done, be sure to close the file to ensure all of your data is written to disk. This is known as flushing and is very important:

image with no caption

Geek Bits

When you use access mode w, Python opens your named file for writing. If the file already exists, it is cleared of its contents, or clobbered. To append to a file, use access mode a, and to open a file for writing and reading (without clobbering), use w+. If you try to open a file for writing that does not already exist, it is first created for you, and then opened for writing.

Brain Power

Consider the following carefully: what happens to your data files if the second call to print() in your code causes an IOError?

Files are left open after an exception!

When all you ever do is read data from files, getting an IOError is annoying, but rarely dangerous, because your data is still in your file, even though you might be having trouble getting at it.

It’s a different story when writing data to files: if you need to handle an IOError before a file is closed, your written data might become corrupted and there’s no way of telling until after it has happened.

image with no caption

Your exception-handling code is doing its job, but you now have a situation where your data could potentially be corrupted, which can’t be good.

What’s needed here is something that lets you run some code regardless of whether an IOError has occured. In the context of your code, you’ll want to make sure the files are closed no matter what.

Extend try with finally

When you have a situation where code must always run no matter what errors occur, add that code to your try statement’s finally suite:

image with no caption

If no runtime errors occur, any code in the finally suite executes. Equally, if an IOError occurs, the except suite executes and then the finally suite runs.

No matter what, the code in the finally suite always runs.

By moving your file closing code into your finally suite, you are reducing the possibility of data corruption errors.

This is a big improvement, because you’re now ensuring that files are closed properly (even when write errors occur).

But what about those errors?

How do you find out the specifics of the error?

Knowing the type of error is not enough

When a file I/O error occurs, your code displays a generic “File Error” message. This is too generic. How do you know what actually happened?

image with no caption
image with no caption
image with no caption

Who knows?

It turns out that the Python interpreter knows...and it will give up the details if only you’d ask.

When an error occurs at runtime, Python raises an exception of the specific type (such as IOError, ValueError, and so on). Additionally, Python creates an exception object that is passed as an argument to your except suite.

Let’s use IDLE to see how this works.

Of course, all this extra logic is starting to obscure the real meaning of your code.

image with no caption

Use with to work with files

Because the use of the try/except/finally pattern is so common when it comes to working with files, Python includes a statement that abstracts away some of the details. The with statement, when used with files, can dramatically reduce the amount of code you have to write, because it negates the need to include a finally suite to handle the closing of a potentially opened data file. Take a look:

image with no caption

When you use with, you no longer have to worry about closing any opened files, as the Python interpreter automatically takes care of this for you. The with code on the the right is identical in function to that on the left. At Head First Labs, we know which approach we prefer.

Geek Bits

The with statement takes advantage of a Python technology called the context management protocol.

Default formats are unsuitable for files

Although your data is now stored in a file, it’s not really in a useful format. Let’s experiment in the IDLE shell to see what impact this can have.

Yikes! It would appear your list is converted to a large string by print() when it is saved. Your experimental code reads a single line of data from the file and gets all of the data as one large chunk of much for your code saving your list data.

What are your options for dealing with this problem?

Geek Bits

By default, print() displays your data in a format that mimics how your list data is actually stored by the Python interpreter. The resulting output is not really meant to be processed further... its primary purpose is to show you, the Python programmer, what your list data “looks like” in memory.

image with no caption
image with no caption

Parsing the data in the file is a possibility...although it’s complicated by all those square brackets, quotes, and commas. Writing the required code is doable, but it is a lot of code just to read back in your saved data.

Of course, if the data is in a more easily parseable format, the task would likely be easier, so maybe the second option is worth considering, too?

Brain Power

Can you think of a function you created from earlier in this book that might help here?

Why not modify print_lol()?

Recall your print_lol() function from Chapter 2, which takes any list (or list of lists) and displays it on screen, one line at a time. And nested lists can be indented, if necessary.

This functionality sounds perfect! Here’s your code from the module (last seen at the end of Chapter 2):

image with no caption

Amending this code to print to a disk file instead of the screen (known as standard output) should be relatively straightforward. You can then save your data in a more usable format.

The Scholar’s Corner

Standard Output The default place where your code writes its data when the “print()” BIF is used. This is typically the screen. In Python, standard output is referred to as “sys.stdout” and is importable from the Standard Library’s “sys” module.

image with no caption

That’s a good point.

This problem is not unlike the problem from the beginning of the chapter, in that you’ve got lines of text in a disk file that you need to process, only now you have two files instead of one.

You know how to write the code to process your new files, but writing custom code like this is specific to the format that you’ve created for this problem. This is brittle: if the data format changes, your custom code will have to change, too.

Ask yourself: is it worth it?

Pickle your data

Python ships with a standard library called pickle, which can save and load almost any Python data object, including lists.

Once you pickle your data to a file, it is persistent and ready to be read into another program at some later date/time:

image with no caption

You can, for example, store your pickled data on disk, put it in a database, or transfer it over a network to another computer.

When you are ready, reversing this process unpickles your persistent pickled data and recreates your data in its original form within Python’s memory:

image with no caption

Save with dump and restore with load

Using pickle is straightforward: import the required module, then use dump() to save your data and, some time later, load() to restore it. The only requirement when working with pickled files is that they have to be opened in binary access mode:

image with no caption

What if something goes wrong?

If something goes wrong when pickling or unpickling your data, the pickle module raises an exception of type PickleError.

Generic file I/O with pickle is the way to go!

image with no caption

Python takes care of your file I/O details, so you can concentrate on what your code actually does or needs to do.

As you’ve seen, being able to work with, save, and restore data in lists is a breeze, thanks to Python. But what other data structures does Python support out of the box?

Let’s dive into Chapter 5 to find out.

Your Python Toolbox

You’ve got Chapter 4 under your belt and you’ve added some key Python techiques to your toolbox.

Python Lingo

  • “Immutable types” - data types in Python that, once assigned a value, cannot have that value changed.

  • “Pickling” - the process of saving a data object to persistence storage.

  • “Unpickling” - the process of restoring a saved data object from persistence storage.

The best content for your career. Discover unlimited learning on demand for around $1/day.