You are previewing Programming Google App Engine.

Programming Google App Engine

Cover of Programming Google App Engine by Dan Sanderson Published by O'Reilly Media, Inc.
  1. Programming Google App Engine
    1. SPECIAL OFFER: Upgrade this ebook with O’Reilly
    2. Preface
      1. Using This Book
      2. Conventions Used in This Book
      3. Using Code Samples
      4. Safari® Books Online
      5. How to Contact Us
      6. Acknowledgments
    3. 1. Introducing Google App Engine
      1. The Runtime Environment
      2. The Static File Servers
      3. The Datastore
      4. The Services
      5. Google Accounts
      6. Task Queues and Cron Jobs
      7. Developer Tools
      8. The Administration Console
      9. Things App Engine Doesn’t Do...Yet
      10. Getting Started
    4. 2. Creating an Application
      1. Setting Up the SDK
      2. Developing the Application
      3. Registering the Application
      4. Uploading the Application
      5. Introducing the Administration Console
    5. 3. Handling Web Requests
      1. The App Engine Architecture
      2. Configuring the Frontend
      3. How the App Is Run
      4. Quotas and Limits
    6. 4. Datastore Entities
      1. Entities, Keys, and Properties
      2. Introducing the Python Datastore API
      3. Introducing the Java Datastore API
      4. Property Values
      5. Keys and Key Objects
      6. Using Entities
    7. 5. Datastore Queries
      1. Queries and Kinds
      2. Query Results and Keys
      3. GQL
      4. The Python Query API
      5. The Java Query API
      6. Introducing Indexes
      7. Automatic Indexes and Simple Queries
      8. Custom Indexes and Complex Queries
      9. Not-Equal and IN Filters
      10. Unset and Nonindexed Properties
      11. Sort Orders and Value Types
      12. Queries and Multivalued Properties
      13. Configuring Indexes
    8. 6. Datastore Transactions
      1. Entities and Entity Groups
      2. What Can Happen in a Transaction
      3. Transactions in Python
      4. Transactions in Java
      5. How Entities Are Updated
      6. How Entities Are Read
      7. Batch Updates
      8. How Indexes Are Updated
    9. 7. Data Modeling with Python
      1. Models and Properties
      2. Property Declarations
      3. Modeling Relationships
      4. Model Inheritance
      5. Queries and PolyModels
      6. Creating Your Own Property Classes
    10. 8. The Java Persistence API
      1. Setting Up JPA
      2. Entities and Keys
      3. Entity Properties
      4. Embedded Objects
      5. Saving, Fetching, and Deleting Objects
      6. Transactions in JPA
      7. Queries and JPQL
      8. Relationships
      9. For More Information
    11. 9. The Memory Cache
      1. The Python Memcache API
      2. The Java Memcache API
    12. 10. Fetching URLs and Web Resources
      1. Fetching URLs in Python
      2. Fetching URLs in Java
      3. Asynchronous Requests in Python
    13. 11. Sending and Receiving Mail and Instant Messages
      1. Enabling Inbound Services
      2. Sending Email Messages
      3. Receiving Email Messages
      4. Sending XMPP Messages
      5. Receiving XMPP Messages
    14. 12. Bulk Data Operations and Remote Access
      1. Setting Up the Remote API for Python
      2. Setting Up the Remote API for Java
      3. Using the Bulk Loader Tool
      4. Using the Remote Shell Tool
      5. Using the Remote API from a Script
    15. 13. Task Queues and Scheduled Tasks
      1. Task Queues
      2. Scheduled Tasks
    16. 14. The Django Web Application Framework
      1. Installing Django
      2. Creating a Django Project
      3. The Request Handler Script
      4. The Django App Engine Helper
      5. Creating a Django Application
      6. Using App Engine Models With Django
      7. Using Django Unit Tests and Fixtures
      8. Using Django Forms
    17. 15. Deploying and Managing Applications
      1. Uploading an Application
      2. Using Versions
      3. Managing Service Configuration
      4. Managing Indexes
      5. Browsing and Downloading Logs
      6. Inspecting the Datastore
      7. Application Settings
      8. Managing Developers
      9. Quotas and Billing
      10. Getting Help
    18. Index
    19. About the Author
    20. Colophon
    21. SPECIAL OFFER: Upgrade this ebook with O’Reilly
O'Reilly logo

Introducing the Python Datastore API

In the Python API for the App Engine datastore, Python objects represent datastore entities. The class of the object corresponds to the entity’s kind, where the name of the class is the name of the kind. You define kinds by creating classes that extend one of the provided base classes.

Each attribute of the object corresponds with a property of the entity. To create a new entity in the datastore, you call the class constructor, set attributes on the object, then call a method to save it. To update an existing entity, you call a method that returns the object for the entity (such as via a query), modify its attributes, then save it.

Example 4-1 defines a class named Book to represent entities of the kind Book. It creates an object of this class by calling the class constructor, then sets several property values. Finally, it calls the put() method to save the new entity to the datastore. The entity does not exist in the datastore until it is put() for the first time.

Example 4-1. Python code to create an entity of the kind Book

from google.appengine.ext import db
import datetime

class Book(db.Expando):

obj = Book()
obj.title = 'The Grapes of Wrath' = 'John Steinbeck'
obj.copyright_year = 1939
obj.author_birthdate =, 2, 27)


The Book class inherits from the class Expando in App Engine’s db package. The Expando base class says Book objects can have any of their properties assigned any value. The entity “expands” to accommodate new properties as they are assigned to attributes of the object. Python does not require that an object’s member variables be declared in a class definition, and this example takes advantage of this using an empty class definition—the pass keyword indicates the empty definition—and assigns values to attributes of the object after it is created. The Expando base class knows to use the object’s attributes as the values of the corresponding entity’s properties.

The Expando class has a funny name because this isn’t the way the API’s designers expect us to create new classes in most cases. Instead, you’re more likely to use the Model base class with a class definition that ensures each instance conforms to a structure, so a mistake in the code doesn’t accidentally create entities with malformed properties. Here is how we might implement the Book class using Model:

class Book(db.Model):
    title = db.StringProperty()
    author = db.StringProperty()
    copyright_year = db.IntegerProperty()
    author_birthdate = db.DateProperty()

The Model version of Book specifies a structure for Book objects that is enforced while the object is being manipulated. It ensures that values assigned to an object’s properties are of appropriate types, such as string values for title and author properties, and raises a runtime error if the app attempts to assign a value of the wrong type to a property. With Model as the base class, the object does not “expand” to accommodate other entities: an attempt to assign a value to a property not mentioned in the class definition raises a runtime error. Model and the various Property definitions also provide other features for managing the structure of your data, such as automatic values, required values, and the ability to add your own validation and serialization logic.

It’s important to notice that these validation features are provided by the Model class and your application code, not the datastore. Even if part of your app uses a Model class to ensure a property’s value meets certain conditions, another part of your app can still retrieve the entity without using the class and do whatever it likes to that value. The bad value won’t raise an error until the app tries to load the changed entity into a new instance of the Model class. This is both a feature and a burden: your app can manage entities flexibly and enforce structure where needed, but it must also be careful when those structures need to change. Data modeling and the Model class are discussed in detail in Chapter 7.

The Book constructor accepts initial values for the object’s properties as keyword arguments. The constructor code earlier could also be written like this:

obj = Book(title='The Grapes of Wrath',
           author='John Steinbeck',
 , 2, 27))

As written, this code does not set a key name for the new entity. Without a key name, the datastore generates a unique ID when the object is saved for the first time. If you prefer to use a key name generated by the app, you call the constructor with the key_name parameter:

obj = Book(key_name='0143039431',
           title='The Grapes of Wrath',
           author='John Steinbeck',
 , 2, 27))


Because the Python API uses keyword arguments, object attributes, and object methods for purposes besides entity properties, there are several property names that are off-limits. For instance, you cannot use the Python API to set a property named key_name, because this could get confused with the key_name parameter for the object constructor. Names reserved by the Python API are enforced in the API, but not in the datastore itself. Google’s official documentation lists the reserved property names.

The datastore reserves all property names beginning and ending with two underscores (such as __internal__). This is true for the Python API and the Java API, and will be true for future APIs as well.

The Python API ignores all object attributes whose names begin with a single underscore (such as _counter). You can use such attributes to attach data and functionality to an object that should not be saved as properties for the entity.

The complete key of an entity, including the key name and kind, must be unique. (We’ll discuss another part to keys that contributes to a key’s uniqueness, called ancestors, in Chapter 6.) If you build a new object with a key that is already in use, then try to save it, the save will replace the existing object. For when you don’t want to overwrite existing data, the datastore API provides an alternate way to create an object. The get_or_insert() class method takes a key name and either returns an existing entity with that key name, or creates a new entity with that key name and no properties and returns it. Either way, the method is guaranteed to return an object that represents an entity in the datastore:

obj = Book.get_or_insert('0143039431')

if obj.title:
    # Book already exists.
    # ...
    obj.title = 'The Grapes of Wrath' = 'John Steinbeck'
    obj.copyright_year = 1939
    obj.author_birthdate =, 2, 27)


The best content for your career. Discover unlimited learning on demand for around $1/day.