You are previewing Programming Google App Engine.

Programming Google App Engine

Cover of Programming Google App Engine by Dan Sanderson Published by O'Reilly Media, Inc.
  1. Programming Google App Engine
    1. SPECIAL OFFER: Upgrade this ebook with O’Reilly
    2. Preface
      1. Using This Book
      2. Conventions Used in This Book
      3. Using Code Samples
      4. Safari® Books Online
      5. How to Contact Us
      6. Acknowledgments
    3. 1. Introducing Google App Engine
      1. The Runtime Environment
      2. The Static File Servers
      3. The Datastore
      4. The Services
      5. Google Accounts
      6. Task Queues and Cron Jobs
      7. Developer Tools
      8. The Administration Console
      9. Things App Engine Doesn’t Do...Yet
      10. Getting Started
    4. 2. Creating an Application
      1. Setting Up the SDK
      2. Developing the Application
      3. Registering the Application
      4. Uploading the Application
      5. Introducing the Administration Console
    5. 3. Handling Web Requests
      1. The App Engine Architecture
      2. Configuring the Frontend
      3. How the App Is Run
      4. Quotas and Limits
    6. 4. Datastore Entities
      1. Entities, Keys, and Properties
      2. Introducing the Python Datastore API
      3. Introducing the Java Datastore API
      4. Property Values
      5. Keys and Key Objects
      6. Using Entities
    7. 5. Datastore Queries
      1. Queries and Kinds
      2. Query Results and Keys
      3. GQL
      4. The Python Query API
      5. The Java Query API
      6. Introducing Indexes
      7. Automatic Indexes and Simple Queries
      8. Custom Indexes and Complex Queries
      9. Not-Equal and IN Filters
      10. Unset and Nonindexed Properties
      11. Sort Orders and Value Types
      12. Queries and Multivalued Properties
      13. Configuring Indexes
    8. 6. Datastore Transactions
      1. Entities and Entity Groups
      2. What Can Happen in a Transaction
      3. Transactions in Python
      4. Transactions in Java
      5. How Entities Are Updated
      6. How Entities Are Read
      7. Batch Updates
      8. How Indexes Are Updated
    9. 7. Data Modeling with Python
      1. Models and Properties
      2. Property Declarations
      3. Modeling Relationships
      4. Model Inheritance
      5. Queries and PolyModels
      6. Creating Your Own Property Classes
    10. 8. The Java Persistence API
      1. Setting Up JPA
      2. Entities and Keys
      3. Entity Properties
      4. Embedded Objects
      5. Saving, Fetching, and Deleting Objects
      6. Transactions in JPA
      7. Queries and JPQL
      8. Relationships
      9. For More Information
    11. 9. The Memory Cache
      1. The Python Memcache API
      2. The Java Memcache API
    12. 10. Fetching URLs and Web Resources
      1. Fetching URLs in Python
      2. Fetching URLs in Java
      3. Asynchronous Requests in Python
    13. 11. Sending and Receiving Mail and Instant Messages
      1. Enabling Inbound Services
      2. Sending Email Messages
      3. Receiving Email Messages
      4. Sending XMPP Messages
      5. Receiving XMPP Messages
    14. 12. Bulk Data Operations and Remote Access
      1. Setting Up the Remote API for Python
      2. Setting Up the Remote API for Java
      3. Using the Bulk Loader Tool
      4. Using the Remote Shell Tool
      5. Using the Remote API from a Script
    15. 13. Task Queues and Scheduled Tasks
      1. Task Queues
      2. Scheduled Tasks
    16. 14. The Django Web Application Framework
      1. Installing Django
      2. Creating a Django Project
      3. The Request Handler Script
      4. The Django App Engine Helper
      5. Creating a Django Application
      6. Using App Engine Models With Django
      7. Using Django Unit Tests and Fixtures
      8. Using Django Forms
    17. 15. Deploying and Managing Applications
      1. Uploading an Application
      2. Using Versions
      3. Managing Service Configuration
      4. Managing Indexes
      5. Browsing and Downloading Logs
      6. Inspecting the Datastore
      7. Application Settings
      8. Managing Developers
      9. Quotas and Billing
      10. Getting Help
    18. Index
    19. About the Author
    20. Colophon
    21. SPECIAL OFFER: Upgrade this ebook with O’Reilly

Property Values

Each value data type supported by the datastore is represented by a primitive type in the language for the runtime or a class provided by the API. The data types and their language-specific equivalents are listed in Table 4-1. In this table, db is the Python package google.appengine.ext.db, and datastore is the Java package

Table 4-1. Datastore property value types and equivalent language types

Data typePython typeJava type
Unicode text string (up to 500 bytes, indexed)unicode or str (converted to unicode as ASCII)java.lang.String
Long Unicode text string (not indexed)db.Textdatastore.Text
Short byte string (up to 500 bytes, indexed)db.ByteStringdatastore.ShortBlob
Long byte string (not indexed)db.Blobdatastore.Blob
Integer (64-bit)int or long (converted to 64-bit long)byte, short, int, or long (converted to long)
Float (double precision)floatfloat or double (converted to double)
Null valueNonenull
Entity keydb.Keydatastore.Key
A Google accountusers.User...api.users.User
A category (GD)db.Categorydatastore.Category
A URL (GD)db.Linkdatastore.Link
An email address (GD)db.Emaildatastore.Email
A geographical point (GD)db.GeoPtdatastore.GeoPt
An instant messaging handle (GD)db.IMdatastore.IMHandle
A phone number (GD)db.PhoneNumberdatastore.PhoneNumber
A postal address (GD)db.PostalAddressdatastore.PostalAddress
A user rating (GD)db.Ratingdatastore.Rating

The datastore types in this table labeled “(GD)” are types borrowed from the Google Data protocol. These are supported as distinct native data types in the datastore, though most of them are implemented as text strings. Notable exceptions are GeoPt, which is a pair of floating-point values for latitude (–90 to +90) and longitude (–180 to +180), and Rating, which is an integer between 1 and 100.

Example 4-2 demonstrates the use of several of these data types in Python.

Example 4-2. Python code to set property values of various types

from google.appengine.ext import webapp
from google.appengine.ext import db
from google.appengine.api import users
import datetime

class Comment(db.Expando):

class CommentHandler(webapp.RequestHandler):
    def post(self):
        c = Comment()
        c.commenter = users.get_current_user()  # returns a users.User object
        c.message = db.Text(self.request.get('message')) =

        # Display the result page...


When you use Python’s db.Expando or Java’s low-level datastore API, types that are widened to other types when stored come back as the wider datastore types when you retrieve the entity. For instance, a Java Integer comes back as a Long. If you use these APIs in your app, it’s best to use the native datastore types, so the value types stay consistent.

The data modeling interfaces offer a way to store values in these alternate types and convert them back automatically when retrieving the entity. See Chapters 7 and 8.

Strings, Text, and Blobs

The datastore has two distinct data types for storing strings of text: short strings and long strings. Short strings are indexed; that is, they can be the subject of queries, such as a search for every Person entity with a given value for a last_name property. Short string values must be less than 500 bytes in length. Long strings can be longer than 500 bytes, but are not indexed.

Text strings, short and long, are strings of characters from the Unicode character set. Internally, the datastore stores Unicode strings using the UTF-8 encoding, which represents some characters using multiple bytes. This means that the 500-byte limit for short strings is not necessarily the same as 500 Unicode characters. The actual limit on the number of characters depends on which characters are in the string.

The Python API distinguishes between short strings and long strings using Python data types. The Python built-in types unicode and str represent short string values. str values are assumed to be text encoded as ASCII, and are treated as UTF-8 (which is equivalent to ASCII for the first 128 characters in the character set). For long strings, the Python API includes a db.Text class, which takes a unicode or str value as an argument for its constructor.

# Short strings.
e.prop = "a short string, as an ASCII str"
e.prop = unicode("a short string, as a unicode value")

# A long string.
e.prop = db.Text("a long string, can be longer than 500 bytes")

The Java API makes a similar distinction, treating String values as short strings, and using the datastore.Text class to represent long text strings.

The datastore also supports two additional classes for strings of bytes, or “blobs.” Blobs are not assumed to be of any particular format, and their bytes are preserved. This makes them good for nontext data, such as images, movies, or other media. As with text strings, the blob types come in indexed and nonindexed varieties. The Python API provides the db.Blob class to represent blob values, which takes a str value as an argument for its constructor.

# A blob.  self.request.body is the body of the request in a
# webapp request handler, such as an uploaded file.
e.prop = db.Blob(self.request.body)

In Java, the blob types are datastore.ShortBlob and datastore.Blob.

Unset Versus the Null Value

One possible value of a property is the null value. In Python, the null value is represented by the Python built-in value None. In Java, this value is null.

A property with the null value is not the same as an unset property. Consider the following Python code:

class Entity(db.Expando):

a = Entity()
a.prop1 = 'abc'
a.prop2 = None

b = Entity()
b.prop1 = 'def'

This creates two entities of the kind Entity. Both entities have a property named prop1. The first entity has a property named prop2; the second does not.

Of course, an unset property can be set later:

b.prop2 = 123

# b now has a property named "prop2."

Similarly, a set property can be made unset. In the Python API, you delete the property by deleting the attribute from the object, using the del keyword:

del b.prop2

# b no longer has a property named "prop2."

In Java, the low-level datastore API’s Entity class has methods to set properties (setProperty()) and unset properties (removeProperty()).

Multivalued Properties

As we mentioned earlier, a property can have multiple values. We’ll discuss the more substantial aspects of multivalued properties when we talk about queries and data modeling. But for now, it’s worth a brief mention.

A property can have one or more values. A property cannot have zero values; a property without a value is simply unset. Each value for a property can be of a different type, and can be the null value.

The datastore preserves the order of values as they are assigned. The Python API returns the values in the same order as they were set.

In Python, a property with multiple values is represented as a single Python list value:

e.prop = [1, 2, 'a', None, 'b']


Because a property must have at least one value, it is an error to assign an empty list ([] in Python) to a property on an entity whose Python class is based on the Expando class.

class Entity(db.Expando):

e = Entity()
e.prop = []  # ERROR

In contrast, the Model base class includes a feature that automatically translates between the empty list value and “no property set.” You’ll see this feature in Chapter 7.

In the Java low-level datastore API, you can store multiple values for a property using a Collection type. The low-level API returns the values as a java.util.List. The items are stored in the order provided by the Collection type’s iterator. For many types, such as SortedSet or TreeSet, this order is deterministic. For others, such as HashSet, it is not. If the app needs the original data structure, it must convert the List returned by the datastore to the appropriate type.

The best content for your career. Discover unlimited learning on demand for around $1/day.