You are previewing Programming Google App Engine.

Programming Google App Engine

Cover of Programming Google App Engine by Dan Sanderson Published by O'Reilly Media, Inc.
  1. Programming Google App Engine
    1. SPECIAL OFFER: Upgrade this ebook with O’Reilly
    2. Preface
      1. Using This Book
      2. Conventions Used in This Book
      3. Using Code Samples
      4. Safari® Books Online
      5. How to Contact Us
      6. Acknowledgments
    3. 1. Introducing Google App Engine
      1. The Runtime Environment
      2. The Static File Servers
      3. The Datastore
      4. The Services
      5. Google Accounts
      6. Task Queues and Cron Jobs
      7. Developer Tools
      8. The Administration Console
      9. Things App Engine Doesn’t Do...Yet
      10. Getting Started
    4. 2. Creating an Application
      1. Setting Up the SDK
      2. Developing the Application
      3. Registering the Application
      4. Uploading the Application
      5. Introducing the Administration Console
    5. 3. Handling Web Requests
      1. The App Engine Architecture
      2. Configuring the Frontend
      3. How the App Is Run
      4. Quotas and Limits
    6. 4. Datastore Entities
      1. Entities, Keys, and Properties
      2. Introducing the Python Datastore API
      3. Introducing the Java Datastore API
      4. Property Values
      5. Keys and Key Objects
      6. Using Entities
    7. 5. Datastore Queries
      1. Queries and Kinds
      2. Query Results and Keys
      3. GQL
      4. The Python Query API
      5. The Java Query API
      6. Introducing Indexes
      7. Automatic Indexes and Simple Queries
      8. Custom Indexes and Complex Queries
      9. Not-Equal and IN Filters
      10. Unset and Nonindexed Properties
      11. Sort Orders and Value Types
      12. Queries and Multivalued Properties
      13. Configuring Indexes
    8. 6. Datastore Transactions
      1. Entities and Entity Groups
      2. What Can Happen in a Transaction
      3. Transactions in Python
      4. Transactions in Java
      5. How Entities Are Updated
      6. How Entities Are Read
      7. Batch Updates
      8. How Indexes Are Updated
    9. 7. Data Modeling with Python
      1. Models and Properties
      2. Property Declarations
      3. Modeling Relationships
      4. Model Inheritance
      5. Queries and PolyModels
      6. Creating Your Own Property Classes
    10. 8. The Java Persistence API
      1. Setting Up JPA
      2. Entities and Keys
      3. Entity Properties
      4. Embedded Objects
      5. Saving, Fetching, and Deleting Objects
      6. Transactions in JPA
      7. Queries and JPQL
      8. Relationships
      9. For More Information
    11. 9. The Memory Cache
      1. The Python Memcache API
      2. The Java Memcache API
    12. 10. Fetching URLs and Web Resources
      1. Fetching URLs in Python
      2. Fetching URLs in Java
      3. Asynchronous Requests in Python
    13. 11. Sending and Receiving Mail and Instant Messages
      1. Enabling Inbound Services
      2. Sending Email Messages
      3. Receiving Email Messages
      4. Sending XMPP Messages
      5. Receiving XMPP Messages
    14. 12. Bulk Data Operations and Remote Access
      1. Setting Up the Remote API for Python
      2. Setting Up the Remote API for Java
      3. Using the Bulk Loader Tool
      4. Using the Remote Shell Tool
      5. Using the Remote API from a Script
    15. 13. Task Queues and Scheduled Tasks
      1. Task Queues
      2. Scheduled Tasks
    16. 14. The Django Web Application Framework
      1. Installing Django
      2. Creating a Django Project
      3. The Request Handler Script
      4. The Django App Engine Helper
      5. Creating a Django Application
      6. Using App Engine Models With Django
      7. Using Django Unit Tests and Fixtures
      8. Using Django Forms
    17. 15. Deploying and Managing Applications
      1. Uploading an Application
      2. Using Versions
      3. Managing Service Configuration
      4. Managing Indexes
      5. Browsing and Downloading Logs
      6. Inspecting the Datastore
      7. Application Settings
      8. Managing Developers
      9. Quotas and Billing
      10. Getting Help
    18. Index
    19. About the Author
    20. Colophon
    21. SPECIAL OFFER: Upgrade this ebook with O’Reilly

Chapter 4. Datastore Entities

Most scalable web applications use separate systems for handling web requests and for storing data. The request handling system routes each request to one of many machines, each of which handles the request without knowledge of other requests going to other machines. Each request handler behaves as if it is stateless, acting solely on the content of the request to produce the response. But most web applications need to maintain state, whether it’s remembering that a customer ordered a product, or just remembering that the user who made the current request is the same user who made an earlier request handled by another machine. For this, request handlers must interact with a central database to fetch and update the latest information about the state of the application.

Just as the request handling system distributes web requests across many machines for scaling and robustness, so does the database. But unlike the request handlers, databases are by definition stateful, and this poses a variety of questions. Which machine remembers which piece of data? How does the system route a data query to the machine—or machines—that can answer the query? When a client updates data, how long does it take for all machines that know that data to get the latest version, and what does the system return for queries about that data in the meantime? What happens when two clients try to update the same data at the same time? What happens when a machine goes down?

As with request handling, Google App Engine manages the scaling and maintenance of data storage automatically. Your application interacts with an abstract model that hides the details of managing and growing a pool of data servers. This model and the service behind it provide answers to the questions of scalable data storage specifically designed for web applications.

App Engine’s abstraction for data is easy to understand, but it is not obvious how to best take advantage of its features. In particular, it is surprisingly different from the kind of database with which most of us are most familiar, the relational database. It’s different enough, in fact, that Google doesn’t call it a “database,” but a “datastore.”

We will dedicate the next several chapters to this important subject.

Entities, Keys, and Properties

The App Engine datastore is best understood as an object database. An object in the datastore is known as an entity.

An entity has a key that uniquely identifies the object across the entire system. If you have a key, you can fetch the entity for the key quickly. Keys can be stored as data in entities, such as to create a reference from one entity to another. A key has several parts, some of which we’ll discuss here and some of which we’ll cover later.

One part of the key is the application’s ID, which ensures that nothing else about the key can collide with the entities of any other application. It also ensures that no other app can access your app’s data, and that your app cannot access data for other apps. You won’t see the app ID mentioned in the datastore API; this is automatic.

An important part of the key is the kind. An entity’s kind categorizes the entity for the purposes of queries, and for ensuring the uniqueness of the rest of the key. For example, a shopping cart application might represent each customer order with an entity of the kind “Order.” The application specifies the kind when it creates the entity.

The key also contains an entity ID. This can be an arbitrary string specified by the app, or it can be generated automatically by the datastore. The API calls an entity ID given by the app a key name, and an entity ID generated by the datastore an ID. An entity has either a key name or an ID, but not both.

Once an entity has been created, its key cannot be changed. This applies to all parts of its key, including the kind and the key name or ID.

The data for the entity is stored in one or more properties. Each property has a name and at least one value. Each value is of one of several supported data types, such as a string, an integer, a date-time, or a null value. We’ll look at property value types in detail later in this chapter.

A property can have multiple values, and each value can be of a different type. As you will see in Multivalued Properties, multivalued properties have unusual behavior, but are quite useful for modeling some kinds of data, and surprisingly efficient.


It’s tempting to compare these concepts with similar concepts in relational databases: kinds are tables; entities are rows; properties are fields or columns. That’s a useful comparison, but watch out for differences.

Unlike a table in a relational database, there is no relationship between an entity’s kind and its properties. Two entities of the same kind can have different properties set or not set, and can each have a property of the same name but with values of different types. You can (and often will) enforce a data schema in your own code, and App Engine includes libraries to make this easy, but this is not required by the datastore.

Also unlike relational databases, keys are not properties. You can perform queries on key names just like properties, but you cannot change a key name after the entity has been created.

And of course, a relational database cannot store multiple values in a single cell, while an App Engine property can have multiple values.

The best content for your career. Discover unlimited learning on demand for around $1/day.