You are previewing Programming Google App Engine.

Programming Google App Engine

Cover of Programming Google App Engine by Dan Sanderson Published by O'Reilly Media, Inc.
  1. Programming Google App Engine
    1. SPECIAL OFFER: Upgrade this ebook with O’Reilly
    2. Preface
      1. Using This Book
      2. Conventions Used in This Book
      3. Using Code Samples
      4. Safari® Books Online
      5. How to Contact Us
      6. Acknowledgments
    3. 1. Introducing Google App Engine
      1. The Runtime Environment
      2. The Static File Servers
      3. The Datastore
      4. The Services
      5. Google Accounts
      6. Task Queues and Cron Jobs
      7. Developer Tools
      8. The Administration Console
      9. Things App Engine Doesn’t Do...Yet
      10. Getting Started
    4. 2. Creating an Application
      1. Setting Up the SDK
      2. Developing the Application
      3. Registering the Application
      4. Uploading the Application
      5. Introducing the Administration Console
    5. 3. Handling Web Requests
      1. The App Engine Architecture
      2. Configuring the Frontend
      3. How the App Is Run
      4. Quotas and Limits
    6. 4. Datastore Entities
      1. Entities, Keys, and Properties
      2. Introducing the Python Datastore API
      3. Introducing the Java Datastore API
      4. Property Values
      5. Keys and Key Objects
      6. Using Entities
    7. 5. Datastore Queries
      1. Queries and Kinds
      2. Query Results and Keys
      3. GQL
      4. The Python Query API
      5. The Java Query API
      6. Introducing Indexes
      7. Automatic Indexes and Simple Queries
      8. Custom Indexes and Complex Queries
      9. Not-Equal and IN Filters
      10. Unset and Nonindexed Properties
      11. Sort Orders and Value Types
      12. Queries and Multivalued Properties
      13. Configuring Indexes
    8. 6. Datastore Transactions
      1. Entities and Entity Groups
      2. What Can Happen in a Transaction
      3. Transactions in Python
      4. Transactions in Java
      5. How Entities Are Updated
      6. How Entities Are Read
      7. Batch Updates
      8. How Indexes Are Updated
    9. 7. Data Modeling with Python
      1. Models and Properties
      2. Property Declarations
      3. Modeling Relationships
      4. Model Inheritance
      5. Queries and PolyModels
      6. Creating Your Own Property Classes
    10. 8. The Java Persistence API
      1. Setting Up JPA
      2. Entities and Keys
      3. Entity Properties
      4. Embedded Objects
      5. Saving, Fetching, and Deleting Objects
      6. Transactions in JPA
      7. Queries and JPQL
      8. Relationships
      9. For More Information
    11. 9. The Memory Cache
      1. The Python Memcache API
      2. The Java Memcache API
    12. 10. Fetching URLs and Web Resources
      1. Fetching URLs in Python
      2. Fetching URLs in Java
      3. Asynchronous Requests in Python
    13. 11. Sending and Receiving Mail and Instant Messages
      1. Enabling Inbound Services
      2. Sending Email Messages
      3. Receiving Email Messages
      4. Sending XMPP Messages
      5. Receiving XMPP Messages
    14. 12. Bulk Data Operations and Remote Access
      1. Setting Up the Remote API for Python
      2. Setting Up the Remote API for Java
      3. Using the Bulk Loader Tool
      4. Using the Remote Shell Tool
      5. Using the Remote API from a Script
    15. 13. Task Queues and Scheduled Tasks
      1. Task Queues
      2. Scheduled Tasks
    16. 14. The Django Web Application Framework
      1. Installing Django
      2. Creating a Django Project
      3. The Request Handler Script
      4. The Django App Engine Helper
      5. Creating a Django Application
      6. Using App Engine Models With Django
      7. Using Django Unit Tests and Fixtures
      8. Using Django Forms
    17. 15. Deploying and Managing Applications
      1. Uploading an Application
      2. Using Versions
      3. Managing Service Configuration
      4. Managing Indexes
      5. Browsing and Downloading Logs
      6. Inspecting the Datastore
      7. Application Settings
      8. Managing Developers
      9. Quotas and Billing
      10. Getting Help
    18. Index
    19. About the Author
    20. Colophon
    21. SPECIAL OFFER: Upgrade this ebook with O’Reilly

Chapter 1. Introducing Google App Engine

Google App Engine is a web application hosting service. By “web application,” we mean an application or service accessed over the Web, usually with a web browser: storefronts with shopping carts, social networking sites, multiplayer games, mobile applications, survey applications, project management, collaboration, publishing, and all of the other things we’re discovering are good uses for the Web. App Engine can serve traditional website content too, such as documents and images, but the environment is especially designed for real-time dynamic applications.

In particular, Google App Engine is designed to host applications with many simultaneous users. When an application can serve many simultaneous users without degrading performance, we say it scales. Applications written for App Engine scale automatically. As more people use the application, App Engine allocates more resources for the application and manages the use of those resources. The application itself does not need to know anything about the resources it is using.

Unlike traditional web hosting or self-managed servers, with Google App Engine, you only pay for the resources you use. These resources are measured down to the gigabyte, with no monthly fees or up-front charges. Billed resources include CPU usage, storage per month, incoming and outgoing bandwidth, and several resources specific to App Engine services. To help you get started, every developer gets a certain amount of resources for free, enough for small applications with low traffic. Google estimates that with the free resources, an app can accommodate about 5 million page views a month.

App Engine can be described as three parts: the runtime environment, the datastore, and the scalable services. In this chapter, we’ll look at each of these parts at a high level. We’ll also discuss features of App Engine for deploying and managing web applications, and for building websites integrated with other Google offerings such as Google Apps and Google Accounts.

The Runtime Environment

An App Engine application responds to web requests. A web request begins when a client, typically a user’s web browser, contacts the application with an HTTP request, such as to fetch a web page at a URL. When App Engine receives the request, it identifies the application from the domain name of the address, either an subdomain (provided for free with every app) or a subdomain of a custom domain name you have registered and set up with Google Apps. App Engine selects a server from many possible servers to handle the request, making its selection based on which server is most likely to provide a fast response. It then calls the application with the content of the HTTP request, receives the response data from the application, and returns the response to the client.

From the application’s perspective, the runtime environment springs into existence when the request handler begins, and disappears when it ends. App Engine provides at least two methods for storing data that persists between requests (discussed later), but these mechanisms live outside of the runtime environment. By not retaining state in the runtime environment between requests—or at least, by not expecting that state will be retained between requests—App Engine can distribute traffic among as many servers as it needs to give every request the same treatment, regardless of how much traffic it is handling at one time.

Application code cannot access the server on which it is running in the traditional sense. An application can read its own files from the filesystem, but it cannot write to files, and it cannot read files that belong to other applications. An application can see environment variables set by App Engine, but manipulations of these variables do not necessarily persist between requests. An application cannot access the networking facilities of the server hardware, though it can perform networking operations using services.

In short, each request lives in its own “sandbox.” This allows App Engine to handle a request with the server that would, in its estimation, provide the fastest response. There is no way to guarantee that the same server hardware will handle two requests, even if the requests come from the same client and arrive relatively quickly.

Sandboxing also allows App Engine to run multiple applications on the same server without the behavior of one application affecting another. In addition to limiting access to the operating system, the runtime environment also limits the amount of clock time, CPU use, and memory a single request can take. App Engine keeps these limits flexible, and applies stricter limits to applications that use up more resources to protect shared resources from “runaway” applications.

A request has up to 30 seconds to return a response to the client. While that may seem like a comfortably large amount for a web app, App Engine is optimized for applications that respond in less than a second. Also, if an application uses many CPU cycles, App Engine may slow it down so the app isn’t hogging the processor on a machine serving multiple apps. A CPU-intensive request handler may take more clock time to complete than it would if it had exclusive use of the processor, and clock time may vary as App Engine detects patterns in CPU usage and allocates accordingly.

Google App Engine provides two possible runtime environments for applications: a Java environment and a Python environment. The environment you choose depends on the language and related technologies you want to use for developing the application.

The Java environment runs applications built for the Java 6 Virtual Machine (JVM). An app can be developed using the Java programming language, or most other languages that compile to or otherwise run in the JVM, such as PHP (using Quercus), Ruby (using JRuby), JavaScript (using the Rhino interpreter), Scala, and Groovy. The app accesses the environment and services using interfaces based on web industry standards, including Java servlets and the Java Persistence API (JPA). Any Java technology that functions within the sandbox restrictions can run on App Engine, making it suitable for many existing frameworks and libraries. Notably, App Engine fully supports Google Web Toolkit (GWT), a framework for rich web applications that lets you write all of the app’s code—including the user interface—in the Java language, and have your rich graphical app work with all major browsers without plug-ins.

The Python environment runs apps written in the Python 2.5 programming language, using a custom version of CPython, the official Python interpreter. App Engine invokes a Python app using CGI, a widely supported application interface standard. An application can use most of Python’s large and excellent standard library, as well as rich APIs and libraries for accessing services and modeling data. Many open source Python web application frameworks work with App Engine, such as Django, web2py, and Pylons, and App Engine even includes a simple framework of its own.

The Java and Python environments use the same application server model: a request is routed to an app server, the application is started on the app server (if necessary) and invoked to handle the request to produce a response, and the response is returned to the client. Each environment runs its interpreter (the JVM or the Python interpreter) with sandbox restrictions, such that any attempt to use a feature of the language or a library that would require access outside of the sandbox fails with an exception.

While using a different server for every request has advantages for scaling, it’s time-consuming to start up a new instance of the application for every request. App Engine mitigates startup costs by keeping the application in memory on an application server as long as possible and reusing servers intelligently. When a server needs to reclaim resources, it purges the least recently used app. All app servers have the runtime environment (JVM or Python interpreter) preloaded before the request reaches the server, so only the app itself needs to be loaded on a fresh server.

Applications can exploit the app caching behavior to cache data directly on the app server using global (static) variables. Since an app can be evicted between any two requests (and low-traffic apps are evicted frequently), and there is no guarantee that a given user’s requests will be handled by a given server, global variables are mostly useful for caching startup resources, like parsed configuration files.

I haven’t said anything about which operating system or hardware configuration App Engine uses. There are ways to figure out what operating system or hardware a server is using, but in the end it doesn’t matter: the runtime environment is an abstraction above the operating system that allows App Engine to manage resource allocation, computation, request handling, scaling, and load distribution without the application’s involvement. Features that typically require knowledge of the operating system are either provided by services outside of the runtime environment, provided or emulated using standard library calls, or restricted in logical ways within the definition of the sandbox.

The best content for your career. Discover unlimited learning on demand for around $1/day.