Today’s most advanced computing technology exists in large buildings containing vast arrays of low-cost servers. Enormous computing resources are housed in well-fortified, secure areas, maintained by teams of highly trained technicians. The photograph in Figure P-1 depicts Google’s 115,000-square-foot data center in Council Bluffs, Iowa, taken from an interactive website describing Google’s amazing network of data centers (see http://www.google.com/about/datacenters/gallery).
In the mainframe era, if you outgrew the capacity of a single computer, you needed to come up with millions of dollars to buy another computer. Modern data centers achieve scalable capacity by allocating tasks across large numbers of commodity servers. In the data center era, you allocate as many inexpensive servers as you need and then relinquish those resources when you’re done.
Until recently, data center resources were accessible by the few engineers fortunate enough to work for a new generation of technology companies. However, over the past few years, a revolution has taken place. Just as earlier revolutions in computer hardware made it feasible for more people to access larger numbers of smaller computers, cloud computing enables even greater access, via the public Internet, to vast clusters of computers in modern state-of-the-art data centers. And just as it did in the past, this expanded accessibility is stimulating tremendous innovation.
In its short history, Google has pioneered many of the techniques and best practices used to build and manage cloud computing services. From Search to Gmail to YouTube to Maps, Google services provide secure, scalable, reliable cloud computing to millions of users and serve billions of queries every day. Now, with Google Compute Engine, the infrastructure that supports those services is available to everyone.
Compute Engine offers many advantages: leading-edge hardware, upgraded regularly and automatically; virtually unlimited capacity to grow or shrink a business on demand; a flexible charging model; an army of experts maintaining computing and networking resources; and the ability to host your resources in a global network engineered for security and performance.
This book provides a guided tour of Google Compute Engine, with a focus on solving practical problems. At a high level, Google Compute Engine is about giving you access to the world’s most advanced network of data centers—the computing resources that power Google itself. Practically speaking, this means providing APIs, command-line tools and web user interfaces to use Google’s computing and networking resources.
Figure P-2 shows how all of Compute Engine’s components fit together. At a high level, Compute Engine instances, networks, and storage are all owned by a Compute Engine project. A Compute Engine project is essentially a named collection of information about your application and acts as a container for your Compute Engine resources. Any Compute Engine resources that you create, such as instances, networks, and disks, are associated with, and contained in, your Compute Engine project. The API offers a way to interact with Compute Engine components and resources programmatically.
We’ll explore the following Google Compute Engine components and resources:
Projects, access modes, and the API (Chapter 1)
Persistent disk (Chapter 3)
Google Cloud Storage (Chapter 4)
Cloud SQL and Cloud Datastore (Chapter 5)
Firewalls, load balancing, and the private network (Chapter 6)
A complete application (Chapter 8)
gcloud compute, a command-line interface; and the Compute Engine API, a RESTful interface.
Most applications require a location for storing their data. The storage available to Compute Engine comes in many flavors, including persistent disks, Google Cloud Storage, Cloud SQL, and Cloud Datastore.
As the name implies, a persistent disk stores data beyond the life of any associated instance(s). Cloud Storage allows you to store, access, and manage objects of any size on Google’s infrastructure. Cloud Storage offers an excellent option for highly durable, high availability data storage. Your data is accessible both inside and outside the scope of Compute Engine via a variety of mechanisms and tools, including the Developers Console, the
gsutil command, and the Cloud Storage API.
Cloud SQL provides a MySQL service in the cloud, managed by Google, while Cloud Datastore provides a Google scale NoSQL data service. Both of these services are available inside and outside the scope of Compute Engine.
In Chapter 8, we present a guided tour of an application that ties together several of the topics covered earlier. Using an example application, this chapter builds a simple but comprehensive cloud computing application, step by step.
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values determined by context.
This element signifies a tip or suggestion.
This element signifies a general note.
This element indicates a warning or caution.
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Book Title by Some Author (O’Reilly). Copyright 2012 Some Copyright Holder, 978-0-596-xxxx-x.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at firstname.lastname@example.org.
Technology professionals, software developers, web designers, and business and creative professionals use Safari Books Online as their primary resource for research, problem solving, learning, and certification training.
Members have access to thousands of books, training videos, and prepublication manuscripts in one fully searchable database from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technology, and hundreds more. For more information about Safari Books Online, please visit us online.
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://shop.oreilly.com/product/0636920028888.do.
To comment or ask technical questions about this book, send email to email@example.com.
For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia
Marc, Kathryn, and Paul would like to thank their expert editor, Rachel Roumeliotis, for her patience and guidance throughout this project. They’d also like to thank their many helpful reviewers and colleagues, who provided invaluable feedback on many revisions: Andrew Kadatch, Ankur Parikh, Benson Kalahar, Dan Miller, Danielle Aronstam, Dave Barth, Elizabeth Markman, Eric Johnson, Greg DeMichillie, Ian Barber, Jay Judkowitz, Joe Beda, Joe Faith, Johan Euphrosine, Jonathan Burns, Julia Ferraioli, Laurence Moroney, Martin Maly, Mike Trinh, Nathan Herring, Nathan Parker, Phun Lang, Rebecca Ward, Renny Hwang, Scott Van Woudenberg, Simon Newton, and Sunil James.
Marc would like to thank his dedicated, talented, and supportive coauthors for their hard work and inspiration. Marc is also grateful for the love and support of his family, including his wife Kimberly and daughter Maya, who tolerated many long hours spent completing this project.
Kathryn would like to thank her husband, James, and daughter, Violet, for their love and support and her coauthors, Marc and Paul, for being awesome.
Paul would like to thank Marc and Kathryn for the opportunity to collaborate on this project, even though he got to the party late.