Practical Perforce

Preface

What Is Perforce?

If you’ve picked up this book simply because of its riveting title, you may be wondering what Perforce is. Perforce is a software configuration management (SCM) system. SCM systems are used by software developers to keep track of all the software they build and all the components that go into it.

A good SCM system can explain the mysteries of software development and head off its disasters—mysteries like lost bug fixes, and disasters like botched file merges. In large-scale and commercial environments, good SCM is absolutely essential to producing good software.

It’s All Software and We’re All Software Developers

SCM was once concerned with files that computer programmers produced. Now it is concerned with files of all types that a business produces. Software, when viewed from the perspective of SCM, is any endeavor that calls a computer home. Documention, web content, spreadsheets, schematics, graphics, sound—it’s all software. If it’s stored in computer files and gets built, embedded, or packaged into a deliverable result, it’s software. The term software developer may not sound like it applies to web content authors, graphic artists, test engineers, and technical writers, but for the purpose of this book, anyone whose work involves creating computer files from intellectual thought is a software developer.

Perforce, like all SCM systems, keeps track of changes as people do concurrent, parallel work on files. It logs activity, reports who did what; compares, merges, and branches files; and stores files and file configurations. Some of Perforce’s most salient features are:

The depot

Perforce stores files in a protected repository known as the depot. The depot is a centrally located, permanent archive of all file content submitted by users.^[*]

Workspaces

Perforce users work on files in workspaces, private disk areas of their own that contain copies of depot files. In this book we’ll describe effective ways developers can use workspaces, and we’ll also discuss how workspaces can be used to automate nightly builds, release packaging, web staging, and other software production tasks.

Changelists

Perforce changelists tie files changed together into single units of work. Every change to the depot can be traced to a changelist, and every changelist marks a known, reproducible state of the depot; the depot evolves as changelists are submitted. In the Perforce view of SCM, it is the changelist—not the file revision, nor the delta—that is the atomic transaction of software development. This book will discuss a variety of ways changelists can be used, including treating them as snapshots and using them to identify file dependencies.

Filespecs and views

The Perforce filespec syntax, and the views that use it, allow selection of files for Perforce operations. Filespecs can define not only the common file collections, like directories, but arbitrary collections of files that constitute codelines, modules, delivery streams, and other containers. They are the key to treating collections of files as versioned objects that can be inspected, rolled back, branched, labeled, compared, and merged at any version.

Jobs

In Perforce you can record externally defined tasks and states—bug reports, feature requests, and project milestones, for example—in objects it calls jobs. Jobs can be linked to changelists to provide a record of software changes related to tasks. Jobs are also the linchpin of any integration between Perforce and external systems, as we’ll see in later chapters.

Branching

Perforce uses Inter-File Branching to model file variants. In the traditional version tree branching model used by most SCM systems, a file can be be branched and merged only into revisions of itself. In Perforce, any two files can have a branching relationship; branched files are peers, not offshoots, of their originals. A number of chapters in this book are dedicated to describing Perforce branching and its unexpectedly useful applications.

Integration history

In Perforce, branching and merging are referred to as integration. Perforce records a history of integration events and uses it to direct merges and prevent unnecessary remerging. In this book you’ll see how Perforce does that and learn how to anticipate the effect of merges you perform.

Change tracking

Perforce combines filespecs, changelists, jobs, and integration history to track changes as they are merged from branch to branch. In this book you’ll learn how these objects can be used to determine whether a change—a bug fix, for example—made in one branch has been merged to another, no matter how distantly related.

In addition to these features, which could be considered the interesting capabilities of Perforce, there are also the standard housekeeping and productivity features you’re likely to find in any SCM system, including labels, triggers, change notification, graphical merge tools, file histories, and so forth.

The Perforce System in a Nutshell

Perforce is a client/server system (see Figure P-1). The domain of a Perforce system encompasses a master file repository (the depot), a database, and a constellation of users running client programs. One Perforce Server typically serves an entire Perforce domain. Its job is to communicate with Perforce client programs, analyze and execute user commands, archive and serve up file content, run event triggers, and record system activity in the Perforce database. It also performs a variety of database housekeeping tasks, some on demand and some automatically.

The client component of Perforce, shown in Figure P-2, manages workspace files and communicates with the server. It’s implemented in a variety of tools designed for users at almost every technical level.

Perforce client tools can be divided roughly into three categories:

Graphical user interfaces

The Perforce GUIs are the point-and-click client applications. This category includes P4V, P4Win, and P4Web. (The latter is actually more of a plug-in, but because it turns your browser into a Perforce GUI it is marketed as a GUI itself.) Although they don’t support every possible Perforce command, the GUIs do support the day-to-day operations of the typical software developer, and they are easy on the eyes. They also provide a variety of data-mining features, including some very nice visualizations of branching and file evolution. For these reasons, even die-hard command-line adherents find them useful. P4V, P4Win, and P4Web can be used interchangeably, although there are some variations in the range of operations they support. All three come with embedded help files that provide rudimentary coaching in how to use Perforce.

Figure P-1. The Perforce client/server system

Figure P-2. The Perforce client component

Plug-ins

The Perforce plug-ins category consists of client programs that run behind the scenes, usually on the user’s machine, to enable other applications to work with Perforce. The most widely used is the Perforce SCC Plug-in, which integrates Perforce with Visual Studio .NET. (That’s SCC as in Microsoft Source Code Control API; any Windows application that supports the Microsoft SCC API is likely to work just fine with the Perforce SCC Plug-in.)

As Perforce’s popularity grows, plug-ins are emerging that wed less technical applications to Perforce. P4FTP, for example, makes Perforce transparent to people using applications that rely on FTP, and P4Report turns the Perforce database into an ODBC data source for Windows spreadsheet and database tools.

Programmable clients

The programmable interfaces to Perforce are P4 and P4API. P4, the Perforce Command-Line Client, can be used in interactive shells and in scripts. It’s the canonical client program—if you can’t do it with P4, it can’t be done. P4API, the Perforce C/C++ API, is available to embed the client component in applications, scripting languages, and other software. P4 and P4API run on all the operating system platforms Perforce supports—and there are a lot of them—and they support all Perforce operations, including administrative and privileged operations.

Why Perforce?

The features and capabilities of an SCM system are important, but equally so is its ability to meet expectations and thrive in its habitat. Perforce runs as a self-reliant, self-contained system, and you don’t need other software or hardware components installed to use it. Unlike many other SCM systems, Perforce fits into almost any computing environment, thanks to the following features:

Speed

Perforce is fast. It doesn’t make developers wait to check out, check in, compare, or update files, and it doesn’t add a processing burden to developers’ machines.

Centralized repository

In the Perforce system, there is one centralized repository per domain for files and SCM data. Very large companies may have several Perforce domains, but that’s typically an organizational choice, not a limitation of domain size. (Perforce domains at some large companies are known to encompass over 1,000 users each.) While it may be argued that a centralized repository puts your SCM system at risk of a single point of failure, that risk is vastly outweighed by several advantages. First, you have only one machine per domain to take care of to protect your assets. Second, you don’t have to worry about where your assets are. Office moves and machine upgrades don’t perturb your central SCM repository. And third, as long as your central SCM server is running, all your users have access to it. The failure of one machine doesn’t impede SCM access for users elsewhere else in your system.

No external database required

Some SCM systems require you to configure and administer an external database system like SQL Server or MySQL. Perforce provides its own database. When you install Perforce, you’re installing a reliable, self-contained database, customized for Perforce SCM.

Because the Perforce database can’t be accessed by any other means than the Perforce Server, there’s not much that can go wrong. It does require that the system administrator schedule regular checkpoints and backups, but other than that, very little hands-on administration is required. Database recovery performed after a disk failure or other misfortune can be done manually by the system administrator or through automated tools. Perforce provides tools for checkpointing, recovery, and for automatically upgrading the database when a new release is installed.

No reliance on networked file sharing

Some SCM tools rely on networked file sharing (NFS) of one kind or another. NFS is not an ideal solution for SCM; network file sharing can be slow, and makes it difficult for the SCM tool to handle file format differences. (Have you ever opened up a file in Notebook only to see all its lines running together? Or opened up a file in vi and seen ^M characters at the end of every line?) NFS is also very machine-dependent; clock synchronization and other interoperability issues make version control difficult.

Perforce does not use NFS. Instead, it does its own file transport using TCP/IP. This gives it control over the files it cares about and, because TCP/IP is so universally supported, makes it capable of running on more operating system platforms.

No HTTP server required

Some SCM systems require you to configure an HTTP server, like Apache or IIS, to perform the duties of an SCM server. Perforce provides its own server and runs independently of your web servers.

Traditionally, software development organizations were formed of developers working together at the same company, at the same location. Most SCM systems, including Perforce, are suited for that kind of organization. But Perforce has built-in features that make it suitable for nontraditional teams, including teams formed of developers who work outside of the office, developers who work in separate divisions, and even developers who work for completely different companies:

Process impartiality

Perforce imposes almost no built-in workflow or process rules. It’s designed with certain software development activities in mind (all of which will be discussed in later chapters), but it can accommodate almost any procedure or methodology. Any workflow or process you have established (or that you would like to establish) can be automated with Perforce.

File types

While some SCM systems have restrictions on handling certain file types, Perforce can store and manage text files, binary files, Unicode files, native Apple files on the Macintosh, Mac resource forks, and Unix symlinks in its repository.

Product distribution and vendor drops

The Perforce Server can access file repositories in other Perforce domains. This makes a seamless, Perfore-to-Perforce distribution of software products possible. In other words, you can distribute your product directly from your Perforce repository to other organizations, as long as they have Perforce, too. And you can receive vendor drops from other organizations directly from their Perforce repositories. In fact, you can even branch or merge files from their repositories directly into yours. All the while, a history of what you’ve released and received is being collected and recorded in your SCM database.

Firewalls and tunnels

As mentioned, Perforce uses TCP/IP to communicate between its components. The firewall that prevents external access to machines inside your network also prevents access to your Perforce repository. However, that doesn’t mean that all your developers have to be inside your firewall. Perforce can be used in a Virtual Private Network (VPN), when one has been created, and authorized users can use Secure Shell (SSH) to tunnel through a firewall with Perforce commands. The advantage here is that you can extend your SCM—and hence your software development projects—to participants all over the world without having to give them direct access to your machines or intranet.

For a commercial product, Perforce is unusually accessible. Many new users are lured to Perforce simply because it’s so much easier to get started with it than it is with any other SCM system:

Easy to install

Unlike open source software, which generally has to be configured and built, Perforce tools are executable out of the box. By comparison, CVS and Subversion may be free, but they aren’t free of the problems of building open source software. If you’ve ever been down the rabbit hole of trying to find, configure, compile, and install all the components in the dependency chain of an open source tool, you’ll appreciate the simplicity of getting Perforce up and running. It’s literally a 10-minute job: you download a couple of binary files, run one to start up a server, and run the other as your client-side interface to it.

Runs everywhere

Perforce runs on a huge variety of operating system platforms. Your laptop, the fully loaded machine at your office, the discount PC in your child’s room, the old VAX you found on the sidewalk on trash collection day, even the ground-breaking new operating system you’re developing—chances are very good that there’s a version of Perforce that runs on it.^[*] Since its inception, Perforce Software has made a point of porting its tools to as many platforms as possible. That’s been relatively easy to do, because the core components of Perforce are small, standalone programs. And to this day, every version of Perforce ever released can be downloaded for free—in prebuilt, executable form, no less—from the Perforce FTP site.

Costs nothing to try

You can download all Perforce software and documentation for free, without having to talk to a sales rep or even fill out a form on the Web. The software you download is fully functional; it’s the vendor’s intent that you try Perforce and really see if it meets your needs before you commit to buying it. If you want to test-drive Perforce in an environment with more that two users, Perforce Software will give you a limited-term license for as many users as you need. So instead of spending time in meetings arguing with everyone else about whether Perforce will meet your needs, you and your colleagues can spend time actually trying it out.

Easy budget planning

It’s easy to plan a budget for Perforce. How many developers will you have? That’s what you’ll be paying for. Perforce is priced per user, regardless of what they’re doing and the environments in which they’re working.

All-inclusive pricing

Once you’ve paid or Perforce, you can download and run as many server programs as you need, on as many operating systems as you have, as long as you don’t exceed the number of users you’ve paid for. You can run your servers anywhere in the world, and any of your users can use any of your servers. (If you allow them to.) The price you pay includes all of the Perforce client programs, plug-ins, and tools.

Free to students, hermits, and saints

In fact, if you’re going to use Perforce for educational purposes—you’re teaching a programming class, or developing software for a school project, for example—the vendor will provide you with a free license to cover as many users as are involved. Just contact Perforce Software and let them know about your project. You need a license, by the way, only if you have more than two users accessing your Perforce repository. That means that if you’re working on a software development project all on your own, or with just one other person, you can use Perforce for free, forever. And if you are one of the saints developing open source software for no remuneration, you can get a free Perforce license to cover you and everyone else working on your project.

About This Book

This book is written with a particular reader in mind. The reader is familiar with SCM in general, and is most likely a programmer, a project manager, or a build engineer involved with software development. This book is written especially for the reader who wears more than one of those hats on the job and is responsible for some or all of the interconnection between the roles they represent. If you’re pursuing better ways to keep it all connected, and are interested in seeing how Perforce fits in, this book is for you.

One purpose of this book is to present Perforce’s potential as a software configuration management tool. This is a strictly academic purpose—you need not be a Perforce user to gain insight from it. Anyone interested in comparative SCM will find worthwhile material in this book.

The second purpose of this book is to help Perforce users understand why Perforce works the way it does. Most users come to this level of understanding on their own eventually; it is the level of understanding that prompts them to post “Aha!” messages to online Perforce discussions. It is also the level of understanding also makes the difference between simply using Perforce to do what any SCM system can do and exploiting Perforce to accomplish what other systems can’t. This book will get you to that level sooner.

There are two parts to this book:

• Part I (Chapters 1–5) describes Perforce commands and concepts. It’s not a tutorial, nor is it a reference—it’s more of a whirlwind technical tour. It will provide you with a baseline knowledge about fundamental Perforce operations.

• Part II (Chapters 6–11) describes the big picture, using Perforce in a collaborative software development environment. It outlines recommended best practices and shows how to implement them with the Perforce operations you were introducted to in Part I.

Tip

The examples in this book are based on perforce 2005.1, although some features new to Release 2005.2 are covered as well.

What’s Not In This Book

This book contains no tutorials, no hands-on exercises, and no getting-started guides. Although it does contain numerous examples of basic and advanced commands, this book is not meant to be a primary source of instruction for new Perforce users. The role of this book is to complement the existing product manuals with tips and ideas for using Perforce to its full advantage.

This book doesn’t document actual case studies. It’s almost impossible to describe actual case studies without detail-laden examples that put a reader right to sleep. So we’ve foresaken realism on the principle that simple, readable examples can be extrapolated to complicated, real-world solutions more easily than simple solutions can be inferred from painstakingly realistic examples.

This book won’t address industry standards, benchmarks, or certification models, although it will surely be of use to practioners of such standards. Perforce’s strength is in its versatility and accessibility. It makes a robust foundation for a compliance process, but it does not itself enforce compliance.

This book is rather light on system administration issues. Perforce is a tool you can use to great effect without knowing anything about installation, security, backups, upgrades, migration, and performance. When you do need to know about these things, you’ll find the Perforce manuals and other materials that are readily available at the Perforce Software web site to be a rich resource.

This book doesn’t start with a chapter explaining SCM. There was a time when SCM was arcane and indistinct, but those days are gone, and the world now abounds with books and web sites designed to bring novices up to speed.

Additional Reading

Software configuration management, as a topic, is finally conquering measurable shelf space in the computer section of bookstores. A number of SCM issues and challenges have been fully explored by other writers, and this book won’t retread that ground. If you’re a complete SCM novice you might want to take a look at some of the introductory or complementary titles available, including:

Real World Software Configuration Management by Sean Kenefick(APress)

If software configuration management is in your job description, this book is for you. It’s a no-nonsense explanation of SCM best practices with down-to-earth advice about getting going and sticking with them.

Software Configuration Management Patterns: Effective Teamwork, Practical Integration, by Steve Berczuk with Brad Appleton(Addsion Wesley)

This is a concept book that manages to be quite practical nevertheless. Its detailed analyses of SCM problems and solutions are for the most part independent of any particular SCM system. It also offers a comprehensive comparison of the terminology used by contemporary SCM systems.

Configuration Management: The Missing Link in Web Engineering, by Susan Dart(Artech)

This wide-ranging survey of risk management and return on investment includes brief case studies of a variety of SCM systems in use.

Open Source Development with CVS, by Karl Fogel and Moshe Bar(Paraglyph)

This very readable book combines a detailed guide to using CVS with an interesting discussion of its history and its application in open source projects. It’s a good source of insight into how today’s SCM terminology and usage conventions have evolved from their earliest progenitors.

Software Configuration Management Strategies and Rational ClearCase, by Brian A. White(Addison Wesley)

With its in-depth coverage of the ClearCase view of problems and solutions, this book presents an interesting contrast to SCM with Perforce.

The Pragmatic Programmer, by Andrew Hunt and David Thomas(Addison Wesley)

Not about SCM per se, this book touches on many software development practices that harmonize with good SCM.

Finally, while this book will teach you about Perforce, it won’t teach you about all the Perforce commands, command forms, and command options available to you. For that level of detail, go to the Perforce web site and look for the following product manuals:

The Perforce Command Reference

An A-to-Z reference to P4 commands. You may wish to bookmark this manual and refer to it to find out more about—or alternatives to—the command forms and options shown in Practical Perforce.

The P4 User’s Guide

A detailed guide to using Perforce for working with files. This manual is geared toward end-users and uses P4 commands in its examples. Consult this manual for in-depth information about the Perforce user environment and a variety of typical developer tasks.

The Perforce System Administrator’s Guide

A detailed guide to setting up a Perforce Server and managing a Perforce system. Consult this manual for in-depth information on backups, security, triggers, scripting, job customization, review daemons, performance, and OS-specific issues.

The online versions of these and other Perforce product manuals are available free at http://www.perforce.com/perforce/technical.html. You can also buy bound, hard copy versions of the same manuals; check the web site for details.

Conventions Used in This Book

This book uses the following typographic conventions:

• Constant width is used for names of commands, command fragments, and command options.

• Italic is used for filenames and for characters used in the context of file identifiers.

• Button labels in graphical application windows are shown in regular text, and are often intercapped.

• Menu → Item → Item represents sequential selections in graphical application menus.

• Examples that show commands as they are typed, but that do not show command output, look like this:

  type this command

• Examples that show commands as they are typed, and that show command output as well, look like this:

  type this command
  and you will see
  output that looks like this

• Examples that show the contents of files and scripts look like this:

  these are lines
  that appear in
  a file

• Examples of Perforce spec forms look like this:

  Field1       value1
  Field2       value2

In addition, the following formats are used to grab your attention and relieve the tedium of what could otherwise be monotonous reading:

Tip

Indicates a tip, suggestion, or general advice.

Warning

Indicates a warning or caution.

Using Code Examples

This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: "Practical Perforce, by Laura Wingerd. Copyright © 2006 O’Reilly Media, Inc., 0-596-10185-6.”

If you feel that your use of code examples falls outside fair use or the permission given here, feel free to contact us at [e-mail].

Safari® Enabled

When you see a Safari® Enabled icon on the cover of your favorite technology book, that means the book is available online through the O’Reilly Network Safari Bookshelf.

Safari offers a solution that’s better than e-books. It’s a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://safari.oreilly.com.

How to Contact Us

We have tested and verified the information in this book to the best of our ability, but you may find that features have changed or that we have made mistakes. If so, please notify us by writing to:

O’Reilly Media, Inc.

1005 Gravenstein Highway North

Sebastopol, CA 95472

[phone] (in the United States or Canada)

[phone] (international or local)

[phone] (FAX)

You can also send messages electronically. To be put on the mailing list or request a catalog, send email to:

[e-mail]

To ask technical questions or comment on the book, send email to:

[e-mail]

We have a web site for this book, where you can find examples and errata (previously reported errors and corrections are available for public view there). You can access this page at:

http://www.oreilly.com/catalog/practicalperforce

Acknowledgements

Practical Perforce couldn’t have been written without the participation and encouragement of many people. I thank Christopher Seiwald, creator of Perforce the product and Perforce the company, for seeing the value in this project from its outset. I thank Kathy Baldanza for coaxing early drafts out of me and asking painful questions like “How’s that book coming?” I thank everyone who reviewed the drafts (especially Jason Kao!) for catching so many of my dumb mistakes. I thank Jonathan Gennick at O’Reilly for making the endgame painless. And I thank Chris Comparini for making many hours spent sitting with a laptop a pleasant and companionable experience.

Above all, I thank the many people—users, customers, consultants, colleagues, and friends—who have indulged my compulsion to talk—at trade shows, at conferences, at the office, and at parties—about how SCM works in general and how Perforce pays off in particular. Their stories, their diagrams, and their sharp insights have shaped my ideas. They are the “we” in this book, the voice that narrates the knowledge I’ve tried to impart.

---

^[*] The data format of the Perforce depot is not proprietary; it is, in fact, consistent with the RCS archive format. Because of this, there is a common misperception that Perforce is an RCS wrapper. It’s not.

^[*] The Perforce web site used to boast that “If the client program doesn’t run on your platform, we’ll port it there.” Paradoxically, while compatibility with exotic, leading-edge platforms gave Perforce a foot up in the SCM market, the market itself has become more homogeneous. Today, established Windows and Linux operating systems seem to be the preferred platforms for even the newest software technology projects.