Managing Projects with GNU Make, 3rd Edition

Preface

The Road to the Third Edition

My first exposure to make was as an undergraduate at Berkeley in 1979. I was thrilled to be working with the “latest” equipment: a DEC PDP 11/70 with 128 kilobytes of RAM, an ADM 3a “glass tty,” Berkeley Unix, and 20 other simultaneous users! Once, when an assignment was due, I remember timing how long it took to log in—five minutes from typing my username until I got a prompt.

After leaving school, it was 1984 before I got to work on Unix again. This time it was as a programmer at NASA’s Ames Research Center. We purchased one of the first microcomputer-based Unix systems, a 68000 (not a 68010 or 20) that had a megabyte of RAM and Unix Version 7—with only six simultaneous users. My last project there was an interactive satellite data analysis system written in C with a yacc/lex command language, and, of course, make.

By 1988, I had gone back to school and was working on a project to build a spline-based geometric modeler. The system consisted of about 120,000 lines of C, spread across 20 or so executables. The system was built using makefile templates that were expanded into normal makefiles by a hand-rolled tool call genmakefile (similar in spirit to imake). The tool performed simple file inclusion, conditional compilation, and some custom logic to manage source and binary trees. It was a common belief in those days that make required such a wrapper to be a complete build tool. Several years earlier, I had discovered the GNU project and GNU make and realized that the wrapper program was probably unnecessary. I rebuilt the build system without the templates or a generator. To my chagrin, I wound up maintaining the build system for the next four years (a pattern I foolishly repeat to this day). The build system was portable to five flavors of Unix and included separate source and binary trees, automated nightly builds, and support for partial checkouts by developers with the build system filling in the missing objects.

My next interesting encounter with make came in 1996, working on a commercial CAD system. It was my job to port 2 million lines of C++ (and 400,000 lines of Lisp) from Unix to Windows NT, using the Microsoft C++ compiler. That’s when I discovered the Cygwin project. As an important byproduct of the port, I reworked the build system to support NT. This build system also supported separate source and binary trees, many flavors of Unix, several varieties of graphics support, automated nightly builds and tests, and partial developer checkouts with reference builds.

In 2000, I began working in Java writing laboratory information management systems. This was one of the first really different development environments I’d worked in for many years. Most of the programmers came from a Windows background and many seemed to know Java as their first programming language. The build environment consisted almost entirely of the project file generated by a commercial Java Integrated Development Environment (IDE). Although the project file was checked in, it rarely worked “out of the box” and programmers often sat in each other’s cubes working through build problems.

Of course, I began to write a build system using make, but an odd thing happened. Many of the developers were reluctant to use any command-line tool. Further, many did not have a firm grasp of such concepts as environment variables, command-line options, or an understanding of the tools used to build programs. The IDE hid all of these issues. To address these issues, the build system I was writing became more complex. I began to add better error messages, precondition checking, management of the developer’s machine configuration, and support for IDEs.

Along the way, I’d read the GNU make manual several dozen times. As I looked for more material, I found the second edition of this book. It was filled with valuable material, but was sadly lacking in details of GNU make. That wasn’t surprising, considering its age. The volume had stood the test of time, but by 2003 needed updating. The third edition focuses primarily on GNU make. As Paul Smith (the GNU make maintainer) writes: “Don’t hassle with writing portable ‘makefiles’, use a portable make instead!”

What’s New in This Edition

Almost all the material in this book is new. I’ve divided the material into three parts.

Part I, provides a moderately detailed examination of the GNU make features and how to use them.

Chapter 1, is a brief introduction to make with a simple, but complete example. It explains the basic concepts of make such as targets and prerequisites, and covers the syntax of makefiles. This should be sufficient to get you going with your first makefiles.

Chapter 2, discusses the structure and syntax of rules. Explicit rules and pattern rules are covered in great detail along with old-fashioned suffix rules. Special targets and simple dependency generation are also discussed here.

Chapter 3, covers simple and recursive variables. This chapter also discusses how makefiles are parsed, when variables are expanded, and the directives for conditional makefile processing.

Chapter 4, examines the wide variety of built-in functions supported by GNU make. User-defined functions are introduced here, with numerous examples ranging from the trivial to illustrations of advanced concepts.

Chapter 5, explains the details of command scripts, covering how they are parsed and evaluated. Here we also discuss command modifiers, checking command exit status, and the command environment. We explore the problems of command-line limits and some ways to work around these problems. At this point, you will know all the GNU make features discussed in this book.

Part II, covers larger topics, such as using make on large projects, portability, and debugging.

Chapter 6, discusses many issues encountered when building large projects in make. The first topic is the use of recursive invocations of make, as well as how to implement these makefiles with a single, nonrecursive makefile. In addition, other issues of large systems are discussed, such as filesystem layout, component management, and automated building and testing.

Chapter 7, discusses issues with portability of makefiles, primarily between various flavors of Unix and Windows. The Cygwin Unix emulation environment is discussed in some detail, along with issues arising from nonportable filesystem features and tools.

Chapter 8, provides specific examples of how to separate source and binary trees and how to create read-only source trees. Dependency analysis is revisited, with an emphasis on language-specific solutions. This chapter and the next tie in closely with many issues raised in Chapter 1.

Chapter 9, explains how to apply make to Java-based development environments. Techniques for managing the CLASSPATH variable, compiling large numbers of files, creating jars, and constructing Enterprise JavaBeans (EJBs) are introduced.

Chapter 10, begins by reviewing the performance characteristics of several make operations to provide context for how to write efficient makefiles. Techniques for identifying and reducing bottlenecks are discussed. The GNU make parallel jobs feature is described in some detail.

Chapter 11, provides two complex examples of real makefiles. The first is the makefile used to create this book. It is interesting, partly due to a fairly extreme degree of automation and partly because it applies make to a nontraditional domain. The other example consists of excerpts from the Linux 2.6 kbuild system.

Chapter 12, delves into the black art of fixing broken makefiles. We introduce techniques for discovering what make is doing under the covers and how to ease development pains.

Part III, includes supplemental material.

Appendix A, provides a reference guide to GNU make’s command-line options.

Appendix B, explores the limits of GNU make with two unlikely capabilities: managing data structures and performing arithmetic.

Appendix C, contains the GNU Free Documentation License, under which the text of the book is distributed.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, file extensions, pathnames, and directories

Constant width

Indicates source code commands, command-line options, the contents of files, or the output from commands

Constant width bold

Shows commands or other text that should be typed literally by the user

Constant width italic

Shows text that should be replaced with user-supplied values

Using Code Examples

This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact O’Reilly for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

O’Reilly appreciates, but does not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Managing Projects with GNU Make, Third Edition, by Robert Mecklenburg. Copyright 2005 O’Reilly Media, Inc., 0-596-00610-1.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact O’Reilly at [e-mail].

Comments and Questions

Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
[phone] (in the United States or Canada)
[phone] (international or local)
[phone] (fax)

O’Reilly maintains a web page for this book, which lists errata, examples, and any additional information. You can access this page at:

• http://www.oreilly.com/catalog/make3

To comment or ask technical questions about this book, send email to:

• [e-mail]

For more information about O’Reilly books, conferences, Resource Centers, and the O’Reilly Network, see O’Reilly’s web site at:

• http://www.oreilly.com

Acknowledgments

I’d like to thank Richard Stallman for providing a vision and the belief that it can come true. Of course, without Paul Smith, GNU make would not exist in its current form today. Thank you.

I’d like to thank my editor, Andy Oram, for his unflagging support and enthusiasm.

Cimarron Software deserves my thanks for providing an environment that encouraged me to begin this project. Realm Systems also deserves thanks for providing an environment that encouraged me to finish the project. In particular, I’d like to thank Doug Adamson, Cathy Anderson, and Peter Bookman.

Thanks to my reviewers, Simon Gerraty, John Macdonald, and Paul Smith, who provided many insightful comments and corrected many embarrassing errors.

The following people deserve thanks for valuable contributions to this work: Steve Bayer, Richard Bogart, Beth Cobb, Julie Daily, David Johnson, Andrew Morton, Richard Pimentel, Brian Stevens, and Linus Torvalds. Many thanks to the cabal that provided a safe haven in stormy seas: Christine Delaney, Tony Di Sera, John Major, and Daniel Reading.

Finally, my profound gratitude and love goes to my wife, Maggie Kasten, and our children, William and James, for their support, encouragement, and love during the last sixteen months. Thank you for sharing this with me.