O'Reilly logo

Learning Unix for Mac OS X Tiger by Dave Taylor

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Managing Files

The tree structure of the Unix filesystem makes it easy to organize your files. After you make and edit some files, you may want to copy or move files from one directory to another, or rename files to distinguish different versions of a file. You may even want to create new directories each time you start a different project. If you copy a file, it's worth learning about the subtle sophistication of the cp command: if you copy a file to a directory, it automatically reuses the original filename and copies the file to the new location. This can save lots of typing!

More than just saving typing, however, the command line is much more precise, offering greater control than the Finder's drag-and-drop interface. For example, if you want to create a new folder in the Finder, you need to mouse up to the File menu and choose New Folder or use a non-mnemonic keystroke combination. On the command line, it's just mkdir to create a new directory. Even more to the point, if you have a folder full of hundreds of files and want to just move those that have temp in their filenames into the Trash, that's a tedious and error-prone Finder task, while the command-line equivalent is the simple rm *temp*.

A directory tree can get cluttered with old files you don't need. If you don't need a file or a directory, delete it to free storage space on the disk. The following sections explain how to make and remove directories and files.

Creating Directories with mkdir

It's handy to group related files in the same directory. If you were writing a spy novel and reviewed restaurants for a local newspaper, for example, you probably wouldn't want your intriguing files mixed with restaurant listings. You could create two directories: one for all the chapters in your novel (spy, for example) and another for restaurants (boston.dine).

To create a new directory, use the mkdir program. The syntax is:

mkdir dirname(s)

dirname is the name of the new directory. To make several directories, put a space between each directory name. To continue this example, you would enter:

$ mkdir spy boston.dine

This means that if you want to create a directory with a space in the name, you'll need to escape the space just as you had to earlier when you referenced files with spaces in them. To create the directory My Favorite Music, you'd use:

$ mkdir "My Favorite Music"

Another trick is that you can create a new directory and include a bunch of subdirectories within that directory, all from one single command. For example, your spy novel most likely has a few chapters in it, and let's say that you need separate directories for each chapter for holding the chapter file, any illustrations you want to add, research notes, whatever. You could use the following command to create the spy novel's main directory and individual subdirectories for the various chapter directories:

$ mkdir -p spy/{ch{01,02,03,04,05,intro,toc,index,bio}}

The curly braces ({ }) are used to contain the string that starts out with ch for each directory, and then appends that with the comma-delimited items in the enclosed string, which gives you the chapter numbers. Run the following command to see the list of directories and subdirectories you've created:

$ ls -F spy
ch01/           ch03/           ch05/           chindex/        chtoc/
ch02/           ch04/           chbio/          chintro/

Try doing that in the Finder! You can't. To do that, you'd have to first create a folder named spy, open that, and then create and rename all those subfolders. Talk about time consuming! But here, the power of Unix goes into action and saves the day.

Copying Files

If you're about to edit a file, you may want to save a copy of it first. That makes it easy to get back the original version should the edit go haywire. To copy files, use the cp program.

The cp program can put a copy of a file into the same directory or into another directory. cp doesn't affect the original file, so it's a good way to keep an identical backup of a file.

To copy a file, use the command:

cp old new

Here, old is a pathname to the original file and new is the pathname you want for the copy. For example, to copy the /etc/passwd file into a file called password in your home directory, you would enter:

$ cp /etc/passwd ~/password
$

You can also use the form:

cp old olddir

This puts a copy of the original file old into an existing directory olddir. The copy has the same filename as the original.

If there's already a file with the same name as the copy, cp replaces the old file with your new copy. This is handy when you want to replace an old copy of a file with a newer version, but it can cause trouble if you accidentally overwrite a copy you wanted to keep. To be safe, use ls to list the directory before you make a copy there.

Also, cp has an -i (interactive) option that asks you before overwriting an existing file. It works like this:

$ cp -i master existing-file.txt
overwrite existing-file.txt? no
$

(You have to either type yes or no to respond to the question; you can also just type y or n and hit Return.)

You can copy more than one file at a time to a single directory by listing the pathname of each file you want copied, with the destination directory at the end of the command line. You can use relative or absolute pathnames (see the sections "Absolute Pathnames" and "Relative Pathnames" in Chapter 3) as well as simple filenames. For example, let's say your working directory is /Users/carol (from the filesystem diagram in Figure 3-3). To copy three files called ch1, ch2, and ch3 from /Users/john to a subdirectory called Documents (that's /Users/carol/ Documents), enter:

$ cp ../john/ch1.doc ../john/ch2.doc ../john/ch3.doc  Documents

Or you could use wildcards and let the shell find all the appropriate files. This time, let's add the -i option for safety:

$ cp -i ../john/ch[1-3].doc Documents
cp: overwrite work/ch2.doc ? n

This tells you that there is already a file named ch2.doc in the Documents directory. When cp asks, answer n to prevent copying ch2.doc. Answering y overwrites the old ch2doc. As you saw in Chapter 3, the shorthand form . (a single dot or period) refers to the working directory, and .. (dot, dot) refers to the parent directory. For example, the following puts the copies into the working directory:

$ cp ../john/ch[1-3].doc .

One more possibility: when you're working with home directories, you can use a convenient shorthand ~ account to represent John and Carol's home directory (and ~ by itself to represent your own). So here's yet another way to copy those three files:

$ cp ~john/ch[1-3.doc] Documents

cp can also copy entire directory trees with the help of the -R option, for "recursive." There are two arguments after the option: the pathname of the top-level directory from which you want to copy and the pathname of the place where you want the top level of the copy to be.

As an example, let's say that a new employee, Asha, has joined John and Carol. She needs a copy of John's Documents/work directory in her own home directory. (See the filesystem diagram in Figure 3-3.) Her home directory is /Users/asha. If Asha's own work directory doesn't exist yet (important!), she could type the following commands:

$ cd /Users
$ cp -R john/Documents/work asha/work

Or, from her home directory, she could have used:

$ cp -R ~john/Documents/work work

Either way, Asha now has a new subdirectory /Users/asha/work with a copy of all files and subdirectories from /Users/john/Documents/work.

Tip

If you give cp -R the wrong pathnames, it can copy a directory tree into itself—running forever until your filesystem fills up!

When cp copies a file, the new copy has its ownership changed to the user running the cp command, too, so not only does Asha have the new files, but they're also owned by her. Here's an example of how that works:

$ ls -l /etc/shells
-rw-r--r--   1 root     wheel  179 Nov 14 03:30 /etc/shells
$ cp /etc/shells ~
$ ls -l ~/shells
-rw-r--r--   1 taylor  taylor  179 Jan  4 07:59 /Users/taylor/shells
$

Notice that the ~ shortcut for the home directory can also be used as a target directory with a cp command. Very helpful!

Problem checklist

The following tips should help you diagnose any error messages cp throws your way:

The system says something like "cp: cannot copy file to itself."

If the copy is in the same directory as the original, the filenames must be different.

The system says something like "cp: filename: no such file or directory."

The system can't find the file you want to copy. Check for a typing mistake. If a file isn't in the working directory, be sure to use its pathname.

The system says something like "cp: permission denied."

You may not have permission to copy a file created by someone else or to copy it into a directory that does not belong to you. Use ls -l to find the owner and the permissions for the file, or use ls -ld to check the directory. If you feel that you should be able to copy a file, ask the file's owner or use sudo (see "Superuser Privileges with sudo" in Chapter 3) to change its access modes.

Copying Mac files with resources

The cp program works on plain files and directories, but the Macintosh system stores applications in bundles that include various resources used by the application. These attributes are known as resource forks and are used extensively in Mac OS applications and documents. (You will also find them in various places on the Mac OS X filesystem.) If you're a Mac OS 9 veteran, you'll remember that the resources in the resource fork were editable only with ResEdit, and otherwise were hidden in the system. A file's resource fork, if it exists, can be seen by looking at a special file called filename/rsrc. For example, notice how the Calculator application is actually saved on disk:

$ cd /Applications
$ ls -ld Calculator.app
drwxrwxr-x   3 root  admin  102 Jun 22  2004 Calculator.app/

By contrast, look at Figure 4-6, where the applications in Mac OS X are shown in the Finder.

Calculator shows up in the Terminal as a directory, but the Finder says it's an application; it's really a bundle

Figure 4-6. Calculator shows up in the Terminal as a directory, but the Finder says it's an application; it's really a bundle

The Unix command output appears rather puzzling, actually. According to the Finder, the Calculator is an application, not a directory, and if you double-click on the icon, the program launches. If it were a directory you'd just move into the directory, right? But the Unix ls command suggests otherwise because in fact it's not even called Calculator but Calculator.app and it is a directory!

Your might try to use cp -R where you'd otherwise use cp, which, logically, should move the entire directory, including whatever resources are needed:

$ cp Calculator.app ~
cp: Calculator.app is a directory (not copied).
$ cp -R Calculator.app ~
$

A quick glimpse at the Finder in Figure 4-7 shows that it didn't actually work (the application icon got lost in transit).

There's a special version of cp that you need to use instead, a program called CpMac which is available on your system only if you installed the Xcode Tools from Tiger's installation DVD.

With the Xcode Tools installed, you can now copy the application without anything getting lost:

$ /Developer/Tools/CpMac /Applications/Calculator.app ~
$

Notice that the Xcode utilities are in a nonstandard directory path. You will need to either specify the path each time, as shown above, create an alias, or modify your PATH to include the /Developer/Tools directory (Chapter 2 showed how to modify your PATH).

Our copy of Calculator lost its icon

Figure 4-7. Our copy of Calculator lost its icon

Renaming and Moving Files with mv

To rename a file, use mv (move). The mv program can also move a file from one directory to another.

The mv command has the same syntax as the cp command:

mv old new

Here, old is the old name of the file and new is the new name. mv writes over existing files , which is handy for updating old versions of a file. If you don't want to overwrite an old file, be sure that the new name is unique. Like cp, mv has an -i option for moving and renaming files interactively:

$ mv chap1.doc intro.doc
$ mv -i chap2.doc intro.doc
mv: overwrite `intro.doc'? n
$

The previous example changed the file named chap1.doc to intro.doc, and then tried to do the same with chap2.doc (answering n cancelled the last operation). If you list your files with ls, you'll see that the filename chap1.doc has disappeared, but chap2.doc and intro.doc are intact.

The mv command can also move a file from one directory to another. As with the cp command, if you want to keep the same filename, you need only give mv the name of the destination directory. For example, to move the intro.doc file from its present working directory to your Desktop, use the following command:

$ mv intro.doc ~/Desktop

If you need to move a directory (or an application or another Mac OS X file that's actually a directory with resource elements) you need to use CpMac's cousin, the MvMac command:

$ alias MvMac="/Developer/Tools/MvMac"
$ MvMac ~/Calculator.app ~/MyApps/MyCalculator.app
$

The preceding command set up an alias for MvMac then used the alias to move the Calculator and all its resources into the MyApps subdirectory. To retain the alias on your next login, don't forget to add it to your .profile or .cshrc file, as appropriate for your login shell.

Removing Files and Directories

You may finish work on a file or directory and see no need to keep it, or the contents may be obsolete. Periodically removing unwanted files and directories frees storage space and saves you from getting confused when there are too many versions of files on your disk.

rm

The rm program removes files. One important thing to point out here, though, is that rm permanently removes the file from the filesystem. It doesn't move the file to the Trash, from which it can be recovered (at least until you select Empty the Trash from the Finder menu). Once you hit Return, that file is gone, so make darn sure that the file you're deleting with rm is something you really want to get rid of. Let me say that again: rm does not offer a way to recover deleted files.

The syntax is simple:

rm filename(s)

rm removes the named files, as the following example shows:

$ ls
chap10       chap2       chap5    cold
chap1a.old   chap3.old   chap6    haha
chap1b       chap4       chap7    oldjunk
$ rm *.old chap10
$ ls
chap1b    chap4    chap6    cold    oldjunk
chap2     chap5    chap7    haha
$ rm c*
$ ls
haha    oldjunk
$

When you use wildcards with rm, be sure you're deleting the right files! If you accidentally remove a file you need, you can't recover it unless you have a copy in another directory or in your backups.

Tip

Do not enter rm * carelessly. It deletes all the files in your working directory.

Here's another easy mistake to make: you want to enter a command such as rm c* (remove all filenames starting with "c"), but instead enter rm c * (remove the file named c and all the other files in the current directory!)

It's good practice to list the files with ls before you remove them. Or, if you use rm's -i (interactive) option, rm asks you whether you want to remove each file.

If you're security conscious, rm's -P option might appeal to you: files are overwritten three times, with zeros, ones, and then zeros again, before they're removed. This makes it impossible for the data to be recovered—even by the most earnest malicious user. The flag doesn't produce any additional output or confirm that it's done a safe delete, however:

$ ls
haha   oldjunk
$ rm -P haha
$

rmdir

Just as you can create new directories with mkdir, you can remove them with the rmdir program. As a precaution, rmdir won't let you delete directories that contain any files or subdirectories; the directory must first be empty. (The rm -r command removes a directory and everything in it, but use the -r flag with caution: it can be dangerous for beginners.)

The syntax is:

rmdir dirname(s)

If you try to remove a directory that contains files, you'll get the following message:

rmdir: dirname not empty

To delete a directory that contains files or subdirectories:

  1. Enter cd dirname to get into the directory you want to delete.

  2. Enter rm * to remove all files in that directory.

  3. Enter cd .. to go to the parent directory.

  4. Enter rmdir dirname to remove the unwanted directory.

One such error you might encounter when using rmdir is that you might still get the dirname not empty message, even after you've deleted all the files inside. If this happens, use ls -a to check that there are no hidden files (names that start with a period) other than . and .. (the working directory and its parent). The following command is good for cleaning up hidden files (which aren't matched by a simple wildcard such as *). It matches all hidden files except for . (the current directory) and .. (the parent directory):

$ rm -i .[^.]*

Working with Links

If you've used the Mac for a while, you're familiar with aliases , empty files that point to other files on the system. A common use of aliases is to have a copy of an application on the Desktop, or to have a shortcut in your home directory. Within the graphical environment, you make aliases by Control-clicking on an item (a file, folder, application, whatever), and then choosing Make Alias from the context menu. This creates a file with a similar name in the same directory. The only difference is that the alias now has the word alias at the end of its filename. For example, if you were to look at this in Unix, you'd see the following:

$ ls -l *3*
-rw-r--r--  1 taylor  taylor  1546099 23 Sep 20:58 fig0403.pdf
-rw-r--r--  1 taylor  taylor        0 24 Sep 08:34 fig0403.pdf alias

In this case, the file fig0403.pdf alias is an alias pointing to the actual file fig0403.pdf in the same directory. But you wouldn't know it, because it appears to be an empty file: the size is shown as zero bytes.

Tip

If you have a tendency to delete the alias part of a filename, as I do, then one quick technique for identifying if a file is an alias or not is to check out its file size: if it's size 0 but there's actually content when you look at it with less, it's an alias. Failing that, check out that directory in the Finder—use open . as a shortcut—and look for the telltale arrow on the icon.

Unix works with aliases differently; on the Unix side, we talk about links, not aliases. There are two types of links possible in Unix, hard links and symbolic links, and both are created with the ln command.

The syntax is:

ln [-s] source target

The -s option indicates that you're creating a symbolic link, so to create a second file that links to the file fig0403.pdf, the command would be:

$ ln -s fig0403.pdf neato-pic.pdf

and the results would be:

$ ls -l *pdf
-rw-r--r--  1 taylor  taylor  1532749 23 Sep 20:47 fig0401.pdf
-rw-r--r--  1 taylor  taylor  1539493 23 Sep 20:52 fig0402.pdf
-rw-r--r--  1 taylor  taylor  1546099 23 Sep 20:58 fig0403.pdf
lrwxr-xr-x  1 taylor  taylor       18 24 Sep 08:40 neato-pic.pdf@ ->
     fig0403.pdf

One way to think about symbolic links is that they're akin to a note saying "the info you want isn't here, it's in file X." This also implies a peculiar behavior of symbolic links (and Aqua aliases): move, rename, or remove the item being pointed to and you have an orphan link. The system doesn't automatically remove or update symbolic links.

The other type of link is a hard link, which essentially creates a second name entry for the exact same contents. That is, if you create a hard link to fig0403.pdf, you can then delete the original file, and its contents remain accessible through the second filename—even though the original file was deleted. Essentially, they're different doors into the same room (as opposed to a note taped on a door telling you to go to the second door, as would be the case with a symbolic link). Hard links are created with the ln command, except you omit the -s option:

$ ln mypic.pdf copy2.pdf
$ ls -l mypic.pdf copy2.pdf
-rw-r--r--  2 taylor  taylor  1546099 24 Sep 08:45 copy2.pdf
-rw-r--r--  2 taylor  taylor  1546099 24 Sep 08:45 mypic.pdf
$ rm mypic.pdf
$ ls -l copy2.pdf
-rw-r--r--  1 taylor  taylor  1546099 24 Sep 08:45 copy2.pdf

Notice that both files are exactly the same size when the hard link is created. This makes sense because they're both names to the same underlying set of data, so they should be identical. Then, when the original is deleted, the data survives with the second name now as its only name. The only difference is that the second field, the link count, shows 2 when there are two filenames pointing to the same data, but when the original is deleted, the link count of the second entry, copy2.pdf, goes back to 1.

Compressing and Archiving Files

Aqua users may commonly use StuffIt's .sit and .hqx formats for file archives or even the ZIP archive capability of Mac OS X itself (Control-click, choose Create Archive from the context menu, and your Mac promptly creates a .zip archive), but Unix users have many other options worth exploring.

Even though Mac OS X is far superior to Windows XP, we unfortunately live in a Windows world, which means you're going to occasionally send and receive email attachments with Windows users. It's also not uncommon to download shareware from a web or FTP site that's been zipped (a file with a .zip extension). Mac OS X gives you many ways to create your own ZIP archives (and to unzip the ones you receive, too). And if you're interacting with other Unix users (such as Linux, FreeBSD, or even Mac OS X), Mac OS X offers a suite of command-line utilities for batching and unbatching files.

There are three compression programs included with Mac OS X, though the most popular is gzip (the others are compress and bzip2; read their manpages to learn more about how they differ). There's also a very common Unix archive format called tar that I'll cover briefly.

gzip

Though it may initially confuse you into thinking that it's part of the ZIP archive toolset, gzip has nothing to do with the ZIP archive files created by Mac OS X's Make Archive capability. Instead, gzip is actually a compression program that does a very good job of shrinking down individual files for storage and transmission. If you're sending a file to someone with a dial-up connection, for example, running the file through gzip can significantly reduce its size and make it much more portable. Just as importantly, it can help save space on your disk by letting you compress files you want to keep but aren't using currently. gzip works particularly well with tar, too, as you'll see.

The syntax is:

gzip [-v] file(s)

The -v flag offers verbose output, letting the program indicate how much space it saved by compressing the file. Very useful information, as you may expect!

$ ls -l ch06.doc
-rwxr-xr-x  1 taylor  taylor  138240 24 Sep 08:52 ch06.doc
$ gzip -v ch06.doc
ch06.doc:                75.2% -- replaced with ch06.doc.gz
$ ls -l ch06.doc.gz
-rwxr-xr-x  1 taylor  taylor  34206 24 Sep 08:52 ch06.doc.gz

You can see that gzip did a great job compressing the file, saving over 75 percent. Notice that it's automatically appended a .gz filename suffix to indicate that the file is now compressed. To uncompress the file, just use gunzip:

$ gunzip ch06.doc.gz
$ ls -l ch06.doc
-rwxr-xr-x  1 taylor  taylor  138240 24 Sep 08:52 ch06.doc

The amount of space saved by compression varies significantly based on the format of the original data in the file. Some file formats lend themselves to compression, but others end up being just as big as the original file:

$ ls -l 10*.m4a
-rw-r--r--   1 taylor  taylor  4645048 Jan  3 21:29 10 Serpentine Lane.m4a
$ gzip -v 10*.m4a
10 Serpentine Lane.m4a:   0.9% -- replaced with 10 Serpentine Lane.m4a.gz
$ ls -l 10*
-rw-r--r--  1 taylor  taylor  4603044 Jan  3 21:29 10 Serpentine Lane.m4a.gz
$

This example resulted in a space savings of less than 1 percent of the file size.

tar

In the old days, Unix system backups were done to streaming tape devices (today you can only see these units in cheesy 60s Sci-Fi films, the huge round tape units that randomly spin as data is accessed). The tool of choice for creating backups from Unix systems onto these streaming tape devices was tar, the tape archiver. Fast-forward to Mac OS X, and tar continues its long tradition as a useful utility, but now it's used to create files that contain directories and other files within, as an archive. It's similar to the ZIP format, but differs from gzip because its job is to create a file that contains multiple files and directories. gzip, by contrast, makes an existing file shrink as much as possible through compression.

The tar program is particularly helpful when combined with gzip, actually, because it makes creating archive copies of directories simple and effective. Even better, if you use the -z flag to tar, it automatically invokes gzip to compress its output without any further work. Here's a fun bit of jargon, too: compressed tar archives are known in the Unix community as tarballs.

The syntax is:

tar [c|t|x] [flags] files and directories to archive

The tar program is too complex to fully explain here, but in a nutshell, tar -c creates archives, tar -t shows what's in an existing archive, and tar -x extracts files and directories from an archive. The -f file flag is used to specify the archive name, and the -v flag offers verbose output to let you see what's going on. As always, man tar produces lots more information about tar's options.

$ du -s Masters\ Thesis
6704    Masters Thesis
$ tar -czvf masters.thesis.tgz "Masters Thesis"
Masters Thesis/
Masters Thesis/.DS_Store
Masters Thesis/analysis.doc
...
Masters Thesis/Web Survey Results.doc
Masters Thesis/web usage by section.doc
$ ls -l masters.thesis.tgz
-rw-r--r--  1 taylor  staff  853574 24 Sep 09:20 masters.thesis.tgz

Tip

Notice that we gave tar the directory name, rather than a list of files. This ensures that when the directory is unpacked, the files are placed in a new directory (Masters Thesis), rather than filling the current directory. This is a good habit for people who make lots of archives.

In this example, the directory Masters Thesis is 6.7 MB in size, and hasn't been accessed in quite a while. This makes it a perfect candidate for a compressed tar archive. This is done by combining the following options: -c (create), -z (compress with gzip), -v (verbose), and -f file (output file; notice that we added the .gz suffix to avoid later confusion about the file type). In under 10 seconds, a new archive file is created, which is less than 1 MB in size, yet it contains all the files and directories in the original archive. To unpack the archive, use the following command:

$ tar -xvfz masters.thesis.tgz

Files on Other Operating Systems

Chapter 8 explains ways to transfer files across a network—possibly to non-Unix operating systems. Mac OS X has the capability of connecting to a variety of different filesystems remotely, including Microsoft Windows, other Unix systems, and even web-based filesystems.

If the Windows-format filesystem is mounted with your other filesystems, you'll be able to use its files by typing a Unix-like pathname. If you've mounted a remote Windows system's C: drive over a share named winc, you can access the Windows file C:\WORD\REPORT.DOC through the pathname /Volumes/winc/word/report.doc. Indeed, most external volumes are automatically mounted within the /Volumes directory.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required