O'Reilly logo

Shell Scripting: Expert Recipes for Linux, Bash, and More by Steve Parker

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

xargs

When a command is called with many arguments (which are often filenames), the kernel has to put that data somewhere for the command to reference it. Sometimes (often when using the find command to trawl through large filesystems), this list is longer than the kernel will allow. This results in messages such as “rm: Argument list too long.” Many people are quite used to this and know the workaround is to use xargs, but not everybody understands quite how it works, which can have some significant implications.

rm 'find . -name "*.core*" -print'
bash: /bin/rm: Argument list too long
$ find . -name "*.core*" -print0 | xargs -0 rm
$

To get around the problem, xargs will read its standard input (which has no such restrictions) and pass those on in manageable chunks to the command it has been told to pass them to. This simple listfiles script shows the effect of this on the command being run. Here, the -L 3 option tells xargs to break the input down into three files per call. This is not how everybody thinks that xargs works, so the result may be surprising.

download.eps
cat listfiles
#!/bin/bash
echo "Listfiles (PID $$) was called with $# arguments:"
i=1
while [ -a "$1" ]
do
  echo "${i}: $1"
  ((i++))
  shift
done

listfiles

find . -print
.
./etc
./etc/hosts
./etc/config.txt
./bin
./bin/ls
./sh
./listfiles
$ find . -print | xargs -L 3 ./listfiles Listfiles (PID 17088) was called ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required