Programming is as much an art as a science. The undisputed “bible” of programming, a 2,500 page multivolume work by Donald Knuth, is called The Art of Computer Programming. Many books have been written on Literate Programming, recognizing that humans, not just computers, must read and understand programs. Here we pick up on some issues of programming style that have important ramifications for the readability of your code, including code layout, procedural versus declarative style, and the use of loop variables.
When writing programs you make many subtle choices about names, spacing, comments, and so on. When you look at code written by other people, needless differences in style make it harder to interpret the code. Therefore, the designers of the Python language have published a style guide for Python code, available at http://www.python.org/dev/peps/pep-0008/. The underlying value presented in the style guide is consistency, for the purpose of maximizing the readability of code. We briefly review some of its key recommendations here, and refer readers to the full guide for detailed discussion with examples.
Code layout should use four spaces per indentation level. You should make sure that when you write Python code in a file, you avoid tabs for indentation, since these can be misinterpreted by different text editors and the indentation can be messed up. Lines should be less than 80 characters long; if necessary, you can break a line inside parentheses, brackets, or braces, because Python is able to detect that the line continues over to the next line, as in the following examples:
>>> cv_word_pairs = [(cv, w) for w in rotokas_words ... for cv in re.findall('[ptksvr][aeiou]', w)] >>> cfd = nltk.ConditionalFreqDist( ... (genre, word) ... for genre in brown.categories() ... for word in brown.words(categories=genre)) >>> ha_words = ['aaahhhh', 'ah', 'ahah', 'ahahah', 'ahh', 'ahhahahaha', ... 'ahhh', 'ahhhh', 'ahhhhhh', 'ahhhhhhhhhhhhhh', 'ha', ... 'haaa', 'hah', 'haha', 'hahaaa', 'hahah', 'hahaha']
>>> if (len(syllables) > 4 and len(syllables) == 3 and ... syllables in [aeiou] and syllables == syllables): ... process(syllables) >>> if len(syllables) > 4 and len(syllables) == 3 and \ ... syllables in [aeiou] and syllables == syllables: ... process(syllables)
Typing spaces instead of tabs soon becomes a chore. Many programming editors have built-in support for Python, and can automatically indent code and highlight any syntax errors (including indentation errors). For a list of Python-aware editors, please see http://wiki.python.org/moin/PythonEditors.
We have just seen how the same task can be performed in different ways, with implications for efficiency. Another factor influencing program development is programming style. Consider the following program to compute the average length of words in the Brown Corpus:
>>> tokens = nltk.corpus.brown.words(categories='news') >>> count = 0 >>> total = 0 >>> for token in tokens: ... count += 1 ... total += len(token) >>> print total / count 4.2765382469
In this program we use the variable
count to keep track of the number of tokens seen, and
total to store the combined length
of all words. This is a low-level style, not far removed from machine
code, the primitive operations performed by the computer’s CPU. The
two variables are just like a CPU’s registers, accumulating values at
many intermediate stages, values that are meaningless until the end.
We say that this program is written in a
procedural style, dictating the machine
operations step by step. Now consider the following program that
computes the same thing:
The first line uses a generator expression to sum the token
lengths, while the second line computes the average as before. Each
line of code performs a complete, meaningful task, which can be
understood in terms of high-level properties like: “
total is the sum of the lengths of the
tokens.” Implementation details are left to the Python interpreter.
The second program uses a built-in function, and constitutes
programming at a more abstract level; the resulting code is more
declarative. Let’s look at an extreme example:
>>> word_list =  >>> len_word_list = len(word_list) >>> i = 0 >>> while i < len(tokens): ... j = 0 ... while j < len_word_list and word_list[j] < tokens[i]: ... j += 1 ... if j == 0 or tokens[i] != word_list[j]: ... word_list.insert(j, tokens[i]) ... len_word_list += 1 ... i += 1
>>> word_list = sorted(set(tokens))
Another case where a loop counter seems to be necessary is for
printing a counter with each line of output. Instead, we can use
enumerate(), which processes a
s and produces a tuple of
(i, s[i]) for each item in
s, starting with
(0, s). Here we enumerate the keys of the
frequency distribution, and capture the integer-string pair in the
word. We print
rank+1 so that the counting appears to start
1, as required when producing
a list of ranked items.
>>> fd = nltk.FreqDist(nltk.corpus.brown.words()) >>> cumulative = 0.0 >>> for rank, word in enumerate(fd): ... cumulative += fd[word] * 100 / fd.N() ... print "%3d %6.2f%% %s" % (rank+1, cumulative, word) ... if cumulative > 25: ... break ... 1 5.40% the 2 10.42% , 3 14.67% . 4 17.78% of 5 20.19% and 6 22.40% to 7 24.29% a 8 25.97% in
It’s sometimes tempting to use loop variables to store a maximum or minimum value seen so far. Let’s use this method to find the longest word in a text.
>>> text = nltk.corpus.gutenberg.words('milton-paradise.txt') >>> longest = '' >>> for word in text: ... if len(word) > len(longest): ... longest = word >>> longest 'unextinguishable'
However, a more transparent solution uses two list comprehensions, both having forms that should be familiar by now:
>>> maxlen = max(len(word) for word in text) >>> [word for word in text if len(word) == maxlen] ['unextinguishable', 'transubstantiate', 'inextinguishable', 'incomprehensible']
Note that our first solution found the first word having the longest length, while the second solution found all of the longest words (which is usually what we would want). Although there’s a theoretical efficiency difference between the two solutions, the main overhead is reading the data into main memory; once it’s there, a second pass through the data is effectively instantaneous. We also need to balance our concerns about program efficiency with programmer efficiency. A fast but cryptic solution will be harder to understand and maintain.
>>> sent = ['The', 'dog', 'gave', 'John', 'the', 'newspaper'] >>> n = 3 >>> [sent[i:i+n] for i in range(len(sent)-n+1)] [['The', 'dog', 'gave'], ['dog', 'gave', 'John'], ['gave', 'John', 'the'], ['John', 'the', 'newspaper']]
It is quite tricky to get the range of the loop variable right.
Since this is a common operation in NLP, NLTK supports it with
trigrams(text), and a general-purpose
Here’s an example of how we can use loop variables in building multidimensional structures. For example, to build an array with m rows and n columns, where each cell is a set, we could use a nested list comprehension:
>>> m, n = 3, 7 >>> array = [[set() for i in range(n)] for j in range(m)] >>> array.add('Alice') >>> pprint.pprint(array) [[set(), set(), set(), set(), set(), set(), set()], [set(), set(), set(), set(), set(), set(), set()], [set(), set(), set(), set(), set(), set(['Alice']), set()]]
Observe that the loop variables
are not used anywhere in the resulting object; they are just needed
for a syntactically correct
statement. As another example of this usage, observe that the
['very' for i in
range(3)] produces a list containing three instances of
'very', with no integers in
Note that it would be incorrect to do this work using multiplication, for reasons concerning object copying that were discussed earlier in this section.
>>> array = [[set()] * n] * m >>> array.add(7) >>> pprint.pprint(array) [[set(), set(), set(), set(), set(), set(), set()], [set(), set(), set(), set(), set(), set(), set()], [set(), set(), set(), set(), set(), set(), set()]]