Chapter 2
Information Theory Basics
2.1 Issues in Information Theory
The ultimate aim of telecommunications is to communicate information between two geographically separated locations via a communications channel, with adequate quality. The theoretical foundations of information theory accrue from Shannon’s pioneering work [133, 215–217], and hence most tutorial interpretations of his work over the past fifty years rely fundamentally on [133, 215–217]. This chapter is no exception in this respect. Throughout the chapter we make frequent references to Shannon’s seminal papers and to the work of various authors offering further insights into Shannonian information theory. Since this monograph aims to provide an all-encompassing coverage of video compression and communications, we begin by addressing the underlying theoretical principles using a light-hearted approach, often relying on worked examples.
Early forms of human telecommunications were based on smoke, drum or light signals, bonfires, semaphores and the like. Practical information sources can be classified as analog and digital. The output of an analog source is a continuous function of time, such as, for example, the air pressure variation at the membrane of a microphone due to someone talking. The roots of Nyquist’s sampling theorem are based on his observation of the maximum achievable telegraph transmission rate over bandlimited channels [218]. In order to be able to satisfy Nyquist’s sampling theorem the analog source ...
