Visual Six Sigma: Making Data Analysis Lean

9.7. Modeling Relationships: Neural Net Models

In her study of data-mining techniques, Mi-Ling has read about how neural nets can be used for classification and prediction. She is interested in exploring neural net classification models for her data. Mi-Ling has read that cross-validation can be used to mitigate the effects of overfitting the data. With this in mind, she decides to fit one neural net model in a naive way, without considering cross-validation, and to select a second neural net model based on the value of a cross-validation statistic for her training data.

9.7.1. Background

Neural net algorithms were originally inspired by how biological neurons are believed to function. Starting in the 1940s, scientists in the area of artificial intelligence pursued the idea of designing algorithms that can learn in a way that emulates neuron function in the human brain.

In fact, the science of biologically informed computation has its origins in a seminal paper called "A Logical Calculus of Ideas Immanent in Nervous Activity."^[] Implicit in this paper's use of logic and computation to describe the nervous system was the concept that ideas are carried by the collection of neurons as a whole, rather than being tied to a specific neuron. Research since these early days has leveraged this idea of distributed processing, and neural nets typically have an input layer of neurons, an output layer, and a hidden layer where processing occurs.^[]

Mi-Ling realizes that in a mathematical sense ...