Synthetic Vision

Synthetic Vision

by Scott Krig
Released August 2018
Publisher(s): De Gruyter
ISBN: 9781501506291

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

In Synthetic Vision: Using Volume Learning and Visual DNA, a holistic model of the human visual system is developed into a working model in C++, informed by the latest neuroscience, DNN, and computer vision research. The author’s synthetic visual pathway model includes the eye, LGN, visual cortex, and the high level PFC learning centers. The corresponding visual genome model (VGM), begun in 2014, is introduced herein as the basis for a visual genome project analogous to the Human Genome Project funded by the US government. The VGM introduces volume learning principles and Visual DNA (VDNA) taking a multivariate approach beyond deep neural networks. Volume learning is modeled as programmable learning and reasoning agents, providing rich methods for structured agent classification networks. Volume learning incorporates a massive volume of multivariate features in various data space projections, collected into strands of Visual DNA, analogous to human DNA genes. VGM lays a foundation for a visual genome project to sequence VDNA as visual genomes in a public database, using collaborative research to move synthetic vision science forward and enable new applications. Bibliographical references are provided to key neuroscience, computer vision, and deep learning research, which form the basis for the biologically plausible VGM model and the synthetic visual pathway. The book also includes graphical illustrations and C++ API reference materials to enable VGM application programming. Open source code licenses are available for engineers and scientists.

Scott Krig founded Krig Research to provide some of the world's first vision and imaging systems worldwide for military, industry, government, and academic use. Krig has worked for major corporations and startups in the areas of machine learning, computer vision, imaging, graphics, robotics and automation, computer security and cryptography. He has authored international patents in the areas of computer architecture, communications, computer security, digital imaging, and computer vision, and studied at Stanford. Scott Krig is the author of the English/Chinese Springer book Computer Vision Metrics, Survey, Taxonomy and Analysis of Computer Vision, Visual Neuroscience, and Deep Learning, Textbook Edition, as well as other books, articles, and papers.

Table of contents

  1. Cover
  2. Title Page
  3. Copyright
  4. Contents
  5. Preface
  6. Chapter 1: Synthetic Vision Using Volume Learning and Visual DNA
    1. Overview
    2. Synthetic Visual Pathway Model
      1. Visual Genome Model
      2. Volume Learning
      3. Classifier Learning and Autolearning Hulls
    3. Visual Genome Project
      1. Master Sequence of Visual Genomes and VDNA
      2. VGM API and Open Source
    4. VDNA Application Stories
      1. Overcoming DNN Spoofing with VGM
      2. Inspection and Inventory Using VDNA
      3. Other Applications for VDNA
    5. Background Trends in Synthetic Intelligence
    6. Background Visual Pathway Neuroscience
      1. Feature and Concept Memory Locality
      2. Attentional Neural Memory Research
      3. HMAX and Visual Cortex Models
      4. Virtually Unlimited Feature Memory
      5. Genetic Preexisting Memory
      6. Neurogenesis, Neuron Size, and Connectivity
      7. Bias for Learning New Memory Impressions
    7. Synthetic Vision Pathway Architecture
      1. Eye/LGN Model
      2. VDNA Synthetic Neurobiological Machinery
      3. Memory Model
      4. Learning Centers and Reasoning Agent Models
      5. Deep Learning vs. Volume Learning
    8. Summary
  7. Chapter 2: Eye/LGN Model
    1. Overview
    2. Eye Anatomy Model
      1. Visual Acuity, Attentional Region (AR)
    3. LGN Model
      1. LGN Image Assembly
      2. LGN Image Enhancements
      3. LGN Magno and Parvo Channel Model
      4. Magno and Parvo Feature Metric Details
      5. Scene Scanning Model
    4. Eye/LGN Visual Genome Sequencing Phases
      1. Magno Image Preparation
      2. Parvo Image Preparation
      3. Segmentation Rationale
      4. Saccadic Segmentation Details
      5. Processing Pipeline Flow
      6. Processing Pipeline Output Files and Unique IDs
    5. Feature Metrics Generation
    6. Summary
  8. Chapter 3: Memory Model and Visual Cortex
    1. Overview
    2. Visual Cortex Feedback to LGN
    3. Memory Impressions and Photographic Memory
    4. CAM Neurons, CAM Features, and CAM Neural Clusters
    5. Visual Cortex and Memory Model Architecture
    6. CAM and Associative Memory
    7. Multivariate Features
    8. Primal Shapes, Colors, Textures, and Glyphs
    9. Feature VDNA
    10. Volume Feature Space, Metrics, Learning
    11. Visual DNA Compared to Human DNA
    12. Spatial Relationship Processing Centers
    13. Strand and Bundle Models
      1. Strand Feature Topology
      2. Strand Learning Example
      3. Bundles
    14. Visual Genome Sequencing
      1. Visual Genome Format and Encodings
    15. Summary
  9. Chapter 4: Learning and Reasoning Agents
    1. Overview
    2. Machine Learning and AI Background Survey
      1. Learning Models
      2. Training Protocols
      3. Reasoning and Inference
    3. Synthetic Learning and Reasoning Model Overview
      1. Conscious Proxy Agents in the PFC
      2. Volume Learning
      3. VGM Classifier Learning
      4. Qualifier Metrics Tuning
      5. Genetically Preexisting Learnings and Memory
      6. Continuous Learning
      7. Associative Learning
    4. Object Learning vs. Category Learning
      1. Agents as Dedicated Proxy Learning Centers
      2. Agent Learning and Reasoning Styles
      3. Autolearning Hull Threshold Learning
      4. Correspondence Permutations and Autolearning Hull Families
      5. Hull Learning and Classifier Family Learning
    5. Autolearning Hull Reference/Target Differences
      1. Structured Classifiers Using MCC Classifiers
      2. VDNA Sequencing and Unique Genome IDs
      3. Correspondence Signature Vectors (CSV)
    6. Alignment Spaces and Invariance
    7. Agent Architecture and Agent Types
      1. Custom Agents
      2. Master Learning Controller: Autogenerated C++ Agents
      3. Default CSV Agents
    8. Agent Ecosystem
    9. Summary
  10. Chapter 5: VGM Platform Overview
    1. Overview
    2. Feature Metrics, Old and New
    3. Invariance
    4. Visual Genomes Database
      1. Global Unique File ID and Genome ID
      2. Neuron Encoder and QoS Profiles
      3. Agent Registry
      4. Image Registry
      5. Strand Registry
      6. Segmenter Intermediate Files
      7. Visual Genome Metrics Files
      8. Base Genome Metrics
      9. Genome Compare Scores
    5. Agent Management
      1. Sequencer Controller
      2. Correspondence Controller
    6. Master Learning Controller (MLC)
    7. CSV Agents
    8. Correspondence Signature Vectors (CSVs)
    9. Group Metric Classifiers (GMCs)
    10. Strand Topological Distance
    11. Interactive Training and Strand Editing
    12. Metric Combination Classifiers (MCCs)
      1. MCC Function Names
      2. MCC Best Metric Search
      3. Metric Combination Classifier (MCC) Summary
    13. VGM Platform Controllers
      1. Image Pre-Processing and Segmenter: lgn
      2. Genome Image Splitter: gis
      3. Compute Visual Genomes: vg
      4. Comparing and Viewing Metrics: vgc
      5. Agent Testing and Strand Management: vgv
    14. Summary
  11. Chapter 6: Volume Projection Metrics
    1. Overview
    2. Memory Structure: 3x3 vs. 3x1
    3. CAM Feature Spaces
    4. CAM Neural Clusters
    5. Volume Projection Metrics
    6. Quantization Space Pyramids
      1. Strand CAM Cluster Pyramids
    7. Volume Metric Details
      1. Volume Impression Recording
      2. Volume Metrics Functions
      3. Volume Metrics Memory Size Discussion
    8. Magno and Parvo Low-Level Feature Tiles
    9. Realistic Values for Volume Projections
    10. Quantized Volume Projection Metric Renderings
    11. Summary
  12. Chapter 7: Color 2D Region Metrics
    1. Overview
    2. Background Research
    3. Color Spaces
      1. RGB Color
      2. LUMA, RGBI, CIELab Intensity
      3. HSL Hue and Saturation
    4. Eye Model Color Ranging
      1. Squinting Model and Sliding Histograms
      2. Sliding Contrast over Cumulative Histograms
      3. Sliding Lightness over Normal Histograms
      4. Sliding Metrics, Centroid, and Best Match
      5. Static Color Histogram Metrics
    5. LGN Model Color Leveling
      1. Color Level Raw
      2. Color Level Centered
      3. Color Level CIELab Constant
      4. Color Level HSL Saturation Boosting
    6. LGN Model Dominant Colors
      1. Leveled Histogram Distance, Moments
      2. Popularity Colors
      3. Standard Colors
    7. Color Metrics Functions
    8. Summary
  13. Chapter 8: Shape Metrics
    1. Overview
    2. Strand Topological Shape Metrics
      1. Single-Image vs. Multiple-Image Strands
      2. Strand Local Vector Coordinate System
      3. Strand Vector Metrics
      4. Strand Set Metrics
      5. Strand Shape Metrics: Ellipse and Fourier Descriptors
    3. Volume Projection Shape Metrics
      1. Statistical Metrics
      2. Ratio Metrics
    4. Genome Structure Shape Metrics
      1. Genome Structure Local Feature Tensor Space
      2. Genome Structure Correspondence Metrics
    5. Shape Metric Function List
    6. Summary
  14. Chapter 9: Texture Metrics
    1. Overview
    2. Volume Projection Metrics for CAM Clusters
      1. 3x1 RGBI Component Textures
      2. 3x3 RGB Textures
      3. Volume Metric Distance Functions
    3. Haralick Features
      1. Haralick Metrics
    4. SDMX Features
      1. SDMX Metrics
    5. Haralick and SDMX Metric Comparison Graphs
      1. Texture Similarity Graphs (Match < 1.0)
      2. Texture Dissimilarity Graphs (Nonmatch > 1.0)
    6. MCC Texture Functions
    7. CSV Texture Functions
    8. Summary
  15. Chapter 10: Region Glyph Metrics
    1. Overview
    2. Color SIFT
    3. Color Component R,G,B,I G-SURF
    4. Color Component R,G,B,I ORB
    5. RGB DNN
    6. MCC Functions for Glyph Bases
    7. Glyph Base CSV Agent Function
    8. Summary
  16. Chapter 11: Applications, Training, Results
    1. Overview
    2. Test Application Outline
    3. Strands and Genome Segmentations
      1. Building Strands
      2. Parvo Strand Example
      3. Magno Strand Example
      4. Discussion on Segmentation Problems and Work-arounds
      5. Strand Alternatives: Single-image vs. Multi-image
    4. Testing and Interactive Reinforcement Learning
      1. Hierarchical Parallel Ensemble Classifier
      2. Reinforcement Learning Process
    5. Test Genomes and Correspondence Results
      1. Selected Uniform Baseline Test Metrics
      2. Test Genome Pairs
      3. Compare Leaf : Head (Lo-res) Genomes
      4. Compare ront Squirrel : Stucco Genomes
      5. Compare Rotated Back : Brush Genomes
      6. Compare Enhanced Back : Rotated Back Genomes
      7. Compare Left Head : Right Head Genomes
      8. Test Genome Correspondence Scoring Results
      9. Scoring Results Discussion
      10. Scoring Strategies and Scoring Criteria
    6. Unit Test for First Order Metric Evaluations
      1. Unit Test Groups
      2. Unit Test Scoring Methodology
      3. MATCH Unit Test Group Results
      4. NOMATCH Unit Test Group Results
      5. CLOSE Unit Test Group Results
    7. Agent Coding
    8. Summary
  17. Chapter 12: Visual Genome Project
    1. Overview
    2. VGM Model and API Futures
    3. VGM Cloud Server, API, and iOS App
    4. Licensing, Sponsors, and Partners
  18. Bibliography
  19. Index

Product information

  • Title: Synthetic Vision
  • Author(s): Scott Krig
  • Release date: August 2018
  • Publisher(s): De Gruyter
  • ISBN: 9781501506291