O'Reilly logo
live online training icon Live Online training

Getting started with machine learning

An introduction to developing production-ready solutions using systems thinking principles

Kishau Rogers

The future of software is being driven by intelligent applications, and 81% of IT leaders are investing or plan to invest in artificial technology solutions. Given the large amounts of data and information available, more companies are pivoting from preprogrammed software applications to intelligent applications that leverage machine learning algorithms. However, although machine learning is a powerful tool for accelerating the rate of learning, it can also lead to unintended and dangerous consequences that are directly linked to the people and data that train the machines to learn. Avoid these issues by learning to better evaluate, analyze, and understand your systems.

Join expert Kishau Rogers for a hands-on introduction to machine learning. You'll explore scalable project workflows and common pitfalls and walk through the steps for developing and deploying production-ready machine learning solutions. Throughout the course, Kishau incorporates systems thinking principles as a foundation for improving algorithmic performance, providing a deeper understanding of the assumptions and elements (systems) that will influence (train) your models. This hybrid approach integrating machine learning with systems thinking improves a project's effectiveness and ultimately supports the goal of developing machine learning solutions that make a positive impact in the areas that matter.

What you'll learn-and how you can apply it

By the end of this live online course, you’ll understand:

  • Machine learning concepts and where ML fits in within the artificial intelligence ecosystem
  • Systems thinking concepts and why systems thinking is important in machine learning projects
  • The workflow for developing machine learning tools
  • Machine learning types and common algorithms by project goal
  • Common pitfalls for building and validating models
  • How to use systems thinking tools such as system archetypes, the iceberg model, and causal loop diagrams for defining system patterns to avoid common pitfalls of deploying production-ready machine learning tools
  • How to develop a roadmap for your machine learning project

And you’ll be able to:

  • Clarify the question to be answered with machine learning
  • Identify adequate datasets
  • Prepare and clean datasets
  • Validate models and evaluate for overfitting and bias
  • Deploy models in production
  • Monitor and adjust to changes with anomaly detection
  • Define a system’s archetype, the common and recurring patterns of a system
  • Recognize features by cycles presented in feedback and causal loop diagrams
  • Transform your idea discovery and problem-solving processes by identifying a system’s high leverage point

This training course is for you because...

  • You're a software developer interested in developing production-ready machine learning solutions and need to understand the typical workflow and pitfalls for machine learning projects.
  • You're the technical leader or owner of a software product seeking to implement a workflow for integrating machine learning solutions into your product(s) and software development operations.


  • Prior programming experience, ideally in Python or R
  • A working knowledge of data analysis (e.g., handling and analyzing data files—cleaning, preparing, visualizing, etc.)

Pre-course work instructions for participants below:

  1. Sample datasets--->https://www.dropbox.com/sh/eq02t7itycyjtie/AAAOGNipS9qvuBDap56mQrx6a?dl=0

  2. Use Case

Recommended preparation:

The State of Artificial Intelligence (video)

Can We Build AI Without Losing Control of It? (video)

"Attacking Machine Learning with Adversarial Examples” (article)

"Infrastructure for Deep Learning" (article)

Peter Senge's Introduction to Systems Thinking (video)

The System Orientation: From Curiosity to Courage” (article)

"What Is Systems Thinking? The Evolution of the System for Thinking About Systems" (article)

How the Game Is Evolving” (chapter 1 in Systems Thinking)

Less Coding, More Thinking” (podcast episode)

About your instructor

  • For over twenty-three years, Kishau has been an active & awarded practitioner, contributor and business leader in the computer science and information technology industry. Her constant priority has been to develop impactful, innovative solutions to complex problems and business challenges. She has been featured in national publications such as the Wall Street Journal, Entrepreneur, NFIB, and JavaWorld for her technical expertise and entrepreneurial ventures. She is a recipient of the NAWBO Wells Fargo STEM award, Lyn McDermid Community Impact Award and the MBL Entrepreneur of the Year Award.

    Under her leadership, the Websmith brand has operated for over fourteen years, partnering with organizations to make their software ideas real. Since 2004, Websmith, Inc has successfully provided technology solutions to many organizations across the globe. Additionally, Kishau's leadership at Websmith has contributed to the launch of successful startups such as TimeStudy, a health enterprise solution which serves tens of thousands of daily users, PeerLoc, an innovative geolocation platform startup co-founded with Dr. Wei Cheng and bigThinking, a technology consulting agency responsible providing tools for developing innovative solutions to complex problems.


The timeframes are only estimates and may vary according to how the class is progressing

Working definitions (5 minutes)

  • Lecture: A working definition of AI and its disciplines as defined by goals—planning, knowledge representation, perception, robotics, natural language processing (NLP), and machine learning; artificially intelligent systems versus preprogrammed systems; a working definition of systems thinking and a comparison with other thinking disciplines; how systems thinking can support the machine learning project workflow

Machine learning (ML) orientation (10 minutes)

  • Lecture: Knowing when to use machine learning (and when to use the term); common misunderstandings regarding machine learning versus artificial intelligence; real-world applications of machine learning; the characteristics of machine learning applications

Classifying applications as ML projects versus preprogrammed or other technology projects (20 minutes)

  • Lecture: How to form a question that can be answered with data; the types of ML systems and the purpose of each; choosing and defining the boundaries of a whole system at a level suitable for the particular purpose under consideration
  • Hands-on exercise: Clarify your purpose and question to determine the type of model that you’re building

Data acquisition (20 minutes)

  • Lecture: Finding appropriate data assets to answer the question; identifying and evaluating data assets; using the iceberg model to evaluate data assets, events and transactions, rules, policies, limiting factors, delays, and historical context
  • Hands-on exercise: Complete an iceberg model diagram for your data assets

Break (10 minutes)

Data preparation and cleaning (20 minutes)

  • Lecture: Avoiding low-quality data (i.e., inconsistent, duplicate, corrupt datasets); eliminating the unnecessary and identifying the appropriate features; building the ability to see complex relationships; articulating reality and defining boundaries (with features)
  • Hands-on exercise: Implement feature selection via feedback loops

Model building (20 minutes)

  • Lecture: Recognizing and avoiding under- and overfitting (i.e., building a model that fails to learn); using a supervised prediction model use case to understand the statistics behind modeling; a holistic review of patterns and trends over time as a part of the natural dynamics of system
  • Hands-on exercise: Identify training sets and test sets (on paper)

Evaluation and validation (20 minutes)

  • Lecture: Identifying biased datasets before training; identifying precision, sensitivity, specificity, and accuracy
  • Hands-on exercise: Evaluate performance and complete a matrix of true/false positives/negatives based on the perspective of the positive class

Break (10 minutes)

Deploying (20 minutes)

  • Lecture: Building models that add real-world value; exploratory versus production prediction engines—determining how this ML system fits in the larger system and the appropriate point of intervention; identifying cause and effect using an expansion of the feedback loop created during data preparation
  • Hands-on exercise: Use causal loop diagrams to determine the point of highest leverage (i.e., the point in the system where the ML solution offers the highest value)

Monitoring (15 minutes)

  • Lecture: Monitoring models for adaptability; how to use anomaly detection algorithms to flag unexpected, weird changes in behaviors over time; how to determine the impact of changes of time; managing interactions rather than actions; common system archetypes

Wrap-up and Q&A (10 minutes)