Learn Unity ML-Agents - Fundamentals of Unity Machine Learning

Learn Unity ML-Agents - Fundamentals of Unity Machine Learning

by Micheal Lanham
Released June 2018
Publisher(s): Packt Publishing
ISBN: 9781789138139

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Transform games into environments using machine learning and Deep learning with Tensorflow, Keras, and Unity

About This Book
  • Learn how to apply core machine learning concepts to your games with Unity
  • Learn the Fundamentals of Reinforcement Learning and Q-Learning and apply them to your games
  • Learn How to build multiple asynchronous agents and run them in a training scenario
Who This Book Is For

This book is intended for developers with an interest in using Machine learning algorithms to develop better games and simulations with Unity.

The reader will be required to have a working knowledge of C# and a basic understanding of Python.

What You Will Learn
  • Develop Reinforcement and Deep Reinforcement Learning for games.
  • Understand complex and advanced concepts of reinforcement learning and neural networks
  • Explore various training strategies for cooperative and competitive agent development
  • Adapt the basic script components of Academy, Agent, and Brain to be used with Q Learning.
  • Enhance the Q Learning model with improved training strategies such as Greedy-Epsilon exploration
  • Implement a simple NN with Keras and use it as an external brain in Unity
  • Understand how to add LTSM blocks to an existing DQN
  • Build multiple asynchronous agents and run them in a training scenario
In Detail

Unity Machine Learning agents allow researchers and developers to create games and simulations using the Unity Editor, which serves as an environment where intelligent agents can be trained with machine learning methods through a simple-to-use Python API.

This book takes you from the basics of Reinforcement and Q Learning to building Deep Recurrent Q-Network agents that cooperate or compete in a multi-agent ecosystem. You will start with the basics of Reinforcement Learning and how to apply it to problems. Then you will learn how to build self-learning advanced neural networks with Python and Keras/TensorFlow. From there you move o n to more advanced training scenarios where you will learn further innovative ways to train your network with A3C, imitation, and curriculum learning models. By the end of the book, you will have learned how to build more complex environments by building a cooperative and competitive multi-agent ecosystem.

Style and approach

This book focuses on the foundations of ML, RL and DL for building agents in a game or simulation

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Learn Unity ML - Agents - Fundamentals of Unity Machine Learning
  3. Dedication
  4. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  5. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  7. Introducing Machine Learning and ML-Agents
    1. Machine Learning
      1. Training models
      2. A Machine Learning example
      3. ML uses in gaming
    2. ML-Agents
    3. Running a sample
      1. Setting the agent Brain
    4. Creating an environment
      1. Renaming the scripts
    5. Academy, Agent, and Brain
      1. Setting up the Academy
      2. Setting up the Agent
      3. Setting up the Brain
      4. Exercises
    6. Summary
  8. The Bandit and Reinforcement Learning
    1. Reinforcement Learning
      1. Configuring the Agent
    2. Contextual bandits and state
      1. Building the contextual bandits
      2. Creating the ContextualDecision script
      3. Updating the Agent
    3. Exploration and exploitation
      1. Making decisions with SimpleDecision
    4. MDP and the Bellman equation
    5. Q-Learning and connected agents
      1. Looking at the Q-Learning ConnectedDecision script
    6. Exercises
    7. Summary
  9. Deep Reinforcement Learning with Python
    1. Installing Python and tools
      1. Installation
        1. Mac/Linux installation
        2. Windows installation
        3. Docker installation
        4. GPU installation
      2. Testing the install
    2. ML-Agents external brains
      1. Running the environment
    3. Neural network foundations
      1. But what does it do?
    4. Deep Q-learning
      1. Building the deep network
      2. Training the model
      3. Exploring the tensor
    5. Proximal policy optimization
      1. Implementing PPO
      2. Understanding training statistics with TensorBoard
    6. Exercises
    7. Summary
  10. Going Deeper with Deep Learning
    1. Agent training problems
      1. When training goes wrong
        1. Fixing sparse rewards
        2. Fixing the observation of state
    2. Convolutional neural networks
    3. Experience replay
      1. Building on experience
    4. Partial observability, memory, and recurrent networks
      1. Partial observability
      2. Memory and recurrent networks
    5. Asynchronous actor – critic training
      1. Multiple asynchronous agent training
    6. Exercises
    7. Summary
  11. Playing the Game
    1. Multi-agent environments
    2. Adversarial self-play
      1. Using internal brains
      2. Using trained brains internally
    3. Decisions and On-Demand Decision Making
      1. The Bouncing Banana
    4. Imitation learning
      1. Setting up a cloning behavior trainer
    5. Curriculum Learning
    6. Exercises
    7. Summary
  12. Terrarium Revisited – A Multi-Agent Ecosystem
    1. What was/is Terrarium?
    2. Building the Agent ecosystem
      1. Importing Unity assets
      2. Building the environment
    3. Basic Terrarium – Plants and Herbivores
      1. Herbivores to the rescue
      2. Building the herbivore
      3. Training the herbivore
    4. Carnivore: the hunter
      1. Building the carnivore
      2. Training the carnivore
    5. Next steps
    6. Exercises
    7. Summary
  13. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Learn Unity ML-Agents - Fundamentals of Unity Machine Learning
  • Author(s): Micheal Lanham
  • Release date: June 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789138139