Playing Pacman With Reinforcement Learning

Deep Reinforcement Learning with Python: Master classic RL, deep RL,  distributional RL, inverse RL, and more with OpenAI Gym and TensorFlow, 2nd  Edition by Sudharsan Ravichandiran

Packt Publishing asked me to review Deep Reinforcement Learning with Python, by Sudharsan Ravichandiran. After spending a few hours with the book, I’m pleased to report that I like it!

The most important aspect of most programming books or courses is how well they support learners in writing the code themselves. As my old boss, Sebastian Thrun, says, “You don’t lose weight by watching other people exercise.”

Ravichandiran’s book cleverly utilizes tools provided by Open AI Gym, along with TensorFlow, to provide lots of short hands-on exercises. The code is provided in Jupyter notebooks, and also explained line-by-line in the text of the book. (Hint: it’s cheating to just hit “Run All Cells” in the notebook! You don’t learn unless you type the code in – and inevitably debug it – yourself.)

I found the book’s mathematical explanations a bit dense, and the typesetting of the mathematical formulas seems off.

But that’s all forgiveable because the book got me clear and effective hands-on experience. Within an hour of opening the book (I did skip around), I trained a deep Q network to play Pacman!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s