Weekend Autonomous Vehicle Link Round-Up

Cattle Roundup Painting by Bill Dunkley
Cattle Roundup is a painting by Bill Dunkley

Playing Pacman With Reinforcement Learning

Deep Reinforcement Learning with Python: Master classic RL, deep RL,  distributional RL, inverse RL, and more with OpenAI Gym and TensorFlow, 2nd  Edition by Sudharsan Ravichandiran

Packt Publishing asked me to review Deep Reinforcement Learning with Python, by Sudharsan Ravichandiran. After spending a few hours with the book, I’m pleased to report that I like it!

The most important aspect of most programming books or courses is how well they support learners in writing the code themselves. As my old boss, Sebastian Thrun, says, “You don’t lose weight by watching other people exercise.”

Ravichandiran’s book cleverly utilizes tools provided by Open AI Gym, along with TensorFlow, to provide lots of short hands-on exercises. The code is provided in Jupyter notebooks, and also explained line-by-line in the text of the book. (Hint: it’s cheating to just hit “Run All Cells” in the notebook! You don’t learn unless you type the code in – and inevitably debug it – yourself.)

I found the book’s mathematical explanations a bit dense, and the typesetting of the mathematical formulas seems off.

But that’s all forgiveable because the book got me clear and effective hands-on experience. Within an hour of opening the book (I did skip around), I trained a deep Q network to play Pacman!

Even When Waymo Goes Wrong, Everything Goes Right

A Waymo went a little rogue in Phoenix recently, with JJRicks in the vehicle, video recording. The vehicle stopped, blocked traffic, and then occasionally took off for a burst while the human support team tried to arrive and wrangle it. It was a bit like watching a rancher try to herd uncooperative livestock.

Everything turned out okay in the end. I’m especially struck by how well the Waymo team, who are kind of a supporting cast in the video, handle the whole incident. The Waymo remote assistance representative stays with Ricks the whole time, and doesn’t appear flustered or panicked. Same thing with the support driver who eventually arrives to take over.

The vehicle never hits anything, or even appears at risk of a collision. It’s a pretty calm scenario, all things considered.

Turning Driving Into An Office Job

Pallet Jack

Bloomberg has a short update on Phantom Auto, a teleoperations startup that has recently partnered with Mitsubishi Logisnext to remotely operate warehouse forklifts.

Phantom Auto was founded by some of the earliest students in the Udacity Self-Driving Car Engineer Nanodegree Program that I led, so I’m excited to see them doing well.

Co-founder Elliot Katz tells Bloomberg that remote operation has “the potential to knock 30% or more off forklift operation costs.” The driver of cost-savings seems to the ability to move jobs to lower-cost areas, since the remote operator doesn’t have to be physically co-located with the equipment.

I imagine that what I’ve heard in agriculture also applies somewhat to logistics, which is that the big advantage of automation (to the extent remote operation counts as automation) is actually increased performance, rather than labor costs. In a field, an automated machine can plant seeds more accurately and squeeze more yield out of the same amount of land, compared to a farm hand. In a warehouse, you could imagine that removing humans from the floor could allow for reconfigurations that would save space, accelerate movement, or otherwise improve efficiency.

I’m also curious how far we can push the migration of in-person jobs into remote operation, and possibly eventual autonomy. Covid-19 has pushed diagnostic telemedicine pretty far in the US, but procedures still require the provider to be physically present. Perhaps in the future, my surgeon or dentist will be operating on me remotely, from another timezone.

Report: Autonomous Vehicle Testing In Europe This Summer

VW ID.Buzz autonomous microbus

Both The Verge and CNBC report that Volkswagen & Argo will begin testing self-driving cars in Europe this summer. That news comes courtesy of a press conference with Brian Salesky, Argo’s CEO, and Christian Senger, a Volkswagen executive.

Announcements about self-driving testing (as opposed to commercial services) have become a little ho-hum in the US, but this seems like a big deal for Europe. So far, Europe has had very little autonomous vehicle testing on public roads, and even less news about such efforts.

Volkwagen is a major investor in Argo, which is headquartered in Pittsburgh, with offices in California and Munich, Germany. Although much of the self-driving software that Volkswagen will test this summer was presumably written in the US, just getting it on the road in Germany will be a meaningful step forward for the European automotive industry.

Cloud Architect Tuesday: Staff Software Engineer, Maps

Cruise is hiring a Staff Software Engineer for Maps. This job looks like a great fit for a cloud architect, even one who has no experience in robotics.

Here are the requirements:

  • 8+ years as a developer
  • Build, ship, debug, and operate full stack web services
  • Distributed systems
  • Microservices
  • API design
  • SOA
  • Node.js (or equivalent)
  • React or Redux
  • PostgreSQL

Now does that list look like a cloud architect, or what?

If you’re a cloud architect, you should be building self-driving cars! They’re amazing!

Email me 🙏

SAE Level 3 Remains A Challenge

My latest Forbes article analyzes SAE’s recent redefinition of its six-level autonomy taxonomy.

“…it’s basically consistent with how the SAE previously defined Level 3: the human passenger isn’t ‘driving’, but has to be ready to take control of the vehicle when the system asks.

That leaves a fair bit of ambiguity, especially about how quickly the human must ‘receive’ requests to intervene. That, in turn raises the quest of how broad the range of other tasks is in which the human can engage, while still allowing enough time to take control if the system requires.”

I think that ultimately vehicle manufacturers will define Level 3 differently in practice. Read the whole thing.

Kyle Vogt’s Insight

GM's Cruise self-driving startup raises $2 billion led by Microsoft -  Electrek

Timothy B. Lee, one of the top journalists covering self-driving technology, just published an article in Ars Technica that asks the question, “Why hasn’t Waymo expanded its driverless service?”

Lee covers several different possibilities, from the cost of remote operations staff, to the difficulty of serving areas like Phoenix Sky Harbor Airport and Arizona State University. Ultimately, he zeros in on a theory he calls, “Kyle Vogt’s Insight.”

“In 2017, Kyle Vogt—Cruise’s founder, then-CEO, and now-CTO—wrote a blog post explaining why Cruise was testing its vehicles in San Francisco.

‘Anyone who has visited San Francisco knows driving here is kind of ridiculous,’ Vogt wrote. ‘Our vehicles encounter challenging (and often absurd) situations up to 46 times more often than other places self-driving cars are tested.'”

Credit to Lee, who owns up to some early skepticism of that theory – skepticism Lee now admits may have been misplaced.

“At the time, I didn’t find Vogt’s argument very convincing. I thought that if Waymo could launch a fully driverless service in an “easy” area like suburban Phoenix, the company would gain a ton of useful experience and data that would give it a leg up in tackling more difficult areas…But now I suspect I was wrong. Maybe after millions of miles in the suburbs, Waymo has reached a performance plateau—it has mastered situations that are common in the suburbs, and it no longer sees enough new challenges to continue improving.”

The whole article is worth a read. It’s a fascinating thought exercise by one of the most informed thinkers in the industry.

To be sure, nobody outside Waymo knows why they haven’t expanded Waymo One since they launched last year. Perhaps they’re going to announce a huge expansion any day now.

But if Lee’s hypothesis is correct, then Kyle Vogt’s contrarian insight will go down as one of the canonical contrarian technology decisions – non-consensus and right – in Silicon Valley history.

Write The Code You Wish You Had

One of my favorite software engineering mantras is, “Write the code you wish you had.” That sentence is a little bit cryptic at first glance, but once I grasped the meaning, it became a great way for me to get unstuck, especially when writing greenfield code.

The idea is that, when writing code, I frequently encounter situations in which I need a function that doesn’t yet exist. These situations can be paralyzing, because I want to continue writing the code I’m focused on right now, but I can’t keep going unless I write this new function. Writing the new function is a context switch that is mentally expensive and I want to avoid it. So I’m stuck.

The trick, and maybe this is obvious to you, even though it wasn’t to me, is that I can keep writing the code I’m focused on now. I can write down proceed with foo = some_function(arg1, arg2); even if I haven’t yet defined, or even declared, some_function(). I just write it down where I need it and keep going!

Of course, the code won’t compile or run, because I’m calling a function that doesn’t exist. But that’s okay, as long as I have some mechanism to ensure that I define the function later.

This works best with test-driven development, an approach whereby I define my automated tests before I write the functional code. That way, I always have a test that fails and reminds me to define the function I wish I had.

I picked up this mantra, “Write the code you wish you had,” through the Ruby on Rails ecosystem, which is fanatical about testing. I don’t remember who, precisely, first taught me the phrase.

But when I search for those words, one of the first results is this short episode of Developer Tea, by Jonathan Cutrell. He describes a similar mental shortcut, which is to think, “Wouldn’t it be nice if…?” He also suspects the phrase originates with Avdi Grimm of RubyTapas.