Over the entire nine month course of the Udacity Self-Driving Car Engineer Nanodegree, only a fraction of the people behind the program ever appear on camera.

There’s myself, of course, and my colleague Ryan Keenan, who taught a number of lessons. A few of my colleagues like Sebastian and Andrew Paster and Andy Brown and Aaron Brown (not related) appear for short cameos.

But there is a small army of colleagues behind the scenes who make everything work. The photo collage above doesn’t even capture everybody.

Here are a few photos I captured recently of the people who make the program happen.

GM and Lyft seem to be heading toward a reckoning, similar to what Google and Uber are experiencing. Minus the allegations of intellectual property theft, at least so far.

Reuters has an article (written by Paul Lienert, a reader of this blog) highlighting the tension between GM’s growing presence in the ridesharing space, on the one hand, and on the other hand GM’s partial ownership, of and partnerships with, Lyft.

On the one hand, GM has invested heavily in Lyft, and holds a 9% ownership stake. GM also benefits from Lyft Express Drive, a Lyft program that leases GM vehicles to Lyft drivers.

On the other hand, GM is launching and expanding a number of programs that are competitive to Lyft.

“Maven can provide GM vehicles directly to ride-sharing drivers who previously leased them through Lyft Express Drive and Uber Vehicle Solutions.”

Similarly, GM’s Cruise subsidiary is beta testing a service called Cruise Anywhere that seems poised to use self-driving cars compete directly with Lyft’s core on-demand transportation service.

Partnerships are tricky, especially because companies’ interests and plans can diverge over time. Scott McNealy famously tweeted:

Ronald Coase won a Nobel Prize in part for theorizing about how ownership affects outcomes. Right now we’re seeing lots of self-driving car companies form partnerships, but I suspect in the future we’ll see many more outright acquisitions. Owning a company, instead of partnering with it, and can help align everyone’s interests.

I am, of course, very proud of the Self-Driving Car Engineer Nanodegree Program we have built at Udacity, which teaches software engineers to become autonomous vehicle engineers. You should enroll!

But there are other educational institutions out there, as well, and one I keep bumping into is the Clemson University International Center for Automotive Research.

CU-ICAR, as they style themselves, is a graduate school about 40 minutes up the road from the main Clemson campus, and it offers master’s and doctoral degrees in automotive engineering across a number of different specialties.

The 250 acre campus in Greenville, South Carolina, is located nearby BMW’s US manufacturing center in Spartanburg, SC, and is a great example of the type of industry-educational partnerships we engage in at Udacity.

I know very little about the Clemson program directly, and I’ve never been to Greenville, but I keep running into their graduates on autonomous vehicle teams at some of our largest hiring partners, so I thought I’d mention them.

I’ve also run into a few Clemson students who are taking the Self-Driving Car Nanodegree Program, so of course that makes me happy 🙂

A couple of days ago I wrote about embedding barcodes into traffic signs to help self-driving cars. Several commenters pointed out a recent academic paper in which researchers (Evtimov, et al.) confused a computer vision system into thinking that a stop sign was a 45 mph sign, with just a few pieces of tape.

This appears to be an extension of a property of neural networks that was already known, which is that they can be fooled in surprising ways. This is called an “adversarial” attack.

Here is an example Justin Johnson gave in the fantastic Stanford CS231n class on convolutional neural networks:

Oops.

So it’s no shocker that the computer vision systems for cars, which rely largely on CNNs, can be fooled.

But notice that it’s not obvious how to apply Justin Johnson’s examples above to an actual printed photo of a goldfish in the real world. The examples above only really work if you have a digital photo of a goldfish.

The breakthrough of the Evtimov et al. paper is that they developed an attack algorithm, which they call Robust Physical Perturbations, that allows them to apply this attack to signs in the real world.

So now we are heading down the road of fooling cars into blowing through stop signs. Is the end nigh?

I’m skeptical.

Hackers hardly need to wait until self-driving cars are on the road before they mess with stop signs. It’s easy enough to cause real carnage today just by removing a stop sign. Indeed, this happens already and the people who do it get convicted of manslaughter. (Although note that particular case was overturned on appeal because it wasn’t clear whether the convicts removed the precise stop sign in question, or a different stop sign.)

I don’t see too many hackers messing with street signs, though, presumably because the result is both fleeting and unpredictable, and the cost (jail time) is high.

In fact, self-driving cars seem even less likely than human drivers to be fooled by tampered stop signs. Self-driving cars are likely to have maps and sensors that could override whatever the car’s camera sees.

It’s possible this paper leads to further breakthroughs in adversarial attacks that could cause more problems, but I don’t think this advance by itself is too worrisome.

Of all the funny stories in the self-driving car world, surely one of the most improbable is the transformation of Velodyne from a subwoofer manufacturer into the world’s premier lidar supplier.

Lidar, an array of lasers, is the key to tracking and understanding the environment around a vehicle, at least until computers get good enough to do this with a camera.

The San Francisco Chronicle has a short writeup of how Dave Hall transformed his audio company into an autonomous sensor company, and I’d love to read the book-length version. It involves the DARPA Grand Challenge and a tinkerer on “the lunatic fringe”. The story is an old-school inventor’s dream.

For now, though, I’m just grateful for Udacity’s two VLP-16 units and our precious HDL-32E.

Also? Velodyne is a Udacity hiring partner.

3M is developing road signs that have specially printed bar codes for self-driving cars, according to Business Insider. This is a clever entry in the vehicle-to-infrastructure communication field.

Often that’s thought of as infrastructure and vehicles communicating back and forth electronically. But this approach, in which the road signs simply have specially encoded information, is much simpler and presumably cheaper.

The article is light on details of how exactly the barcode is written onto the sign, although supposedly the barcode is invisible to humans. Even without that requirement, though, you could imagine tagging each road sign with a small visible barcode, the same way canned goods have barcodes.

Information on the barcode can include the type of sign, of course, but also the GPS coordinates, which would be super-helpful for localization. Other information, about upcoming waypoints or intersections, could also be valuable.

Pretty simple, but effective, and cheap and easy to roll out.

The latest entrant into the ridesharing world is Cruise, which has alpha-launched Cruise Anywhere, a self-driving ride-sharing service for their San Francisco employees.

The service is available to 10% of Cruise’s employees, and only within San Francisco, making the “Anywhere” portion of the title fairly aspirational.

Nonetheless, I remain impressed by the progress of Cruise, which GM bought for a reported $600MM-$1BB a year and a half ago. Often when big industrial behemoths purchase small Silicon Valley startups, the startup gets sucked into the corporate vortex, the employees flee, and in a few years there’s nothing left.

GM has managed to keep Cruise running like something approximating a startup, and Cruise keeps pushing the envelope, with what I believe are more fully autonomous miles driven than any other automotive manufacturer. If Cruise gets to the point where they are putting actual, non-employee passengers in the car, that will be yet another step forward.

Cruise also released a promotional video highlighting Cruise Anywhere. The best part? Cruise Anywhere is dog-friendly.

My boss, Sebastian Thrun, somewhat famously won the 2005 DARPA Grand Challenge. The car built by his Stanford team successfully traversed the 150-mile desert race course. That led to Sebastian’s role building the Google Self-Driving Car Project, and now the Udacity Self-Driving Car Engineer Nanodegree Program.

Less well-known is the 2004 DARPA Grand Challenge, the year prior, in which no vehicle finished. In fact, no vehicle made it further than 7 miles. Most vehicles just died altogether.

Wired has a pretty neat oral history of the 2004 DARPA Grand Challenge. It’s short and worth a quick read.

The most impressive aspect of the 2004 race, really, is that there even was a 2005 race. After watching every vehicle fail in 2004, DARPA threw down the gauntlet again in 2005, and the rest is history.

A reporter asked, “Well, what are you gonna do?” I said, “We’re gonna do it again, and this time it’s going to be a $2 million prize.” It was so successful and yet so not successful, I had to do it again.