I am, of course, very proud of the Self-Driving Car Engineer Nanodegree Program we have built at Udacity, which teaches software engineers to become autonomous vehicle engineers. You should enroll!

But there are other educational institutions out there, as well, and one I keep bumping into is the Clemson University International Center for Automotive Research.

CU-ICAR, as they style themselves, is a graduate school about 40 minutes up the road from the main Clemson campus, and it offers master’s and doctoral degrees in automotive engineering across a number of different specialties.

The 250 acre campus in Greenville, South Carolina, is located nearby BMW’s US manufacturing center in Spartanburg, SC, and is a great example of the type of industry-educational partnerships we engage in at Udacity.

I know very little about the Clemson program directly, and I’ve never been to Greenville, but I keep running into their graduates on autonomous vehicle teams at some of our largest hiring partners, so I thought I’d mention them.

I’ve also run into a few Clemson students who are taking the Self-Driving Car Nanodegree Program, so of course that makes me happy 🙂

A couple of days ago I wrote about embedding barcodes into traffic signs to help self-driving cars. Several commenters pointed out a recent academic paper in which researchers (Evtimov, et al.) confused a computer vision system into thinking that a stop sign was a 45 mph sign, with just a few pieces of tape.

This appears to be an extension of a property of neural networks that was already known, which is that they can be fooled in surprising ways. This is called an “adversarial” attack.

Here is an example Justin Johnson gave in the fantastic Stanford CS231n class on convolutional neural networks:

Oops.

So it’s no shocker that the computer vision systems for cars, which rely largely on CNNs, can be fooled.

But notice that it’s not obvious how to apply Justin Johnson’s examples above to an actual printed photo of a goldfish in the real world. The examples above only really work if you have a digital photo of a goldfish.

The breakthrough of the Evtimov et al. paper is that they developed an attack algorithm, which they call Robust Physical Perturbations, that allows them to apply this attack to signs in the real world.

So now we are heading down the road of fooling cars into blowing through stop signs. Is the end nigh?

I’m skeptical.

Hackers hardly need to wait until self-driving cars are on the road before they mess with stop signs. It’s easy enough to cause real carnage today just by removing a stop sign. Indeed, this happens already and the people who do it get convicted of manslaughter. (Although note that particular case was overturned on appeal because it wasn’t clear whether the convicts removed the precise stop sign in question, or a different stop sign.)

I don’t see too many hackers messing with street signs, though, presumably because the result is both fleeting and unpredictable, and the cost (jail time) is high.

In fact, self-driving cars seem even less likely than human drivers to be fooled by tampered stop signs. Self-driving cars are likely to have maps and sensors that could override whatever the car’s camera sees.

It’s possible this paper leads to further breakthroughs in adversarial attacks that could cause more problems, but I don’t think this advance by itself is too worrisome.

Of all the funny stories in the self-driving car world, surely one of the most improbable is the transformation of Velodyne from a subwoofer manufacturer into the world’s premier lidar supplier.

Lidar, an array of lasers, is the key to tracking and understanding the environment around a vehicle, at least until computers get good enough to do this with a camera.

The San Francisco Chronicle has a short writeup of how Dave Hall transformed his audio company into an autonomous sensor company, and I’d love to read the book-length version. It involves the DARPA Grand Challenge and a tinkerer on “the lunatic fringe”. The story is an old-school inventor’s dream.

For now, though, I’m just grateful for Udacity’s two VLP-16 units and our precious HDL-32E.

Also? Velodyne is a Udacity hiring partner.

3M is developing road signs that have specially printed bar codes for self-driving cars, according to Business Insider. This is a clever entry in the vehicle-to-infrastructure communication field.

Often that’s thought of as infrastructure and vehicles communicating back and forth electronically. But this approach, in which the road signs simply have specially encoded information, is much simpler and presumably cheaper.

The article is light on details of how exactly the barcode is written onto the sign, although supposedly the barcode is invisible to humans. Even without that requirement, though, you could imagine tagging each road sign with a small visible barcode, the same way canned goods have barcodes.

Information on the barcode can include the type of sign, of course, but also the GPS coordinates, which would be super-helpful for localization. Other information, about upcoming waypoints or intersections, could also be valuable.

Pretty simple, but effective, and cheap and easy to roll out.

The latest entrant into the ridesharing world is Cruise, which has alpha-launched Cruise Anywhere, a self-driving ride-sharing service for their San Francisco employees.

The service is available to 10% of Cruise’s employees, and only within San Francisco, making the “Anywhere” portion of the title fairly aspirational.

Nonetheless, I remain impressed by the progress of Cruise, which GM bought for a reported $600MM-$1BB a year and a half ago. Often when big industrial behemoths purchase small Silicon Valley startups, the startup gets sucked into the corporate vortex, the employees flee, and in a few years there’s nothing left.

GM has managed to keep Cruise running like something approximating a startup, and Cruise keeps pushing the envelope, with what I believe are more fully autonomous miles driven than any other automotive manufacturer. If Cruise gets to the point where they are putting actual, non-employee passengers in the car, that will be yet another step forward.

Cruise also released a promotional video highlighting Cruise Anywhere. The best part? Cruise Anywhere is dog-friendly.

My boss, Sebastian Thrun, somewhat famously won the 2005 DARPA Grand Challenge. The car built by his Stanford team successfully traversed the 150-mile desert race course. That led to Sebastian’s role building the Google Self-Driving Car Project, and now the Udacity Self-Driving Car Engineer Nanodegree Program.

Less well-known is the 2004 DARPA Grand Challenge, the year prior, in which no vehicle finished. In fact, no vehicle made it further than 7 miles. Most vehicles just died altogether.

Wired has a pretty neat oral history of the 2004 DARPA Grand Challenge. It’s short and worth a quick read.

The most impressive aspect of the 2004 race, really, is that there even was a 2005 race. After watching every vehicle fail in 2004, DARPA threw down the gauntlet again in 2005, and the rest is history.

A reporter asked, “Well, what are you gonna do?” I said, “We’re gonna do it again, and this time it’s going to be a $2 million prize.” It was so successful and yet so not successful, I had to do it again.

Having grown up in Virginia, I generally keep tabs on Virginia news, especially when it comes to self-driving cars.

This story from ARLNow about a fake self-driving car in Virginia is a little silly. But it’s also bizarre enough to make me wonder what on earth is going on.

I assume that the Virginia Tech Transportation Institute is running some study on whether drivers will even notice self-driving cars. Mission accomplished on that one.

The follow-up tweet by Adam Tuss of NBC is the best. “I’m with the news, dude!”

Motion planning might be the area of autonomous vehicle development that is most open to new discovery right now.

As part of the Udacity Self-Driving Car Nanodegree Program we teach a one-month module on Path Planning that covers environmental prediction, behavioral planning, and trajectory generation. These are the three key components of a planner.

Of these three components, trajectory generation is well understood, and environmental planning involves so much uncertainty that basic estimates are fine.

But behavioral planning remains an unsolved problem. How do you best determine which maneuver to make when cars and bikes and pedestrians are moving around?

One approach is to build a finite-state machine. This is in fact what we and our partners at Mercedes-Benz teach in the Nanodegree Program. Finite state machines work well for highway driving, which is structured. But it can break down in the chaos of urban driving; urban driving requires so many states.

So what other options are available?

Wikipedia’s entry on Motion Planning actually provides a pretty thorough high-level overview.

There are so many options! If you’re interested in becoming a path planner, it’s worth a quick read.