Last fall, I went with some Udacity colleagues to a Silicon Valley Artificial Intelligence event that hosted a panel of speakers from startups in the world of self-driving cars.

One of the speakers was Austin Russell from a then-stealth company producing lidar. We asked his colleague about the name of the company and were told it was a secret.

Later in the event, the crowd started goading another attendee, George Hotz, into grilling the speakers. George rose to the occasion and asked Austin, “So, this all sounds great, but when is Luminar going to ship?”

So much for keeping the name secret.

This week, Luminar went public with all sorts of details about the company, their product, and the first-ever production run of sensors, starting this year.

Austin is a colorful and likeable character, and most of what I’ve read about Luminar quotes him stressing the superior performance of Luminar’s lidar sensors.

That’s awesome, but the main issue with lidar right now seems to be cost and volume, not performance.

Since Luminar has already talked about building 10,000 units, my question might be, what’s the cost?

Wired has a good article out about hacking autonomous vehicles, and about “autonomotive attack surfaces” in particular.

The article centers on Charlie Miller, who several years ago hacked a Jeep and took it over remotely while it was driving on the highway (don’t worry, it was a demonstration, not a malicious attack).

Miller talks about the interesting problem of securing vehicles from ride-sharing passengers. In a world where anybody can hail and hop into a self-driving Uber or Lyft, securing those vehicles from hackers who are physically in the car can be a huge challenge.

One example is a hacker who gets into a self-driving car, uses the OBD-II port to install software on the system, and then gets out. Later on, the hacker might use the latent software to take over the car when other riders are inside.

Gives a whole new meaning to “carjacking”.

Miller talks about the “attack surface” of vehicles, which encompasses any opening an attacker can use to hack a vehicle. A quick search for “automotive attack surface” led me to the graphic above, which comes from an academic research paper by Checkoway, et al.

“We discover that remote exploitation is feasible via a broad range of attack vectors (including mechanics tools, CD players, Bluetooth and cellular radio), and further, that wireless communications channels allow long distance vehicle control, location tracking, in-cabin audio exfiltration and theft.”

The further complication is that ridesharing companies are often layering their self-driving software and hardware on top of production automotive vehicles, built by somebody else. It creates a situation where the manufacturer may not design the car to be secure in the same ways that the after-market modifier (in this case, the ridesharing company) needs.

All sorts of interesting topics in this set of student posts, including some inside stories from the creator of ALVINN!

Emphatic Camera Calibration With OpenCV

Chris X Edwards

While trying to undistort his camera images, Chris walked into a store and asked to take a photo of their floor. Then things got really weird.

“I wrote a program that iterated through all possible grid sizes and looked at all images. Now I was finding grids. Ah ha! Turning to the documentation to figure out what exactly was going on, I noticed the function had a parameter, flags, which could be set to enable certain grid finding techniques. I set one of the flags and the grids I could detect changed quite a bit. Now I added to my program another inner loop to iterate through all the detection modes.”

Output Appearance Reliability Estimation

Dean Pomerleau

Dean Pomerleau, the creator of ALVINN, responded to Param Aggarwal with some cool stories about how ALVINN took advantage of confusion in the network to estimate how confident it was about its own steering ability:

“Using the OARE technique and a related one called Input Reconstruction Reliability Estimation (IRRE), ALVINN was able to localize itself (e.g. ‘I’ve reached the fork in the road!’), tell the human safety driver (me) when it needed help, arbitrate between networks trained on different road types, and even tell when there was crap on the windshield in front of the camera obstructing its view of the road.”

Cutting-edge (high-tech) career path.

Uki Dominique Lucas

Uki riffs here on all of the various projects he could be working on, how he chooses to spend his limited time, and where that intersects with career development.

“The next part of the career development is keeping up with the computer science basics. Honestly, it does not matter how much programming you do on daily basis, you will not pass the “whiteboard hazing” without any preparation. I lost countless of interviews with fine companies like Amazon, to what I thought was a “power trip” of some engineer without any social skills in a cookie factory — for years I was saying, “Why do I need that? I can make good money on my own”. Only later, I have read books and articles on interviewing and realized that the “whiteboard” is simply a thing they do and that people prepare for it for months.”

Vehichle detection using LIDAR: EDA, augmentation and feature extraction (Udacity/Didi challenge)

Vivek Yadav

Vivek goes into detail on his voxel-based approach for identifying cars based on the KITTI dataset for the Udacity-Didi Challenge. If you don’t know what a voxel is, read on:

“A voxel is a volume unit in space, similar to pixel in 2D images. I first constrained our space so x-dimension (front), y-dimension (L-R) varied between -30 and 30, and vertical dimension varied between -.1.5 and 1 m. I next constructed voxels of width and length .1 m and height 0.3125 m. I then computed maximum height in each voxel and used this value as the height of the point cloud in that voxel. This gave us a height map of 600X600X5 features. We specifically chose 5 height maps because Udacity’s data uses vlp-16 lidar and having more fine discretization can result in height slices without any points.”

Make sense of Kalman Filter

An Nguyen

What is a Kalman filter? Why do we use it? An gives a more intuitive explanation here than you will find on Wikipedia:

“Assume the car makes the lane change successfully to get in front of me, I still continuously observe the car and adjust my speed so my car can always stay in the safe zone. If the car goes slow, I predict the car will still be slow in the next seconds and I’ll stay at a slow speed behind it. However, if it suddenly goes fast, I can speed up a little bit (as long as under speed limit) and update my belief. What I did there is a continuous process of prediction and update.”

Last week my colleague Yousuf and I spoke at the Open Source Software for Decision Making Conference at Stanford.

It was a lot of fun! Thanks to Mykel Kochenderfer and Tim Wheeler for inviting us.

Yousuf and I spoke about building the Udacity open source self-driving car. If you’re interested in what Udacity and our students have done, check it out:

You can find all the presentations, including some pretty impressive academic work, at the conference website.

In a few weeks, I’ll be speaking at Car HMI USA, so please say hi if you’re there.

HMI stands for Human-Machine Interaction, and while I’m at the conference, I’m really excited to hear from UX and HMI engineers about what the future holds for riders of autonomous vehicles.

The Motley Fool predicts that self-driving cars will be great for Netflix and terrible for radio companies, which seems likely, but not particularly creative.

If we spend close to an hour per day in a self-driving car, how will we use that?

Maybe we’ll use it like we use our leisure time: 55% watching TV, 14% socializing, and 8% gaming.

I like to think we can do better. We could use self-driving cars to spend more time with our families — maybe we’ll drag our kids to work with us and have the self-driving car take them home. Maybe we’ll use that time to do housework like paying the bills or online grocery shopping.

Anything but more TV.

From OnMilwaukee.com comes this important April 1st story about the dangers of autonomous snowplowing:

With the 2016–17 winter nearly behind us, the tally is in: The DPW’s fleet of 200 self-driving snowplows destroyed 3,019 parked cars, killed or injured 29 stray cats, created 17,898 potholes/sinkholes and sent one elderly South Side man to the hospital after burying him in a snow drift.

“Yeah, I guess Milwaukee isn’t quite ready for this technology,” admits Kowolski. “We are considering ‘hiring’ monkeys to drive the plows next season.”

But consider the vendor:

In its pilot program, the City had considered using well-tested self-driving plows built by Google and Tesla, but instead opted to install hardware from RadioShack in its existing trucks.

They tested the equipment in Mountain View, of course.

Our guiding star in developing the Udacity Self-Driving Car Engineer Nanodegree Program is to help students get jobs working on autonomous vehicles.

To that end, we’ve built hiring partnerships with some of the most exciting employers in the world of autonomous vehicles.

Our newest hiring partners include giants like Fiat-Chrysler and Lockheed Martin, critical suppliers like Delphi and Velodyne and Dataspeed, as well as exciting startups like Renovo.

We work directly with recruiters at these companies to identify open positions that Udacity students might be interested in. Then we announce those positions in the Udacity Career Resource Center, and encourage students to apply.

Once students apply, we connect students to employers and guide students through the interview process.

Udacity has been focused on careers for several years, but this level of support is new to the Self-Driving Car Program, and we’re really excited about it. As Sebastian said recently, you can’t talk about education today without talking about jobs.