My colleague Jessica pointed me to this video taken from the dashcam of a Cruise self-driving car, as it travels through San Francisco.
Pretty neat stuff, especially when you consider that a computer is doing all the driving.
Baidu is competing with the Udacity Open-Source Self-Driving Car 😛
I jest, of course. Baidu is going to release a “free” operating system for self-driving cars, although the news leaves unclear whether this will be “free, as in lunch” or “free, as in speech”.
But Baidu’s goal quite clearly seems to be commercialization, whereas Udacity’s project is primarily educational.
The article in the MIT Technology Review supposes that Baidu is undertaking this as a catch-up move in a race with the Silicon Valley giants. Perhaps by getting more companies on its operating system, it can get more data.
If so, this really does look like a repeat of the mobile phone wars, and Baidu might need to be prepared to spend a lot of money to support its free operating system.
Here are posts from five Udacity Self-Driving Car students, sharing what they’ve learned about the program, their projects, Docker, and even how to hack your own car!
Andrew provides the most comprehensive review (in Spanish) of the Self-Driving Car Nanodegree Program that I have seen yet. He covers the forums, the mentors, the hiring partners, the classes, and all of the projects. It’s a very positive review, which is flattering:
“Clonacion de Comportamiento es uno de los proyectos con los que más he aprendido nunca y con los que he visto la potencia pero también la dificultad de diseñar y entrenar redes neuronales. Merece la pena hacer el nano-degree sólo por este proyecto.”
Gungor provides a concise tutorial for students looking to spin up Docker for the Self-Driving Car Nanodegree Program:
“I just realized there are still a lot of people having problem with Docker and starter kit for Self Driving Car Nanodegree program. In this post, I will give you a step by step guide.”
Muddassir covers some really cool data augmentation he performed on his Behavioral Cloning training set. By the end, his network is able to drive multiple laps around the crazy jungle track!
“I used a python generator in order to feed training batches to the network. The generator I designed also augments the data before generating a batch. I apply different types of augmentation to the data such as varying the brightness, color saturation, adding random shadows, translations, and horizontal flips to the images.”
Harish reflects on the challenges of the program, what was awesome about it, and what we need to improve:
“During this period, I have spent many all-nighters chugging down on redbull and coffee in an attempt to consume enough caffeine to stick it out and get through the various hurdles the course throws at you. I have on multiple ocassions spent days working on an idea only to get so frustrated with the results and progress to go ahead and scrap it entirely. Only to later realize that I had been right all along but made a tiny error in executing it!
In hindsight(*spoiler alert), it was worth all the trouble!”
Ariel is building a self-driving car from scratch, and has learned all sorts of practical lessons. This is a great list to read if you want to learn from somebody who is hacking a car:
“Can buses are good, dual can buses are great. They allow you to be able to separate key traffic in order to be able to ‘replace’ a factory module like the LKAS or the ACC. Get an Arduino due with dual can. ($70)”
In addition to being my boss, Sebastian Thrun is a legend in the field of autonomous vehicles.
We’ll be filming a Q&A with him this week about self-driving cars and the Udacity Self-Driving Car Nanodegree Program.
This is an awesome opportunity to pick the brain of the winner of the DARPA Grand Challenge, the founder of the Google Self-Driving Car Program, and the driving force between the Udacity Self-Driving Car Nanodegree Program!
Submit questions here: https://goo.gl/forms/I8Ik3n0g1EZjFneG2
Video posts on Friday!
Self-driving trucks have been a concept in mining operations for many years, because of the well-structured, private roads and dependable routes. Dump trucks basically drive the same route over and over, which makes them an ideal target for autonomous technology.
Diginomica has a good rundown of Caterpillar’s latest work on self-driving mining vehicles in Australia.
“Fortescue Mining Group’s Solomon Hub comprises the Firetail and Kings Valley iron ore mines in the Pilbara region of Australia’s North West which together have a production capacity of over 70 mega tonnes each year. When the project was scoped in 2010, the initial feasibility study called for 75 manned trucks but in July 2011 FMG ordered 12 autonomous 793F vehicles as a pilot. Now with the mines up and running, FMG operates 54 driverless dumpsters which alone results in a $100 million capital saving on twenty trucks.”
There’s also this:
“By replacing the drivers, Westrac and Caterpillar also found they can make further cost savings by eliminating some comfort and safety features on the trucks with weight savings of up to four tonnes per vehicle.”
I’ve had a few people come to me recently asking about how to get up and running in this industry. I’m not that knowledgeable about mining, but the fact that people are asking makes me think this isn’t yet a solved problem.
The California DMV reported this week that it had granted Apple a license to test three autonomous vehicles, and the Internet went wild.
Apple’s autonomous vehicle work has been an open secret for a few years, so I’m skeptical that this announcement will change much or lead the way to a more meaningful understanding of what Apple is working on.
Beyond just Apple, though, this has me back to thinking about the tradeoff between stealth and transparency.
Transparency in product development seems like an aggressive approach. By opening up about what they’re doing, companies hopes to attract the best talent, the best partners, the earliest and best customers.
Conversely, a stealth approach seems cautious. Companies developing products in secret seem nervous with competitors and the press. Competitors might steal key elements of a developing product, while the press might pressure a company to alter its schedule or pricing or go-to-market strategy.
All else equal, it seems more fun to be aggressive than cautious, but of course all else is never equal. A company is in a good position and has a lot to lose has a lot more reason to be cautious and stealthy. A company in poor position, with nothing to lose, is likely to act aggressively and transparently.
What gets interesting is when companies like Apple, which seems to have nothing to lose in the automotive industry, approaches product development secretively, perhaps because of its culture.
When GM announced its $1 billion acquisition of Cruise Automation, I was skeptical. Cruise was a San Francisco software startup; GM is a venerable American automotive company with a bureaucracy that I presume is optimized for rolling vehicles off the assembly line. It was not obvious that this was a match made in heaven.
But the acquisition is now over a year old and it seems to be working out really well.
The most recent news is that GM will be adding 1100 jobs to its San Francisco office over the next five years.
Self-driving Chevy Bolts are maybe not quite ubiquitous in San Francisco, but they’re normal enough to make me guess that GM is probably the number two tester of Level 4 autonomous vehicles in the Bay Area (and the world?), after Google.
Last fall, I went with some Udacity colleagues to a Silicon Valley Artificial Intelligence event that hosted a panel of speakers from startups in the world of self-driving cars.
One of the speakers was Austin Russell from a then-stealth company producing lidar. We asked his colleague about the name of the company and were told it was a secret.
Later in the event, the crowd started goading another attendee, George Hotz, into grilling the speakers. George rose to the occasion and asked Austin, “So, this all sounds great, but when is Luminar going to ship?”
So much for keeping the name secret.
This week, Luminar went public with all sorts of details about the company, their product, and the first-ever production run of sensors, starting this year.
Austin is a colorful and likeable character, and most of what I’ve read about Luminar quotes him stressing the superior performance of Luminar’s lidar sensors.
That’s awesome, but the main issue with lidar right now seems to be cost and volume, not performance.
Since Luminar has already talked about building 10,000 units, my question might be, what’s the cost?
Wired has a good article out about hacking autonomous vehicles, and about “autonomotive attack surfaces” in particular.
The article centers on Charlie Miller, who several years ago hacked a Jeep and took it over remotely while it was driving on the highway (don’t worry, it was a demonstration, not a malicious attack).
Miller talks about the interesting problem of securing vehicles from ride-sharing passengers. In a world where anybody can hail and hop into a self-driving Uber or Lyft, securing those vehicles from hackers who are physically in the car can be a huge challenge.
One example is a hacker who gets into a self-driving car, uses the OBD-II port to install software on the system, and then gets out. Later on, the hacker might use the latent software to take over the car when other riders are inside.
Gives a whole new meaning to “carjacking”.
Miller talks about the “attack surface” of vehicles, which encompasses any opening an attacker can use to hack a vehicle. A quick search for “automotive attack surface” led me to the graphic above, which comes from an academic research paper by Checkoway, et al.
“We discover that remote exploitation is feasible via a broad range of attack vectors (including mechanics tools, CD players, Bluetooth and cellular radio), and further, that wireless communications channels allow long distance vehicle control, location tracking, in-cabin audio exfiltration and theft.”
The further complication is that ridesharing companies are often layering their self-driving software and hardware on top of production automotive vehicles, built by somebody else. It creates a situation where the manufacturer may not design the car to be secure in the same ways that the after-market modifier (in this case, the ridesharing company) needs.
All sorts of interesting topics in this set of student posts, including some inside stories from the creator of ALVINN!
While trying to undistort his camera images, Chris walked into a store and asked to take a photo of their floor. Then things got really weird.
“I wrote a program that iterated through all possible grid sizes and looked at all images. Now I was finding grids. Ah ha! Turning to the documentation to figure out what exactly was going on, I noticed the function had a parameter,
flags, which could be set to enable certain grid finding techniques. I set one of the flags and the grids I could detect changed quite a bit. Now I added to my program another inner loop to iterate through all the detection modes.”
Dean Pomerleau, the creator of ALVINN, responded to Param Aggarwal with some cool stories about how ALVINN took advantage of confusion in the network to estimate how confident it was about its own steering ability:
“Using the OARE technique and a related one called Input Reconstruction Reliability Estimation (IRRE), ALVINN was able to localize itself (e.g. ‘I’ve reached the fork in the road!’), tell the human safety driver (me) when it needed help, arbitrate between networks trained on different road types, and even tell when there was crap on the windshield in front of the camera obstructing its view of the road.”
Uki riffs here on all of the various projects he could be working on, how he chooses to spend his limited time, and where that intersects with career development.
“The next part of the career development is keeping up with the computer science basics. Honestly, it does not matter how much programming you do on daily basis, you will not pass the “whiteboard hazing” without any preparation. I lost countless of interviews with fine companies like Amazon, to what I thought was a “power trip” of some engineer without any social skills in a cookie factory — for years I was saying, “Why do I need that? I can make good money on my own”. Only later, I have read books and articles on interviewing and realized that the “whiteboard” is simply a thing they do and that people prepare for it for months.”
Vivek goes into detail on his voxel-based approach for identifying cars based on the KITTI dataset for the Udacity-Didi Challenge. If you don’t know what a voxel is, read on:
“A voxel is a volume unit in space, similar to pixel in 2D images. I first constrained our space so x-dimension (front), y-dimension (L-R) varied between -30 and 30, and vertical dimension varied between -.1.5 and 1 m. I next constructed voxels of width and length .1 m and height 0.3125 m. I then computed maximum height in each voxel and used this value as the height of the point cloud in that voxel. This gave us a height map of 600X600X5 features. We specifically chose 5 height maps because Udacity’s data uses vlp-16 lidar and having more fine discretization can result in height slices without any points.”
What is a Kalman filter? Why do we use it? An gives a more intuitive explanation here than you will find on Wikipedia:
“Assume the car makes the lane change successfully to get in front of me, I still continuously observe the car and adjust my speed so my car can always stay in the safe zone. If the car goes slow, I predict the car will still be slow in the next seconds and I’ll stay at a slow speed behind it. However, if it suddenly goes fast, I can speed up a little bit (as long as under speed limit) and update my belief. What I did there is a continuous process of prediction and update.”
David Silver 