Urmson’s Carnegie Mellon team won the DARPA Urban Challenge in 2006. From there he joined my current boss, Sebastian Thrun, at the Google Self-Driving Car Project, back when Sebastian was running that. Urmson led the Google Self-Driving Car Project himself after Sebastian departed to launch Udacity. Most recently, Urmson left Google to found Aurora Innovation, his own self-driving car startup.
The interview is an hour long and Swisher lives up to her reputation as the best of tech journalists. They cover the early days of the DARPA challenges, how Urmson got to Google, why he left Google, what Aurora is doing, how the automotive industry and tech industry will partner and compete, when we will see fully autonomous vehicles on the road, and the moral responsibility autonomous vehicle engineers have to the drivers they might put out of work.
The Drive.ai partnership sounds especially promising, since its written with the tone that this effort is going to start soon, maybe before the end of the year.
There are self-driving cars being tested on public streets in cities around the world: Pittsburgh, Phoenix, San Francisco, Singapore, Austin. Some of those cities even have self-driving cars testing with a limited set of pre-screened passengers.
But right now, as far as I know, there is only one city where any civilian off the street can show up and hail a self-driving vehicle. That would be Pittsburgh, where Uber has opened their fleet to public use (with safety drivers).
Drive.ai testing in the Bay Area would bring that number to two.
Wired has a nice rundown of the key parts of the bill. The major feature seems to be moving autonomous vehicle regulation from a patchwork of 50 different state regimes over to a single set of federal regulations. You’ll no longer have to worry about getting arrested when you take your self-driving car across state lines.
The law was pushed by a consortium of interested companies — Ford, Waymo, Lyft, Uber, and Volvo. To be honest, on self-driving cars, I trust my old employers at Ford more than I trust any department of motor vehicles.
The “unanimous” passage of the bill makes me a little nervous — as if not enough people were paying attention. Surely there is something in there somebody objects to. Personally, I kind of like the “laboratories of democracy” aspect of state regulation.
But since it seems destined to become law anyway, we should all hope this bill speeds the development of safe autonomous vehicles that prevent many of the 35,000 annual motor vehicle fatalities in the US.
At Udacity, where I work, we have a self-driving car. Her name is Carla.
Carla’s technology is divided into four subsystems: sensors, perception, planning, and control.
Sensors
Carla’s sensor subsystem encompasses the physical hardware that gathers data about the environment.
For example, Carla has cameras mounted behind the top of the windshield. There are usually between one and three cameras lined up in a row, although we can add or remove cameras as our needs change.
Carla also has a single front-facing radar, embedded in the bumper, and one 360-degree lidar, mounted on the roof.
This is what a lidar sensor looks like, and this is what lidar data looks like. It’s a point cloud.
Sometimes Carla will utilize other sensors, too, like GPS, IMU, and ultrasonic sensors.
Data from these sensors flows into various components of the perception subsystem.
Perception
Carla’s perception subsystem translates raw sensor data into meaningful intelligence about her environment. The components of the perception subsystem can be grouped into two blocks: detection and localization.
The detection block uses sensor information to detect objects outside the vehicle. These detection components include traffic light detection and classification, object detection and tracking, and free space detection.
The localization block determines where the vehicle is in the world. This is harder than it sounds. GPS can help, but GPS is only accurate to within 1–2 meters. For a car, a 1–2 meter error range is unacceptably large. A car that thinks it’s in the center of a lane could be off by 1–2 meters and really be on the sidewalk, running into things. We need to do a lot better than the 1–2 meters of accuracy that GPS provides.
Fortunately, we can localize Carla to within 10 centimeters or less, using a combination of high-definition maps, Carla’s own lidar sensor, and sophisticated mathematical algorithms. Carla’s lidar scans the environment, compares what it sees to a high-definition map, and then determines a precise location.
Carla localizes herself by figuring out where she is on a high-definition map.
The components of the perception subsystem route their output to the planning subsystem.
Planning
Carla has a straightforward planning subsystem. The planner builds a series of waypoints for Carla to follow. These waypoints are just spots on the road that Carla needs to drive over.
Each waypoint has a specific location and associated target velocity that Carla should match when she passes through that waypoint.
Carla’s planner uses the perception data to predict the movements of other vehicles on the road and update the waypoints accordingly.
For example, if the planning subsystem were to predict that the vehicle in front of Carla would be slowing down, then Carla’s own planner would likely decide to decelerate.
The final step in the planning process would be for the trajectory generation component to build new waypoints that have slower target velocities, since in this example Carla would be slowing down as she passes through the upcoming waypoints.
Similar calculations affect how the planning subsystem treats traffic lights and traffic signs.
Once the planner has generated a trajectory of new waypoints, this trajectory is passed to the final subsystem, the control subsystem.
Control
The control subsystem actuates the vehicle by sending acceleration, brake, and steering messages. Some of these messages are purely electronic, and others have a physical manifestation. For example, if you ride in Carla, you will actually see the steering wheel turn itself.
The control subsystem takes as input the list of waypoints and target velocities generated by the planning subsystem. Then the control subsystem passes these waypoints and velocities to an algorithm, which calculates just how much to steer, accelerate, or brake, in order to hit the target trajectory.
There are many different algorithms that the control subsystem can use to map waypoints to steering and throttle commands. These different algorithms are called, appropriately enough, controllers.
Carla uses a fairly simple proportional-integral-derivative (PID) controller, but more sophisticated controllers are possible.
A Self-Driving Car
That’s how Carla works!
First, the sensor subsystem collects data from Carla’s cameras, radar, and lidar. The perception subsystem uses that sensor data to detect objects in the world and localize the vehicle within its environment. Next, the planning subsystem uses that environmental data to build a trajectory for Carla to follow. Finally, the control system turns the steering wheel and fires the accelerator and brake in order to move the car along that trajectory.
We’re very proud of Carla. She’s driven from Mountain View to San Francisco, and done lots of driving on our test track.
The most exciting thing about Carla is that every Udacity student gets to load their code onto her computer at the end of the Nanodegree Program and see how well she drives on our test track.
The list, which I believe is in chronological order, starts with big automotive companies that everybody recognizes, and then includes many smaller, anonymous startups toward the end. Udacity is pretty much in the middle 🙂
An interesting exercise would be to dive into this list and figure out where each company is with their self-driving car development efforts.
David Silver 