Elon Musk says lidar is 'a fool’s errand,' and others are starting to agree

Tesla presented its latest and greatest vehicle autonomy features at its recent Autonomy Investor Day. The brain of the system is the Tesla Full Self-Driving (FSD) computer, which is now included in all Teslas being produced. The Autopilot hardware suite includes 8 cameras, 12 ultrasonic sensors, radar, GPS, an inertial measurement unit, and sensors that measure the angle of the steering wheel and accelerator pedal.

Above: Tesla's camera and Waymo's lidar (Images: Tesla / Waymo)

One feature it does not have is lidar, a technology that uses pulsed laser light to measure distance to a target, and that is favored by most other companies working on autonomous driving (Uber, Waymo, Cruise). In his Autonomy Investor Day presentation, the ever-iconoclastic Musk dismissed lidar in terms reminiscent of those he has used to describe hydrogen fuel cells. Lidar is “a fool’s errand,” he said. It’s “expensive” and “unnecessary,” and “anyone relying on lidar is doomed. It’s like having a whole bunch of expensive appendices. Like, one appendix is bad, well now you have a whole bunch of them. It’s ridiculous, you’ll see.”

As TechCrunch reports, proponents of lidar cite its ability to see through adverse conditions such as rain, fog or dust better than cameras. The downsides are that it’s expensive, and uses a lot of power, and that’s why Tesla’s system depends on cameras. Andrej Karpathy, Senior Director of AI, explained that visual recognition gives a better picture of the real world - he said lidar systems have trouble distinguishing things like the difference between a plastic bag and a rubber tire. “In that sense, lidar is really a shortcut,” Karpathy said. “It sidesteps the fundamental problem, the important problem of visual recognition, that is necessary for autonomy. It gives a false sense of progress, and is ultimately a crutch.”

Together with cameras, Tesla is relying on the vast neural network of real-world driving information recorded by the thousands of Autopilot-equipped Tesla vehicles on the road. Using various AI techniques, Tesla is teaching its system to recognize and react to the vast variety of situations that might be encountered in the wild.

“Everyone’s training the network all the time,” Musk said. “Whether Autopilot is on or off, the network is being trained. Every mile that’s driven for the car that’s [equipped with Autopilot Hardware version 2 or above] is training the network.”

Above: Anthony Levandowski, former Google/Waymo engineer, now backtracks on lidar and says, "Elon is right" (Youtube: TechCrunch)

Some autonomy experts agree with Musk. As Gizmodo reports, Cornell researchers argue that cameras can rival lidar if they’re mounted properly. Lidar systems are designed to provide a 3D picture of a vehicle’s surroundings and the road ahead, which is why they’re often mounted atop a vehicle for the best vantage point. However, in a paper that will be presented at the upcoming Conference on Computer Vision and Pattern Recognition, the Cornell team explains that a pair of inexpensive cameras mounted behind a vehicle’s windshield can produce stereoscopic images that can be converted to 3D data almost as precise as that generated by lidar, at a fraction of the cost.

Mashable reports that British startup Wayve is a member of the no-lidar-needed camp. The company says it doesn’t need multiple data sources - just a GPS system, cameras and a powerful computer are all it requires to teach cars to drive as well as humans.

Other players do not agree. A Chinese autonomous driving firm called AutoX deployed a camera-based self-driving system in 2017, but has now added lidar sensors to its vehicles for redundancy and extra input.

Musk isn’t totally opposed to lidar - SpaceX uses it in some applications. For vehicle autonomy, however, he believes Tesla’s sensor suite and trove of real-world driving data will be quite sufficient. 

===

Written by: Charles Morris; Source: TechCrunch, Gizmodo, Mashable