For the past few weeks, an inconspicuous white Ford has traversed New York City avenues, showcasing the type of tech that may one day take over more tasks from human drivers.
Unbeknown to motorists and pedestrians, the vehicle intuitively slowed to avoid collisions with aggressive drivers. It yielded to jaywalkers weaving between traffic and swerved around delivery trucks parked in the middle of busy streets. It sailed across bridges leading to Manhattan’s bustling city center at various times of the day. And all of that happened without a driver intervening.
The ongoing project is part of a test that Mobileye announced last week. The company, best known for its advanced driver-assistance systems, was the first to receive a special permit from the state, allowing backup drivers in autonomous-enabled vehicles to keep their hands off the steering wheel in highly automated cars on public roads.
It’s the type of robotic-vehicle demonstration that has unfolded in Arizona and California for years with companies like Waymo and Cruise as well as in Paris and Tokyo.
But Mobileye is the first to road test in the Big Apple, home to one of the most challenging driving environments in the world.
“If we want to build something that will scale, we need to be able to drive in challenging places and almost everywhere,” Amnon Shashua, CEO of Mobileye, said at a media event Tuesday in Times Square.
Mobileye, which Intel snapped up for $15.3 billion in 2017, showcased an immersive version of the driving experience last week at an event in Washington, D.C. last week the company revealed a 40-minute, unedited YouTube video highlighting its advance into new territory.
Mobileye received a special permit from New York state, allowing manufacturers of “autonomous vehicle technology” to test on public streets. The permit requires that drivers be present in the vehicle but allows them to keep their hands off the steering wheel yet “be prepared to take control when required to . . . operate the vehicle safely and lawfully.” So far, it’s the only company wielding permission for such tasks in the state. It’s unclear whether others may have applied. The state hasn’t responded to a request for comment.
The law permitting longer-term tests expires in 2023, so other autonomous vehicle firms have time to apply.
MobileEye used a camera-based vision system to give the car a sense of what’s around it. The car’s decision-making capabilities hinge on the company’s Responsibility Sensitive Safety (RSS) model, which is basically a mathematical model for how a person would drive in common scenarios.
The operating tests kick-started last month, running during daylight and nighttime hours and revealing some of the city-specific situations that self-driving systems need to adapt to. One of those is the striking amount of light emitted from massive billboards and other signs peppering Midtown Manhattan.
“Though Paris gets the ‘city of lights’ moniker, Manhattan is electrified at night. The visual noise and light pollution (are) daunting to an AV’s sensing system,” Mobileye said in a release. The firm’s vehicles could handle the lights, “with only a bit of algorithm tuning,” the company says.
In one released video, the camera-boosted car is shown cruising through Times Square and in front of Radio City Music Hall at night without drawing attention from pedestrians or other motorists. In a 360-degree video shown during a media briefing in Washington D.C., Mobileye’s vehicle is shown responding to a car up ahead that stopped to let a person out. The automated vehicle analyzes upcoming traffic, approaches the car parked in the road and smoothly swerves around it.
Mobileye released unedited footage, too, posting the lengthy video on YouTube showing the system in action. Viewers can see that the safety driver’s hands sit in his lap while the vehicle’s steering turns left and right on its own as if someone were manually controlling it. External camera footage shows how the car interprets crosswalks, road signs and other vehicles.
The test revealed what many already know about New York driving. Motorists tend to be more assertive than in other areas. Roads are dense with cabs, food carts and horse-drawn carriages. Construction is always happening. And several micro rides like scooters, bikes and skateboards add to the mayhem. Mobileye says its software is learning from the experience and is tuning machines to respond to jaywalking, double-parked cars and the like.
Mobileye wants to add another vehicle to the New York City fleet in the fall.
Separately, the company is working on a lidar and radar-based system that’s strong enough to power autonomous vehicles. It plans to fuse that with the camera system used in New York for an AV tech “that is at least three orders of magnitude safer than humans.”
Still, it’s unclear how much the cars of tomorrow will achieve or how widespread the underlying tech will be. Things look promising in some regard: Last year, Waymo launched a “fully driverless” taxi business in the suburbs of Phoenix. But the service area is limited compared to operating in places where streets aren’t as wide, precipitation is more plentiful and pedestrians present a greater threat.
Also, prices would have to come down tremendously for fully automated systems to be viable for personal car ownership. Mobileye calculates those self-driving systems might add $15,000 to $40,000 to the cost of consumer vehicles. But conquering New York would edge the firm closer toward widespread adoption, which it projects will start happening in 2025.
“Driving in complex urban areas such as New York City is a crucial step in vetting the capabilities of an autonomous system and moving the industry closer to commercial readiness,” Mobileye says.