SAE International Bridging the Gap between Open Loop Tests and Statistical Validation for Highly Automated Driving 2017-01-1403

Highly automated driving (HAD) is under rapid development and will be available for customers within the next years. However the evidence that HAD is at least as safe as human driving has still not been produced. The challenge is to drive hundreds of millions of test kilometers without incidents to show that statistically HAD is significantly safer. One approach is to let a HAD function run in parallel with human drivers in customer cars to utilize a fraction of the billions of kilometers driven every year. To guarantee safety, the function under test (FUT) has access to sensors but its output is not executed, which results in an open loop problem. To overcome this shortcoming, the proposed method consists of four steps to close the loop for the FUT. First, sensor data from real driving scenarios is fused in a world model and enhanced by incorporating future time steps into original measurements. Second, recorded time-shifted data is used to identify intentions of each driver of the surrounding traffic. Third, the real scene is used as start scene for simulation. Reactions of surrounding traffic to the FUT output are simulated using identified intentions from step two. Fourth, simulation output is evaluated for erroneous behavior of the FUT to give developers input for further improvement and provide statistics about the safely driven distance. Main contribution is the first complete method for passive HAD safety assessment with a solution to the open loop problem. Results are presented for driver intention identification and criticality assessment.