Safety first in autonomous systems

Autonomous systems are a major challenge for the automotive industry. In addition to the required functions (most of which are already implemented), it is necessary to guarantee the safety of the systems under all (in)conceivable circumstances, in order to gain broad social acceptance of autonomous systems by reducing the number of accidents.

The new SOTIF standard has defined extensive regulations for Level 1 and 2 of the 6 automation levels. But additional regulations are needed for Level 3 and 4. Under the overall control of Daimler, automotive manufacturers and suppliers conducted a study and published the results in the whitepaper Safety First for Automated Driving (SaFAD).

We interviewed Marc Ruppert, Dipl.-Inform., to learn more about the additional requirements needed for development of the systems. He is a consultant at F+S Fleckner and Simon Informationstechnik GmbH.

Mr. Ruppert, if the functional problems have been solved, why are autonomous systems not safe?

Marc Ruppert, Dipl.-Inf.: The system has an algorithm that works - that means functional. Let us look at the example of traffic sign detection: We have algorithms that detect a sign and interpret its meaning. The functional challenge is to detect all signs around the world, including older and obsolete signs, as well as those warning of kangaroos in Australia or elks in Sweden. Including the meaning, i.e. the specific danger and the required action on the part of the driver (or vehicle).

Safe means however that every sign will always be detected, at least with the same reliability as that of a human driver. Also in case it is partially covered by bushes, bleached or weathered, or even a snow-covered stop sign.

But isn’t detection of a weathered sign also a functional challenge?

Ruppert: In a way, yes. If I am familiar with the requirements, I can expand the algorithm accordingly. But depending on the weather, visibility and road conditions, other road users, and deviations in the production process, the system reaches its limits at some point and either fails to detect a sign or misinterprets it. And that can lead to dangerous situations.

Optimization of functionality is only one possibility for flawless behaviour. There will always be limitations, and then other solutions are required to prevent accidents.

And what solutions has the study developed?

Ruppert: First, it was necessary to determine the scope. This was achieved by defining Twelve Principles that must be fulfilled by an autonomous system. These 12 principles can be regarded as 12 safety goals that must be ensured by every autonomous system.

On the basis of these principles SaFAD determines which additional development steps are needed and how to implement them – in a nutshell “Safety by Design”.

A third focus is detailed verification and validation of systems for proving their safety and, especially during the early stages of development, for determining additional specific requirements. This requires a combination of simulations, test driving on the testing grounds and in the real world. Verification confirms that all 12 principles are fulfilled.

Mr. Ruppert, could you please explain the 12 principles in more detail?

Ruppert: Gladly. For better clarity I will divide the principles into three areas:

The first is responsible for ensuring that the system always functions safely. This requires a clear definition of the area of use (driving task, route, weather, etc.), behaviour that is always comprehensible to other road users, and fallback levels for problematic situations and mistakes. This is described in the following four principles:

• Operational Design Domain (ODD)
• Behaviour in traffic
• Safe operation
• Safe layer

The second area ensures responsibility between the drive and vehicle, i.e. the driver’s handing over responsibility to the autonomous system or vice versa, but also monitoring of the driver’s fitness and his potential incorrect reactions after assuming responsibility. This requires compliance with the following four principles:

• Vehicle operator-initiated handover
• User responsibility
• Vehicle initiated handover
• Interdependency

The third area is made up of single principles that are necessary as support measures. They are

• Security
• Passive safety
• Data recording
• Safety assessment

Seems to make sense. But what exactly is meant by “Safety by Design”?

Ruppert: Just as the quality of a system is attained not only by testing, it is also necessary to integrate safety in the architecture from the very beginning. The study shows how this can be achieved.

To this end, on the basis of the 12 principles SaFAD derives 13 capabilities that must be ensured by an autonomous system in order to fulfil the principles. Seven capabilities constitute the actual rules for autonomous driving, in addition to an initial monitoring level. They are fail-safe. The other six capabilities detect problems, such as critical weather conditions or system errors. They constitute a fallback level as fail-degraded capabilities that diminish an error. In case of a problem, they prevent failure of the system and therefore an impending accident. They always put the vehicle into a minimal risk condition (MRC).

And how is that related to design?

Ruppert: That is the first step. In a second step, the 12 capabilities are assigned elements, such as sensors, algorithms and actuators at a functional level that implement the capabilities. These elements are then ordered in a generic architecture and interfaces between them are defined. This architecture also follows the Sense – Plan – Act design paradigm.

The result is a functional generic architecture, which can be implemented for all autonomous systems. By using the single steps described above I can transfer my specific requirements to the architecture, which – if implemented correctly – will result in a safe system.

“If correctly implemented” does not exactly inspire confidence!

Ruppert: Of course, errors can occur at any time and place. To detect and rectify these errors there is a third area: Verification and Validation.

Similar to design, five test challenges of the autonomous system are first identified, which did not exist previously, or existed only to a limited degree.

To derive this test strategy the team uses the 5W2H questions (who, what, where, when, why, how, how well). These questions are answered for each challenge, to create a test strategy that “only” has to be implemented.

However, the established test methods retain their validity, and all past tests must still be carried out at all levels.

Mr. Ruppert, that all sounds very theoretical.

Ruppert: Yes and no. The SaFAD whitepaper, of course, explains everything in much more detail than I can here. In addition, the principles are illustrated based on four development examples that can be used for orientation (traffic jam pilot and highway pilot as level 3, city pilot and parking pilot as level 4 examples).

Similar to the E-GAS monitoring concept developed by the VDA, which became an unofficial standard for the development of safety-related E/E systems in vehicles until it was replaced by ISO 26262, we have a template here that provides orientation. It contains many easily overlooked details and pitfalls, together with suggested solutions. Using the whitepaper as a basis allows more precise analysis and planning in the development and safety of autonomous systems.

Mr. Ruppert, could you please explain that based on the example of the traffic sign?

Ruppert: Gladly. As already mentioned, the weather may make it impossible to detect signs. By extending the functionality, it is possible to detect a stop sign even if is covered with snow. But in difficult visibility conditions such as fog or snowstorms, which are beyond the operation design domain (ODD), when a defined restriction is reached the system must prompt the driver to take control of the vehicle, before the signs are overlooked or misinterpreted.

The limits can be determined first in computer simulations, and then in climatic test chambers. On the test grounds, one can set up suitable (poorly visible) signs and simulate weather conditions to a certain extent. If possible, test drives should be planned according to the weather. Driving in a real environment, especially in critical weather conditions where signs have already been incorrectly detected, provide additional empirical information.

So one only has to follow the steps in the whitepaper like a recipe in a cookbook?

Ruppert: It’s not that simple. The requirements are still very abstract. Similar to application of a standard such as Automotive SPICE or ISO 26262, the whitepaper can serve as basis for deriving specific tasks for a given system and environment, and also to adapt and supplement existing processes. But here again there are pitfalls that can be costly. Our customers benefit from our experience in dealing with such requirements and implementing them as specific, pragmatic process steps in a real technological environment.

Then I hope that many customers take advantage of your services and, with your help, develop safe autonomous systems that no one will be afraid to use.