Reducing risk in intelligent applications

In artificial intelligence (AI), unexpected outcomes leading to ethics breaches are a big risk factor. For example, is the AI biased and giving skewed or inaccurate search results, recommendations or predictions?

Part of a standard checklist for reducing risk in AI applications is to build AI forensics tools, use those tools to profile the AI algorithm, use the profile to anticipate behavior, and discuss the anticipated behavior with a diverse risk mitigation team.

AI forensics analyzes an AI model’s tendencies based on its output. You need a good log of input and corresponding output. You also need tools that can discover the most influential factors in how the model makes decisions. Using the tools, you can build a profile: the decisions the AI makes, the factors that it considers, and the weight of each factor.

By building a profile, you gain insight into the model’s behavior and tendencies. Profiling an algorithm doesn’t sacrifice performance. It’s not necessary to alter the function of the underlying algorithm. Nor is it necessary for the inner workings of the underlying algorithm to be fully transparent and explainable.

Some of the most useful profiling and analysis tools will, themselves, be based on machine learning. It turns out that machine learning is one of the best ways to protect against the unintended consequences of machine learning.

The final and most important step in the AI-ethics checklist is to discuss the results of forensics with a diverse group of people. Decide whether the potential security risk of an unexpected outcome is acceptable or whether changes need to be made before allowing the algorithm to continue in production.

Continue reading the position paper, Rethink risk and enterprise security in a digital world.