Ethics and AI

A wise AI should say “I don’t know”

Artificial Stupidity is being built every day.

Dany Majard
3 min readMay 11, 2022

--

One of the hallmark of wisdom is to recognize the vast amount of knowledge one doesn’t have. Not just the known unknown, but the size of the Unknown Unknowns. Or if look at the our current AIs under the prism of the Dunning-Kruger effect, every system we use will fall in the “low ability” category.

Photo by Michal Matlon on Unsplash

Yesterday, I attended a talk at #PyData London from one of the makers of scikit-Lego, Vincent D. Warmerdam. Beside being of great quality and inspirational, it framed some of my musings in a way that is both relevant to enhancing ethical use of AI and easy to communicate.

The first takeaway is this simple sentence:

Machine Learning algorithms are champions of interpolation, not extrapolation.

I think that were we to survey all data scientists and engineers that are involved in putting these algorithms in production, we’d be surprised at proportion that don’t have an active understanding of that fact, or potentially worse, decide to ignore it.

It is with great surprise that in the past, I found myself facing a misconception in conversations about modeling, where the behavior of said model wasn’t well understood outside of the boundary it was defined on. In technical terms the behavior of the algorithm far from the training manifold was misunderstood.

Photo by David Kovalenko on Unsplash

This is what he dubbed “Artificial Stupidity”. In their hurry to get more predictive models out, companies optimize accuracies, MAEs, precision, recall etc… And deploy models as is. In short, the AI is not immune to AS as it fails the basic tenant of intelligence: it doesn’t know when it doesn’t know.

When presented a new case, it will make a prediction regardless of how weird the case is. There is no standard API that returns a “I don’t know”. At best it will provide an answer with a very low confidence. Or we know that humans have a difficult time dealing with probabilities. We saw it first hand after the 45th president of the USA was elected. When…

--

--