AI & Data Strategy
We offer companies new market opportunities through the use of cutting-edge technologies and AI-based data analysis processes.
I decided to write this article to bring order to the reflections and deductions that I have developed in recent months regarding the concept of bias. The need to bring order was born from the confusion that arose in me when I noticed that the term bias had become part of everyday language, sometimes taking on ambiguous meanings for me.
By writing this article I do not harbor any certainty or absolute truth. Indeed, I write it to note down what I think regarding bias and I do so, as always, publicly because I trust that it can turn into a useful opportunity for discussion with those who have the patience to read my reflections.
Before starting this reading, relax and repeat after me:
Without explainability, artificial intelligence is useless and dangerous
In this article I talk about bias and, as always, I prefer to start from the definition of bias to which I refer in my reflections.
Biases are the manifestation of systematic distortions of judgment resulting from cognitive heuristics of which we have lost control, i.e. mental shortcuts that we use to simplify the decision-making process which have been pushed to such a level of trivialization of reality that we lose contact with reality. same reality from which they are generated, negatively impacting the decision-making model adopted by any biological or artificial intelligent agent (Kahneman & Egan, 2011).
Biases can influence any decision model to make it ineffective. Even where we think we have prepared a decision-making model based on bias-free heuristics, Tversky's studies demonstrate that these take on a fundamental role in the analysis of reality, producing consequences that are not necessarily detectable or detectable in the short term.
The awareness of the structural and structuring role assumed by biases in decision-making processes based on heuristics paradoxically makes them a "false problem" of the processes themselves. A heuristic model based on biases that are admissible and functional for the purpose of the model does not make the model itself free of bias. A decision-making process in which there are no dangerous, macroscopic distortions of reality leads me to think that the biases present in the model are invisible to our analysis but effective in influencing the decision-making process. A well-orchestrated constellation of biases assumes in the decision-making process the same mechanics as the small weights with which the wheels of our cars are balanced: at a certain threshold, they demonstrate a powerful conditioning of the system. The existence of this conditioning process was witnessed by Alexander Nix, CEO of Cambridge Analytica, in his speech “From Mad Men to Math Men” presented at the Online Marketing Rockstars conference held in Hamburg in 2017. The potentially cataclysmic force of this conditioning was tested through the psychometric conditioning that Cambridge Analytica implemented during the 2010 general elections in Trinidad and Tobago in favor of the United National Congress (UNC) through the “Do So” campaign.
The analysis of a decision-making model must therefore not focus on the simple identification of the presence of obvious biases such as racial or gender biases but must be able to understand how much the method of administration of the individual results of the analysis carried out is capable of generating strategic micro-conditioning similar to that produced by AlphaGo with the “U-37” move.
The awareness that biases are not the real problem of a decision-making model is also given to me by the fact that biases are not the cause of an anomaly but only and always a mere consequence of the latter. To be clear, a bias is to an anomaly what the flu is to an infection: it's just a symptom.
Stating that a decision-making system is affected by bias is obvious to me as the entire decision-making process is almost always based on heuristics. At the same time, talking about bias is also an admission of inadequacy. The inadequacy is determined by the fact that treating a bias is the equivalent of a symptomatic treatment caused by the inability to understand the origin of the anomaly and/or to correct the anomaly itself.
Artificial intelligence systems are not free from bias because these systems also operate through clustering processes or abstraction processes that are based on biases that are admissible and functional to the analysis.
In this article I explain with a "step-by-step" approach the logical path that led me to my conclusions, summarized already in this introduction to share with those who read me the awareness that mitigating the risk generated by cognitive dynamics that manifest in the form of bias does not exclude the presence of bias whose impact is equally serious but not immediately detectable by our ability to evaluate.
I have always been fascinated by the mechanisms by which the mind analyzes the world and human relationships. I devoured the TV series “Brain Games” by National Geographic and the essay written by Sergio Della Sala and Michaela Dewar entitled “Never trust the mind” which, through experiments bordering on magic, show us how unknown it is, even today, the human brain and how much the mechanisms that govern it in the daily effort of analysis and adaptation are linked to errors, to illusions of thought, to inconsistencies of mental processes, to imperfections of memory that lead to the development of real decision-making shortcuts.
Decision-making shortcuts are the strategy our brain uses to save energy. They manifest themselves every time we are faced with challenges, problems and decisions to make for which we prefer to adopt a "heuristic" approach, that is, an approach that makes use of generalizations, empirical rules and assumptions.
The heuristic approach is a decision-making model that is based on a set of strategies, techniques and creative processes that help us find solutions more quickly and easily. With this approach, decisions are made considering a limited number of alternatives, with partial awareness of the consequences of each of these. This process is driven by "heuristics", which are rules of thumb used to solve problems or do certain types of calculations and based on the knowledge that available information is never perfect and that human abilities are limited and fallible. As the psychiatrist Mauro Maldonato says: "Unlike formal calculation, heuristics are an immediate solution."
The strategies, techniques and creative processes that make up the heuristic approach are useful distortions of reality. These distortions simplify the analysis of the facts and aim to provide a subjective view based on the knowledge that we can recognize only a limited number of alternatives and are aware of only some of the consequences of each alternative. In most cases, these distortions allow us to interpret and, where possible, predict reality quickly and effectively.
The processes of simplification of reality are based on schemes and categories with which the knowledge we use in the processes of perception, memory and thought is organised.
The schemes and categories that we use to organize our knowledge describe people, objects and events through only the characterizing, common or most frequent details, excluding anything that can be traced back to a specific phenomenal manifestation.
Knowledge schemas are based on associations that are immediately available to our awareness and represent what is most common or considered typical. To be clear, when I talk about the beauty of dogs, no one thinks of the beauty of the Bracco Italiano or the Spinone Italiano because everyone thinks of the generic and subjective image of the dog that has been built up over the years.
Knowledge schemes are fundamental for a correct classification of the world which necessarily requires the implementation of an abstraction process with which a set of non-identical elements is created although belonging to the same phenomenal category.
Abstraction processes are fundamental for simplifying the processes of understanding and adaptation. We can say that they are the basis of the mechanisms that govern survival and evolution.
Without an abstraction process we would be incapable of making decisions because each phenomenon would produce a separate element that cannot be compared with similar others. The "environmental dependence syndrome" would develop (Lhermitte, 1986) which makes one unable to inhibit actions stimulated by any single input. In a similar condition, conifers or the single species of which they are composed would not exist (e.g. Scots pine, larch, fir, spruce) but only the single tree that is different from another due to the characteristic assumed by each single leaf.
Although the importance of abstraction processes is shared by all, it should be remembered that in abstractions exceptions or diversity are not taken into consideration. For this reason, when we talk about Africans we don't think about the white-skinned African population, even if it exists.
This tendency of schemes to generalize and exclude exceptions leads to prejudice when we do not have sufficient information on what we are talking about.
The simplification processes that we find at the basis of the heuristic model have an important flaw: the limit to which they go is constituted only by the common sense of those who apply them. For this reason, in some cases, the heuristic process goes beyond the simple simplification of reality and generates real banalizations from which preconceptions arise which, although they may be derived from reality, no longer retain any objective link with it.
The trivialization of reality leads to the development of preconceptions which reverberate in decision-making processes through inevitable errors of evaluation which can be more or less serious. Such errors, regardless of their nature, are generically called "cognitive biases" or more simply "bias".
Cognitive biases are systematic errors of thought that, by causing us to deviate from logic or rationality, influence the way we perceive reality, make decisions and the way we formulate our judgments.
The difference between bias and heuristics is, therefore, represented by the fact that heuristics are convenient and quick shortcuts closely linked to reality and which lead to quick conclusions. Cognitive biases are also shortcuts but they manifest themselves through prejudices that have lost all connection with reality and which are acquired, in most cases, without a critical spirit or judgment.
It is not easy to understand at what point a simplification process turns into a trivialization from which a cognitive bias arises. I believe that it is impossible to set a sort of threshold that allows us to understand that we are in the presence of a simplification process of which we have lost control to the point of declaring it dysfunctional to the decision-making process. For this reason, perhaps, we realize the existence of a bias always, so to speak, once the decision-making process has manifested its effects on the environment and people.
A world of absolute uniqueness in which it is not possible to create groups through abstract processes is a world in which any form of intelligent life is impossible. As unreasonable as it may seem, organizing knowledge by schemes and from these abstractions is common to all intelligent or teleological agents, even of an alien (non-human) nature. For my dog, birds are birds regardless of whether they fall within the species for which he was selected and trained to hunt. You could argue that my dog chases everything that moves purely out of predatory instinct. However, his reluctance to prey on objects unknown to him is common to all other dogs. I still remember the way he behaved when he saw, for the first time in his life, a balloon rolling on the floor because it was moved by the wind and how he behaved in subsequent encounters with this environmental phenomenology.
Abstractions are not lacking even in plant intelligence that implements clustering schemes in learning and adaptation processes. A testimony of this ability is given to us by Stefano Mancuso through his observations regarding the evidence collected by the French naturalist and botanist Lamarck (1744-1829) regarding the behaviors that the "mimosa pudica", so called because it closes its leaves as soon as it is touched, implements in a presumable attempt to defend against herbivores.
A specific aspect is, in my opinion, assumed by Artificial Intelligence (AI) systems which, although not life forms, operate as teleological agents and do so by implementing abstraction and classification processes not dissimilar to those produced by other living species. As noted by Nello Cristianini, every time an AI system implements a classification it does so with its theoretical construct based on its form of intelligence.
It is not possible to know what are the characteristics of the knowledge schemes that an AI adopts to distinguish a dog from a cat or to classify the world. If we could ever find out, we would find that it has nothing to do with our criteria based on human sensory data. I would not be surprised to find in an AI a classification similar to that proposed by Jorge Luis Borges in which the animal world is divided into:
The issue of biases that manifest themselves in AI systems is much more complex if we consider the fact that the statistical correlations that are used in abstraction processes are often, if not always, defined on data which, in addition to being naturally affected by bias, they could hide weakly correlated statistical links that are not evident to humans and capable of generating negative effects on the analysis and decision-making process. To understand the importance of weak correlations and their danger, I report a beautiful definition produced by the Ammagamma team which, in my opinion, David Bevilacqua teaches and disseminated on the topic: "the [weak correlations are] weaker relationships between the variables which influence a phenomenon [and are] difficult to read and interpret. Our mind is not able to grasp them, unlike strong correlations, but by equipping ourselves with a mathematical model it is possible to identify them [and use them to our advantage]". The awareness of the importance that weak correlations assume in the abstraction processes generated by an AI also comes from the studies conducted by James Pennebaker which demonstrate the feasibility of a psychometric segmentation of a user through the linguistic structure adopted in the exposition of their opinions alone. Thanks to its studies and weak correlations, Facebook can cluster groups of people starting only from the likes expressed on users' images and public posts.
Recognizing the existence of abstraction processes in every intelligent agent allows us to understand that biases can be present in every heuristic process regardless of the nature of the agent that brings it into being. Furthermore, I find Borges's provocation a useful tool for understanding that our principles of classification and ordering of the world are anything but obvious and natural as it is possible to hypothesize infinite other ways of organizing the objects of our experience such for example, the paradoxical one I reported above.
At this point in my reasoning, it is best for me to give a brief summary of what I have attempted to explain so far.
Point 1 - Heuristic processes are based on simplifications of reality which, even if functional to achieving the result, are the matrix from which biases arise.
Point 2 - Biases, being linked to simplification processes, are not the result of a specific level of abstraction but, rather, the result of a limit determined only by the level of unreliability that our common sense finds admissible in our cognitive processes and decision-making. In the terms set out, bias is present in every heuristic process and every time we deviate from objective data.
Point 3 - Simplification processes are necessary to implement the abstraction processes that allow us to understand the world regardless of specific phenomenal manifestations. I have also found this capacity for abstraction in agents endowed with intelligence alien to ours.
Bias, understood as a deviated form of simplification and abstraction mechanisms, is present in every heuristic process because it is through the adoption of one or more shortcuts that one can avoid the adoption of a logical-scientific approach which is always very expensive in terms of computing resources and data acquisition and verification time.
The presence of bias in all heuristic processes is also demonstrated by the experiment carried out by psychologist Emily Pronin who, in 2002, described the "blind spot bias" as the natural inclination of human logic to consider ourselves always more objective than anyone else. Another demonstration of the bias-heuristic link comes from the psychologist Paolo Legrenzi and the neurologist Carlo Umiltà who, in the book "Many unconscious for a brain", write
Given the enormous flow of information, we tend to select those that we already know, those with which we agree, those that we can assimilate better thanks to the presence of mental patterns and categories that are familiar to us and which are already consolidated. Furthermore, we are inclined to share this information with those who think like us and with those who we know will appreciate why they think like us. These new forms of life give rise to a sort of collective unconscious which translates into the radicalization of people's opinions. Individuals are comforted by sharing a current of opinions that is simple, clear, and requires low cognitive and attentional efforts
The role of biases in cognitive processes has led to a careful classification of them which, in the absence of taxonomic proposals or reference models, has generated a list of over 150 items divided into four macro-areas over the years.
With such a large list of items, I find it obvious to consider biases as an inseparable part of heuristics, despite the fact that in some cases they become the element that shows the fallacy of some simplification/abstraction processes.
Nobody likes the idea that heuristic processes are based on more or less effective biases because it demonstrates that every choice is always wrong or, if you prefer, right until proven otherwise. This scenario, however, is not as deplorable as it seems since it is precisely thanks to biases that it is possible to accelerate the analysis processes, improve the detection of critical choice factors in changing or uncertain situations and arrive at a more streamlined decision-making model. This is due to the fact that bias is closely linked to the schemes and categories with which the knowledge that underlies the processes of perception, memory and thought is organized.
Heuristics are necessarily based on biases even if, in most circumstances, these biases do not have harmful effects on the context or object of our decisions. In such a condition, as much as we don't like it, it is no longer necessary to ask ourselves whether a decision is made on the basis of a model whose mechanisms show bias or not. Rather, we need to ask ourselves what relevance is assumed by the biases certainly present in the current decision-making process. In essence, since choices are always based on errors of evaluation, let's focus on the distinction between serious errors and slightly irrelevant errors whose effects are, however, only apparently of low impact.
The vision of bias as a problem that explains the anomaly revealed downstream of a decision-making process is misleading because it transforms bias from the effect of an anomaly into a cause of the anomaly. Biases are always the symptom of a problem affecting the decision-making model and, for this reason, they do not exist except as a distorted manifestation of a cognitive process. To be clearer, I don't think it's correct to say that the anomaly found in a decision-making process can be traced back to a bias or is produced by a bias. When faced with an anomaly, it should be said that the problem from which it is generated manifests itself in the form of one or more biases.
The interpretation of bias as a symptomatic manifestation of a cognitive problem requires some reflections. The first is represented by the fact that the correction of the anomaly does not involve correcting the biases through which the anomaly manifests itself (it would be like lowering the fever instead of curing the infection). The second is represented by the fact that the anomaly we find through one or more biases does not mean that it has not produced others of equal importance but not yet identified.
A bias understood as an effect and not as a cause requires the adoption of a completely different approach from the one currently adopted for the analysis of the efficiency of AI systems in which one must necessarily be able to identify which abstraction process generated the anomaly that we identify as bias. To carry out such an analysis of the causes, one must know in detail the parameters that contribute to the generation of the decision and, even more, one must know which combination of "weak signals" generates the specific statistical signature that generated the anomaly. To date, there is no way to be certain which model generated the response.
The presence of biases is, therefore, not useful in explaining something since biases are present in every heuristic process and are present both in analysis processes whose outcomes conform to expectations and in those that do not produce the expected results.
Saying that a decision-making system is affected by bias means knowing why the bias was generated, why it was not avoided, and why correcting the anomaly does not generate a different one in pursuit of the complexity typical of the first Microsoft service packs of the years '90.
But what if the bias is instead a peculiarity of the decision-making model? A decision model is always focused on a contained set of data. This means that, even if there were no extreme discrimination phenomena, we would still find ourselves in a context in which it is not possible to exclude that we are in the presence of a bias worthy of AlphaGo's famous "U-37" move whose effects show themselves as a poison at a time and in a way that makes it impossible to understand the origin of the disease and identify an adequate cure.
Without a selective attention decision-making system, we would be at the mercy of environmental stimuli, even if irrelevant or inconsistent with what we are doing. We might think that it is possible to be supported by an AI in the data reading process but, even in this case, no one could exclude that the agent adopted to support us does not itself fall victim to the complexity of the world by developing the technological equivalent of " environmental dependence syndrome" (Lhermitte, 1986) which makes one unable to inhibit actions stimulated by any single input.
Article source: Linkedin Article by Vincenzo Gioia
