AI unleashed: Bridging the divide for experts beyond algorithmic realm

Authored by AI expert Roli Khanna.

As I sat across from my doctor (let’s call her Dr Patel), discussing a recent health concern, our conversation took an unexpected turn. Dr Patel, a dedicated physician with years of experience, shared her journey of integrating AI into her medical practice. She recounted how navigating the labyrinthine world of algorithms and data-driven diagnostics felt like deciphering an alien language. Instead of feeling empowered, she felt overwhelmed by the impenetrable wall of technological jargon.

Her experience epitomises a pervasive challenge in our AI-driven era: the glaring disconnect between cutting-edge AI systems and the domain experts who could wield them most effectively. It underscores a growing imperative to make AI systems transparent and accessible to professionals like Dr Patel, whose expertise lies far from the world of algorithms and machine learning, yet whose insights are invaluable in harnessing AI’s transformative potential.

The solution to Dr Patel’s predicament and the importance of making AI systems comprehensible and user-friendly for professionals across diverse fields forms the core of the emerging field of explainable AI. My experience with Dr Patel, among many others, served as my motivation to step into the world of explainable AI as a graduate student at Oregon State University. There, along with my team members, I developed a process known as “AAR/AI” or “After Action Review for AI” [1] to help demystify AI systems, making them comprehensible and navigable for professionals from various fields, particularly those not well-versed in AI. AAR/AI’s purpose is to assist domain experts in localising an AI agent’s bugs effectively. The aim is to provide users with the ability to determine when they can confidently rely on an AI agent and when they should exercise caution.

Processes such as AAR/AI empower individuals to make informed decisions and collaborate effectively with AI systems, and much more. For example, creating transparent and easy to use AI systems helps bolster trust and accountability in them.

Imagine a scenario where a self-driving car makes a critical decision in a life-threatening situation. As passengers, we would like to know why the car made the choice it did. Was it a decision based on ethical principles, road conditions, or something else? The ability to understand and explain these decisions builds trust in AI systems. Transparency ensures that AI’s actions are not perceived as mysterious or arbitrary, leading to greater trust in AI-driven technologies.

Moreover, transparency fosters accountability. When AI systems are transparent, developers and organisations can be held responsible for their actions. If a self-driving car makes a questionable decision due to a flaw in its programming, it’s crucial to pinpoint the cause and rectify it promptly. Accountability not only ensures safety, but also pushes AI developers to strive for excellence in their designs.

Another key to the puzzle of explainable AI is ethics: since the data that AI systems are trained on is often biased, the resulting AI systems are inadvertently biased as well. Without transparency, it becomes challenging to detect and correct these biases. 

Moreover, enabling processes that promote transparent AI provides valuable insights into how AI systems make decisions. This information can be used to improve the models, algorithms, and data used in AI development, enhancing their accuracy and effectiveness.

This constant improvement is essential in critical applications like autonomous vehicles or medical diagnosis, where understanding AI decision-making is vital for safety. In cases of system failures or unexpected behaviour, being able to explain AI’s reasoning can prevent accidents and save lives.

Making AI systems transparent and explainable is essential for harnessing the full potential of AI. When domain experts, who may not be AI specialists, can understand and interact with AI systems effectively, it leads to collaborative innovation. My goal with creating processes such as AAR/AI was to empower experts like Dr Patel to leverage AI’s potential effectively, ensuring that the benefits of this technology reach far and wide in solving complex problems across industries.

As we continue to integrate AI into our lives, transparent and explainable AI systems are not merely a luxury, but an ethical and practical necessity, guiding us toward a future where AI truly augments human potential.

[1] Roli Khanna, Jonathan Dodge, Andrew Anderson, RupikaDikkala, Jed Irvine, ZeyadShureih, Kin-ho Lam, Caleb R. Matthews, Zhengxian Lin, MinsukKahng, Alan Fern, and Margaret Burnett. 2021. “Finding AI’s Faults with AAR/AI: An Empirical Study”. ACM Transactions on Interactive Intelligent Systems (2021).