Masters thesis in Data & AI: Automated Practicality Evaluation of Counterfactual Explanations Using GPT Veenendaal • Info Support

Dive into the intersection of AI and human practicality in this thesis. We’re crafting a metric that measures how achievable AI-generated advice is—for real people. Think of it as teaching AI common sense; for instance, you can’t just age a decade instantly. Your challenge: integrate advanced language models like GPT to gauge if a “what if” scenario is useful or just a tech fantasy. Join us in shaping a more sensible, user-centric AI future.

ð¡Areas of Interest: XAI/Explainable AI, LLMs/Large Language Models, ChatGPT, NLP/Natural Language Processing

Counterfactual explanations are considered an intuitive and user-friendly form of explainable AI. A counterfactual explanation ­proposes minimal changes to the input data that would lead to a different model output. As an example, a bank could make use of a model to determine if a customer can get a loan or not. If a customer was denied a loan, a counterfactual explanation could tell them that, for instance, “if their savings were increased by 10.000€, they could get the loan”.

However, counterfactual explanations are not aware of the semantics of the input features. It might be that the minimal change for a customer to get a loan is to “increase their age from 25 to 35 years old” which, from a human perspective, is evidently unfeasible in a short time span.

In this project, we would like you to design a new evaluation metric to automatically determine the practicality of counterfactuals, that is, how attainable a counterfactual explanation is (or if it is even humanly attainable). Since the determination of practicality involves semantic understanding of the input features, our idea would be to leverage GPT and/or other large language model-based technologies to accomplish this.

A user study could be conducted initially following the methodology proposed by Spreitzer et al (1) to evaluate the practicality of different state-of-the-art counterfactual techniques. The user study can then be extended to evaluate the accuracy of the proposed metric compared to the human evaluations.

(1) Spreitzer, H. Haned, & I. van der Linden, “Evaluating the Practicality of Counterfactual Explanations”

About Info Support

Info Support specializes in custom software, data/AI solutions, management, and training and is active in the Finance, Industry, Agriculture, Food & Retail, Mobility & Public, and Healthcare sectors. We provide solid and innovative solutions for complex and critical software issues. Our headquarters are located in Veenendaal (NL) and Mechelen (BE). At present, approximately 500 employees are employed by Info Support.

Info Support's working method is characterized by a number of core values: solidity, integrity, craftsmanship, and passion. These core values are intertwined in our work and the way we interact with each other.

To ensure that all employees are always up to date with the latest developments, Info Support has an in-house knowledge center that eagerly satisfies the hunger for more or different knowledge and skills.

B2 language proficiency in Dutch is required.