The PROBONO project aims at providing a plan to turn traditional neighbourhood into green and sustainable environments through various sets of innovations in the domains of energy or construction, and in particular through the use of digital twins. Indeed, digital twins are a powerful tool for urban planning, allowing stakeholders to make informed decisions by simulating various scenarios and analyzing their outcomes.
Trust issues in Digital Twins
While developing digital twins in PROBONO, we are noticing many challenges in their deployment. One of them is earning the trust of the different stakeholders in the digital twin’s existence. In the case of a digital twin representing parts of a green neighbourhood, some examples of such stakeholders are of the inhabitants who live in them, the architects who design them, the contractors building them, or the whole supply chain. The reason for this initial lack of trust is simple: digital twins typically rely on collecting and manipulating data that might be either private, unreliable or related to a critical task. As such, people understandably tend to be more protective and skeptical. Usually, this is solved by using reassuring measures, such as offering guarantees on how what is to be deployed works, or the general assumption that malfunctions can be understood, and thus fixed. Again, in the case of a building or a residential area, these usually take the form of following the applicable regulations and recommendations, as well as following the industry standards like ISO19650 (International Organization for Standardization, 2018), which allows for increased confidence in the information of projects.
Yet, digital twins typically include AI technologies to deliver value. Consequently, the distrust can be further exacerbated and the problem of earning trust becomes considerably more complicated. Indeed, it has been demonstrated that AI systems still have challenges to overcome in ethics (for example fighting embedded biases), data quality, data privacy, accountability, or security, as considered in recommendations like the NIST AI Risk Management Framework, standards ISO42001 (International Organisation for Standardisation, 2023), or policies such as the recent AI Act (Regulation EU 2024/1689, 2024).
Explainability and transparency of AI technologies are also one of the main issues. They usually are black boxes. This means that we have no conceptual understanding of the different steps that lead to an output from an input. From this follow, in particular, two considerations. Firstly, it is not possible to clearly identify the causes of a possible malfunction. To put it differently, we have no way of knowing where an error might come from and so we cannot hope to be able to purposedly repair it. Secondly, because we are not able to decompose the computation into a set of clear and relevant steps, it is not possible to offer guarantees that the AI universally works as it should. In other words, there will always be a risk that the AI computes an incorrect output on some occasion.
These considerations obviously impede on restoring trust in the AI technology that may be used, and by extension in the digital twin that includes it. Importantly, if not dealt with, they might only reinforce prior distrust. One may think that not using any AI technology might be a solution, since they are a major source of blurred understanding and functioning. However, these technologies are critical because they allow us to solve problems and perform tasks that we have no mean to achieve otherwise, either because they are the only ones able to do it, or because the other solutions are not efficient enough in practice. Hence, the solution would instead be to decrease the opacity surrounding the use of AI.
Solving the AI Trust Problem: Explainability
Given the increasing popularity of AI techniques and that their opacity is an inherent characteristic, the necessity to make them more understandable has obviously already been identified. In fact, the study of this problem echoes previous work that shared the same ambitions but applied to expert systems and has now become its own field of research: Explainability (or Explicability). The primary aim of Explainability is, in general, to find ways of getting insights as to how an artificial intelligence system in a large sense works. Nowadays, there is a strong focus on AI systems relying on Machine Learning methods in particular.
There are several ways to gain such insights. A first one is to compute values that are representative of a certain behavior of the system. These values are usually called scores, measures or metrics. They typically represent how a system performs regarding a specific notion. Examples of such notions are fairness, robustness, privacy, reliability, etc. Representing these notions using values allows us to compare different AI systems on how they perform, and deciding which one is “fairer”, or “more robust”. These can then be used as prior guarantees as to how a particular AI system behaves. We refer the reader to (Zhou, Gandomi, Chen, & Holzinger, 2021) for a recent survey on this approach. Another way to gain insights of an artificial system is to compute explanations. In general, explanations can be almost anything, but they have the function of helping us in understanding how a certain outcome was reached. We can then use this understanding to spot potential origins of failures and better fix them. In particular, the study of explanations is not restricted to AI systems, even if such systems probably need explanations the most. There are globally two main approaches to explanations: the first one is called abductive and consists in searching through the cause-effect chain the cause that led to the output (Kakas & Michael, 2020), the second one is called counterfactual and consists in searching for changes in the input that lead to a different output (see for instance (Stepin, Alonso, Catala, & Pereira-Farina, 2021)). In the context of AI systems, we tend to use a different taxonomy. Typically, we classify explanations as either global or local and either model agnostic or model specific (model agnostic then meaning not specific to a particular model).
Popular examples of explanations methods for AI systems are LIME (Ribeiro, Singh, & Guestrin, 2016) and SHAP (Lundberg & Lee, 2017). LIME is a local model agnostic method which basically approximates an arbitrarily complex model that we do not understand by a simpler model that we can understand. SHAP is a global model agnostic method that determines which parts of the input played the most relevant role in the computation of an output. In other words, it allows us to understand on which parts of an input a model focuses on for some decision. While LIME and SHAP may be the most known methods, many more exist or are under study or development, as Explainability is an active field of research. Initiatives like these let us unveil, even if only a little for now, the black boxes of AI technologies and thus give us means to reassure those that are impacted by them.
In conclusion, the PROBONO project largely relies on digital twins, the deployment of which faces many challenges. Among those is the problem of earning the trust of the different actors impacted by digital twins, a problem exacerbated by the usual presence of AI technologies in digital twins because of their opacity. To mitigate this opacity is the objective of the research field known as Explainability. LIME and SHAP are well known examples of solutions produced by this field, but since Explainability is still an active area of research, there exists many more solutions that may be better suited to specific cases.
References
International Organization for Standardization. (2018). Organization and digitization of information about buildings and civil engineering works, including building information modelling (BIM) — Information management using building information modelling. (ISO Standard No. 19650-1: 2018).
International Organization for Standardization. (2023). Information technology - Artificial intelligence - Management system. (ISO Standard No. 42001:2023).
Kakas, A. C., & Michael, L. (2020). Abduction and Argumentation for Explainable Machine Learning: A Position Survey. CoRR.
Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Proceedings of the Annual Conference on Advances in Neural Information Processing Systems (NIPS), (pp. 4765-4774). Long Beach.
Regulation EU 2024/1689. (2024, June 13). Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Retrieved from https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32024R1689
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD) (pp. 1135-1144). San Francisco: ACM.
Stepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021). A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence. IEEE Access, 11974-12001.
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021). Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics. Electronics, 593.
Comments