The Influence of Organizational Factors on the Risk of AI Systems

Valentino, Justin

Permalink

https://hdl.handle.net/2142/121815 Copy

Description

Title

The Influence of Organizational Factors on the Risk of AI Systems

Author(s)

Valentino, Justin

Issue Date

2023

Keyword(s)

• Organizational
• Probabilistic risk assessment
• Artificial intelligence (AI)
• Imagination

Abstract

Artificial intelligence (AI) systems are rapidly advancing, and there is an increasing need for the regulation of AI systems. AI has been used to draft legislation that proposes the regulation of AI, but it is important to recognize that AI currently utilizes large-scale machine learning algorithms that are incapable of deploying imagination and creativity. Historically, a lack of imagination when considering the risk triplet (i.e., what can go wrong? What are the consequences? How likely is it?) has been a contributor to major accidents. This paper is part of a line of research evaluating the influence of organizational factors on complex socio- technical system risk, specifically in the regulation and management of AI system risk. Historically, organizational factors have been significant contributors to accidents and incidents in high-consequence industries. The explicit incorporation of organizational factors into PRA is critical for risk assessment (in addition to risk management) because this explicit incorporation can help generate a more realistic estimation of human error, software error, and equipment failure probabilities. Socio-Technical Risk Analysis (SoTeRiA) refers to the explicit and theory-based integration of underlying failure mechanisms (i.e., hardware, software, human and organizational) into the PRA of complex technological systems. This research expands on the SoTeRiA framework to consider the organizational influencing factors on the development, deployment, and management of AI systems. For example, Organizational safety culture for AI is a critical aspect of managing AI system risk. It involves creating a culture within an organization that values and prioritizes safety in the development and deployment of AI technologies. This includes fostering an environment where employees are encouraged to identify and report potential risks associated with AI systems without fear of repercussions. Organizational practices that prioritize protection over profit are crucial in managing AI system risk effectively. This means that organizations should prioritize the protection of society and stakeholders over financial profit when developing and deploying AI technologies. It involves putting in place robust risk management practices, even if they may impact the profitability or competitiveness of the organization. Organizational practices that prioritize protection over profit include thorough risk assessments, rigorous testing and validation of AI systems, and a commitment to ethical and responsible AI development and deployment. Furthermore, organizational factors such as leadership commitment, employee training, and accountability mechanisms also play a crucial role in managing AI system risk. Strong leadership commitment to safety and risk management, coupled with adequate training for employees involved in AI development and deployment, can foster a culture of safety and responsible AI practices. Accountability mechanisms, such as regular audits and reviews, can ensure that organizations are adhering to established controls and procedures and continuously improving their risk management practices. In conclusion, managing AI system risk requires organizations to prioritize safety, adopt risk- informed decision-making, and implement organizational practices that prioritize protection over profit. By considering these concepts and integrating them into their organizational processes, organizations can effectively address the influence of organizational factors on AI system risk and ensure responsible and safe development and deployment of AI technologies.

Type of Resource

text

Genre of Resource

Conference Paper/ Presentation

Language

eng

Handle URL

https://hdl.handle.net/2142/121815

Owning Collections