Abstract

In the early 2010s, when Artificial Intelligence (AI) emerged from the latest "winter" of disillusionment, discussions about its use in decision-making tended to be dichotomous: humans or AI. More recently, consensus has shifted toward a complementary humans-and-AI perspective, recognising that augmentation, not substitution, is the more appropriate frame for AI's role in the foreseeable future. This shift foregrounds the challenge of evaluating outcomes in collaborative decision-making contexts.The thesis proposes a simulation-based framework, drawing on system dynamics, to evaluate whether complex, time-varying interactions between humans and AI agents result in collectively rational behaviour. The framework extends the observation of dynamic complexity to settings where algorithmic components handle pattern recognition and prediction while human judgement governs goal-setting, threshold parameters, and cross-process coordination. Two testbeds test the framework's viability: a stylised supply chain model and a bike-share system.In the supply chain testbed, forecasting product returns is handled by machine learning, while human judgement determines how forecasts translate into capacity and inventory decisions. A partial model tests local rationality, while a fuller model places forecasting in a broader operational context to assess holistic rationality. In the bike-share case, inventory balancing relies on crowdsourcing, where incentivised users take action. Using open data from New York's Citi Bike, the study highlights misalignments between incentives and system needs, underscoring the complexity of user behaviour mediated by bike availability.The findings confirm that AI models evaluated in isolation may deliver disappointing results when task performance is considered holistically. These cases demonstrate the central contribution: a simulation-based framework for assessing the joint performance of human and AI decision-makers, supported by a simulation workbench that enables systematic exploration of alternative policies.

Awarding Institution(s)

University of Plymouth

Supervisor

Guido Siestrup, Martin Knahl, Marco Palomino, Nathan Clarke

Document Type

Thesis

Publication Date

2026

Embargo Period

2026-02-13

Deposit Date

February 2026

Creative Commons License

Creative Commons Attribution-NonCommercial 4.0 International License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

Download

Share

COinS