Beyond Translation Quality: A Risk-Based Framework for Monolingual Reliance on AI Translation

Abstract

AI translation has improved to the point where many monolingual professionals now use it directly in their work. Healthcare professionals, lawyers, teachers, businesses, and public organisations increasingly rely on AI-generated translations without involving a bilingual reviewer. Most research has focused on one question. How accurate are these translations? To answer this, researchers use evaluation methods such as Multidimensional Quality Metrics (MQM), Direct Assessment, and the Crosslingual Optimized Metric for Evaluation of Translation (COMET). However, these methods do not answer the question that matters most in practice: Can this translation be relied on safely, for this purpose, in this situation? This paper argues that translation quality and translation dependability are different concepts. Quality evaluates how well a translation has been produced, whereas dependability considers whether it is safe to rely on that translation when making a real decision. Addressing this problem requires a framework that supports decisions about using AI translations rather than simply measuring system performance. Drawing on Pym's (2015) theory of translation risk and Lee and See's (2004) theory of trust in automation, this paper proposes a four-tier framework supported by a decision tree and a practical checklist. The framework is illustrated across eight professional domains to show how it can guide decisions about the safe use of AI translation. Finally, the paper compares the framework with existing translation evaluation methods and outlines directions for future empirical validation.