The Effectiveness of ChatGPT-Generated Translations on The Post-Editing Process of Academic Text

Abstract

This study investigates how effective ChatGPT-generated translations are during the post-editing phase of academic texts. It is driven by the increasing use of artificial intelligence (AI) in professional translation workflows and the need to identify its strengths and limitations. Based on translation quality assessment theory, the research aimed to determine how ChatGPT helps human translators produce accurate and readable academic translations. Two native Indonesian translators participated—one from the Association of Indonesian Translators (HPI) and another from the Association of Indonesian Government Translators and Interpreters (IPPI). One acted as the posteditor improving ChatGPT’s Indonesian translation of an English academic abstract, while the other served as an independent rater evaluating the final output. The study used Translog-II software for keystroke logging, OBS Studio for screen recordings, and several assessment tools including Human-targeted Translation Edit Rate (HTER), Translation Quality Index (TQI), a function-based Translation Quality Assessment (TQA) form, and a post-task questionnaire. Results showed an HTER score of 30.5% and a TQI score of 69.5, classifying the translation as “Improvable.” This indicates that although ChatGPT generated coherent and understandable text, significant revisions were necessary to meet academic standards. The analysis highlighted ChatGPT’s main strengths in grammar correction, syntactic fluency, and adapting tone for formal academic contexts, which helped lessen the cognitive and mechanical effort of the post-editor. Its weaknesses included difficulties with cultural nuances, idiomatic expressions, and contextual accuracy, requiring ongoing human expertise. Overall, the study suggests that when used ethically and collaboratively, ChatGPT can be a valuable tool that boosts translators’ productivity and efficiency, but human intervention remains essential to ensure accuracy, cultural appropriateness, and contextual integrity in academic translation.