Can postgraduate translation students identify machine-generated text?
Abstract
Given the growing use of generative artificial intelligence as a tool for creating multilingual content and bypassing both machine and traditional translation methods, this study explores the ability of linguistically trained individuals to discern machine-generated output from human-written text (HT). After brief training sessions on the textual anomalies typically found in synthetic text (ST), twenty-three postgraduate translation students analysed excerpts of Italian prose and assigned likelihood scores to indicate whether they believed they were human-written or AI-generated (ChatGPT-4o). The results show that, on average, the students struggled to distinguish between HT and ST, with only two participants achieving notable accuracy. Closer analysis revealed that the students often identified the same textual anomalies in both HT and ST, although features such as low burstiness and self-contradiction were more frequently associated with ST. These findings suggest the need for improvements in the preparatory training. Moreover, the study raises questions about the necessity of editing synthetic text to make it sound more human-like and recommends further research to determine whether AI-generated text is already sufficiently natural-sounding not to require further refinement.
Published in
Machine Translation Summit XX: proceedings, volume 1. 23-27 June 2025; pp. 432-441.
Engaging copy translated literally into English, without taking account of differences in linguistic, semantic and cultural expressions, at best leaves much to be desired and at worst provokes hysterical laughter.
Thanks to my scientific background, I specialize in technical translations. Over the years I have acquired experience in transcreating advertising copy and press releases primarily for the promotion of technology products.
All clients of mine who have sent me urgent requests are satisfied with my response. 
