UW Interactive Data Lab
IDL logo

Human Effort and Machine Learnability in Computer Aided Translation

Spence Green, Sida Wang, Jason Chuang, Jeffrey Heer, Sebastian Schuster, Christopher D. Manning. Empirical Methods in Natural Language Processing, 2014
Analyses of computer aided translation typically focus on either frontend interfaces and human effort, or backend translation and machine learnability of corrections. However, this distinction is artificial in practice since the frontend and backend must work in concert. We present the first holistic, quantitative evaluation of these issues by contrasting two assistive modes: post-editing and interactive machine translation (MT). We describe a new translator interface, extensive modifications to a phrase-based MT system, and a novel objective function for re-tuning to human corrections. Evaluation with professional bilingual translators shows that post-edit is faster than interactive at the cost of translation quality for French-English and English-German. However, re-tuning the MT system to interactive output leads to larger, statistically significant reductions in HTER versus re-tuning to post-edit. Analysis shows that tuning directly to HTER results in fine-grained corrections to subsequent machine output.
  title = {Human Effort and Machine Learnability in Computer Aided Translation},
  author = {Green, Spence AND Wang, Sida AND Chuang, Jason AND Heer, Jeffrey AND Schuster, Sebastian AND Manning, Christopher},
  booktitle = {Empirical Methods in Natural Language Processing},
  year = {2014},
  url = {https://idl.uw.edu/papers/interactive-translation},
  doi = {10.3115/v1/d14-1130}