When Machine Translation Usefulness Is Higher Than Quality

Often usefulness of machine translation quality is regarded as the same thing as its quality. However, machine translation can be used in many ways. And in different use cases different things matter. Here we represent one case where translation’s usefulness depends only partially on the machine translation quality.

The example is about a newsletter which was machine translated to different languages.  The newsletter recipients were asked to evaluate both the quality of the translation and how useful they think the translation is. (Read the complete case study here.) The graph below shows the results.

Source: http://www.roi-learning.com/dvm/pubs/articles/tatc-24/

These results are quite interesting. The graph shows that although the machine translation quality was evaluated far from perfect, the translations’ usefulness was regarded as higher than its quality. However, this applies only when translation quality is above certain threshold. Bad or poor quality machine translations are naturally deemed as useless.

Although this case study is rather old, it seems that the respondents were rather tolerant towards translation errors. The quality of machine translation is nowadays quite different than it was in the beginning of the 21st century. It’s hard to tell what were the respondents general conceptions about the overall quality of machine translation. It is probable that today the use of free machine translation services has taught people to expect even less from machine translation.

The most important thing here is to realize that the quality (or the lack of it) doesn’t directly determine the usefulness of the machine translation. The quality of machine translation is more than just correct grammar.


Localize Your Software with MultilizerTranslate PDF documents automatically with Multilizer. Now also free Multilizer PDF Translator available for translating short PDF documents. Read more and download here!

Published by

Multilizer / Niko Papula

I am managing director of Multilizer, a Finnish software company specialising in software for enhancing translation quality, speed and cost.