Estimating machine translation quality is difficult.

About Estimating the Quality of Machine Translations

Estimating machine translation quality is difficult.It is usually very difficult to estimate or evaluate the quality of automatic translations. The main reason for this is that typically the user translates a piece of text either from or to a language that he doesn’t understand good enough. (If he did understand, then he wouldn’t need to translate it.) Thus he cannot really know whether the translation contains errors or not.

The traditional way to get an understanding of the machine translation quality has involved comparing the machine translation with a translation made by a professional translator. This has been done mainly for research and development purposes by the developers of machine translation technology. However, this kind of method is not useful in real-life situations because usually there isn’t available any professional translation for material which needs to be machine translated.

There is definitely a demand for a technology for automatic estimation of the quality of machine translations. The general idea behind such a technology would be creating a system which would automatically tell the user whether some machine translation is good or bad or something between. This kind of technology would benefit us in several ways:

  1. With reliable machine translation quality estimates automatic translations can be used in more demanding situations. Currently machine translation is often used for translating information where translations errors would cause only minimal or no harm at all. This has seriously limited the use of automatic translation.
  2. Bad translations can be filtered out of a post-editing process. The post-editing can be made more efficiently because the post-editor does not need the spend time with useless translations.
  3. Deciding whether the machine translation is good enough for publishing will become easier.
  4. The system can be used to select the best among several machine translations. This will obviously improve the overall perceived machine translation quality as well as avoid embarrassing mistakes.

Lately we at Multilizer have made significant steps in estimating machine translation quality automatically. The quality estimation will be also one of the subjects in a scientific workshop about machine translation in Montreal at the beginning of June.


Localize Your Software with MultilizerLocalize your software efficiently with advanced Multilizer localization Tools. Free Trials available!

Published by

Multilizer / Niko Papula

I am managing director of Multilizer, a Finnish software company specialising in software for enhancing translation quality, speed and cost.

2 thoughts on “About Estimating the Quality of Machine Translations”

  1. I agree when you said that estimating or evaluating the quality of machine translations is very difficult to do.

Comments are closed.