If you’ve ever investigated or deployed Machine Translation (MT), you know that quality is often a moving target. Everyone has different expectations, and a common misconception is that raw MT can be publishable as human quality. Humans still have to get involved—post-editors—to bring the quality up to snuff. And you most often hear about light versus full post-editing. But questions abound like: who is best suited to post-edit? What training do they need? What is the right level to hit defined quality targets?
To understand post-editing (PE) better, I interviewed Moravia’s expert, Paula Estrella. She is a teacher and researcher in the field of Machine Translation (MT) and Natural Language Processing (NLP) with a PhD in Computer Science—MT translation evaluation. She’s also a Professor in the Natural Language Processing Department at the University of Córdoba in Argentina.
Here’s what we talked about.
Paula, talk to me a bit about your background. Are you a translator?
I’m a computer scientist who has been working on machine translation and evaluation and translator training for years. I come at the problem of MT from a technology perspective, not from a linguistics perspective. But they’re related.
What qualifications does a post-editor need? I assume they need to be a linguist.
This is not straightforward. The experience and skills you need to do a good job can vary. A post-editor could be a translator or a linguist, computer science student, or a subject matter expert. It’s a misconception that post-editors must be linguists.
If they are computer scientists—not experts in the field of translation—they will understand the technical part of how MT works, but will probably not catch or understand the translation problems related to MT output.
Then you have subject matter experts. Sometimes the content is technical, like in product descriptions or user or service manuals. So instead of working with translators, we work with experts in the subject matter. If it’s too technical, a translator is going to have a hard time trying to understand the content and wouldn’t be able to edit it properly.
And sometimes you can only use a linguist to do the work—someone who is bilingual and really understands the structure of the source and target languages.
It totally depends on the requirements of the project and the content the person is expected to post-edit. You need to analyze the project’s content first and the client’s requirements, and then see who would be the best fit for the post-editing task.
How do you train people to post-edit?
No matter who the resource is, we show them what machine translation is about, and what it means to post-edit it. Also, we cover what kinds of errors they will find and the kinds of errors that they themselves can produce if they don’t spot them correctly.
If they are translators who know nothing about the MT technology, then you should complement their education with technology training. And if they’re technical people, then you should teach them something about translation.
So it’s a mix of technology and translation training—depending on who you are training.
What are the main challenges when you conduct post-editing?
Expectations that are too high—especially when it comes to quality. If you ask five different people to judge the quality, you will probably get four different answers. There are factors that can influence quality, and you need to educate people about them: for example, we don’t have the right technology in place, or good source content, or we don’t have enough data. It’s very dependent on the context, on the client, on the purpose of the translation, on the target audience, etc.
You have to give information to everyone so that they’re all on the same page about the question of quality. Give the client samples of what their results could be with either full post-editing or light post-editing, raw machine translation, and translation only by humans, so there is a better idea of what post-editing can give them.
Are there other challenges?
The work is actually quite difficult for translators. This is sometimes why translators are not the best post-editors. Translators think of their profession as an art, and post-editing is certainly not that. They are not very happy to do the work sometimes. But translators have to learn it if they want to provide value to any post-editing work they do. There is also the issue of staying relevant in the industry.
What trends in PE are you seeing?
We have more tools and algorithms, and techniques that can help in the task of post-editing. Automation is coming into different places in the process. You can pre-edit text automatically, perhaps by defining a rule for things that you don’t want to translate. Sometimes you find names that shouldn’t be translated, or you add glossaries—and automate the implementation. After translation, some post-editing can be done automatically, like checks for punctuation or grammar.
I keep hearing about neural MT.
Oh yes. Neural MT helps in many cases. It’s just a matter of time. Maybe if we wait a couple of months or years, we will have a neural MT system ready to use and ready to be customized.
(See our podcasts on Neural MT here.)
Lastly, what tips do you have for a post-editor to succeed?
It’s the attitude. You have to accept the job as a different job. It’s not translation or editing, it’s post-editing. If you’re going to do it, you need to understand it’s quick work. Especially if it’s light post-editing—you cannot read every word. You can’t regret not having changed a word for a better synonym.
Paula, thanks so much for your time. You’ve given us some new ideas to ponder and cleared up some misconceptions. Everyone is interested to see what happens next in this field.
What questions do you have about post-editing? Put them in the comments section and let’s discuss!