Issue #51 - Takeaways from the 2019 Machine Translation Summit

Introduction 

As you will have probably seen across our website and channels, the Machine Translation Summit took place a few weeks ago on our doorstep in Dublin, Ireland. In addition to sponsoring, hosting the MT social, and presenting our own paper, we also attended many talks, had a lot of great conversations, and since the conference, we have spent some time reading up on some of the work we didn't get to catch. As expected, we came across many interesting pieces of research . In this post, we summarise three papers that we found particularly interesting!

Controlling the Reading Level of Machine Translation Output (Marchisio et. al. (2019) )

Depending on the target audience, we may want to change the complexity of the resulting translation output. Marchisio et. al. (2019) in their work make a successful attempt to incorporate the same. 

In the first approach, they change the training data. They add a text token at the end of each source sentence depending on the level of complexity required. They used only two levels of complexity: “simple” and “complex”. 

In the second approach, they change the underlying model. They train an encoder-decoder model with a shared encoder and two decoders—one for “complex” decoding, and the other for “simple” decoding. At inference time, they pass a flag indicating the desired complexity to select the right decoder.  

From the results, they were able to control some complexity in the output. They observe that for both approaches, the system was able to generate relatively simple and complex outputs.  

To illustrate, we can see in this example from the paper where the complex model chooses initiate and simple model chooses start.  

•           Source: Pero mis provocaciones están dirigidas a que se inicie una conversación.
•           Simple Translation: But my provocations are meant to start a conversation.
•           Complex Translation: But my provocations are directed to initiate a conversation.

Improving Neural Machine Translation Using Noisy Parallel Data through Distillation (Dakwale and Monz (2019))

Dakwale and Monz (2019) propose an approach to use noisy parallel data. In general we filter the noisy data to have only clean sentences for training and this brings us better results compared to using all noisy data. However, Dakwale and Monz (2019) show that we can train a model on clean data and use this clean model as a teacher model to train our model on “clean + all noisy data”. In this way, we can better utilize the noisy data and the resulting model will be better than any of the following models:  

1) trained on only clean parallel data  

2) trained on clean data and all noisy data  

3) trained on clean and filtered noisy data, where the filtering is performed by dual cross entropy filtering method (Marcin Junczys-Dowmunt 2019) 

4) trained on clean and back-translated noisy data

Selecting Informative Context Sentence by Forced Back-Translation (Kimura et al. (2019))

What's next?

To avoid repetition, in this post we deliberately skipped the four publications we covered in our previous post “Previewing the 2019 Machine Translation Summit''. We suggest you go through the previous post though, if you missed it. Even with these two posts it is hard to detail all of the in depth work covered in these papers and other interesting papers published during the MT Summit 2019. Therefore, it is highly advisable to go through the published MT Summit 2019  proceedings! 

Stay tuned for next week's post where Dr. Patrik Lambert will review interesting papers from another important research conference - ACL 2019.