We have mixed feelings about translation tests here at Moravia. We do them, of course, since they are often required to reach the next round of an RFP.
However, the test may not be a true indication of quality. It may be a bit like asking a construction company to build a beautiful doll-house for you to make sure they can be trusted with building the real thing. Why?
- Vendors will pull out all the stops for a test. Not to say that they don’t drive the highest quality on all projects, but ‘real’ projects typically have more budget and time constraints. Would any vendor not treat a translation test with tender care? They would always put their most senior, most specialized resources on a test; they will always add a couple extra QA steps that may not be part of a regular translation process; they may write up reports that are typically not required in a real project.
- Test translations may often get evaluated by parties who are not objective. For example, another vendor or an in-house translator may feel threatened. A reviewer may have personal preferences and state them as errors, when a translation truly is accurate and good. In that situation a potential vendor is doomed, no matter how good the result of the test is.
But most importantly, a translation test doesn’t measure the value of long-term relationships or partnership approach.
Clearly, we question how a client can evaluate any company’s worth based on one test. A better way of evaluating a vendor’s capabilities is to ask for publicly available samples – web content, a user manual, etc.
But if you MUST do a test, then here’s how you could make the most of it to evaluate your vendor in a meaningful way.
We propose you change the goal of your test. Ideally you can get away from the sole, problematic purpose of evaluating linguistic quality. (Let’s face it: most of your potential vendors will deliver a translation of acceptable quality because all will take all the steps to achieve it).
Your ultimate goal should be to understand how a company will work with you. So, you might seek answers to questions like:
- Are they just phoning it in or do they really try to engage?
- How do they communicate with you? (In this case you have to be open to back/forth with the vendor, whether by phone or email).
- Do they ask relevant questions?
- Will they / do they work with your in-country reviewers to provide a response to the feedback?
- What do they deliver? Is it just files, or an email/report talking about their approach? Do they summarize issues or suggestions?
Of course, you will want to evaluate translation results, but that will not be the main focus of your test. The new objective will help evaluate their customer service philosophy and approach. Would you rather understand the long-term approach to a localization program or the quality of one small sample?
One last suggestion: don’t forget to give the vendor feedback after the test has completed. Don’t leave them hanging after they did all that free work!
Buyers: what information have you gained from past translation tests? Are they worth it?