How to check game localization quality for free?

Recently I have been exchanging e-mails with Cassie Nantais from Borealys Games. We were wrapping up Polish localization of Mages of Mystralia.

When she wrote:

I think we were lucky because we didn’t have much complaint from the community for the translation except from some Chinese players. It was at that moment that we really realized that we don’t have the power and money to verify the quality of the translation of the company we hire. It’s good to know people in this field so you can hire freelancers you trust. But it’s hard to know someone for all the languages. We had the chance to know someone personally for the Japanese version of our game and to have you for the Polish version.

It struck me I have to expand this article (which was almost ready at that moment), as the need for quality localizations is ubiquitous and developers require some ways to control the translations they receive form their localization specialists. Yet the developers are restricted in this field mostly with time and money.

So is there a quick and inexpensive way you – a game developer – can check the localization quality of strings translated to foreign languages by your language professionals?

Let’s focus on the following facts:

– you do not know the foreign language the game was translated into,
– you have very limited knowledge about the translation process,
– you do not have financial resources to look for the external reviewer,
– nor do you have time to find a qualified one.

In this case the answer must be NO.

Checking the quality of localization is impossible for you… Or is it?

Surely you will not be able to check grammar, vocabulary or style of the text written in a foreign language you do not know, but there are some other aspects you can verify. And you can do this without knowing the language, digging into the translation process, hiring additional language specialists or spending a lot of time. What is more, your quick, easy and automated quality control process can render pretty amazing results and is inexpensive at the same time.

Yes, but how is that possible?

You will do it manually. There is a simple, yet powerful application – namely Xbench. We already are aware of what you cannot check, so what can you check with Xbench? All types of technical errors.

OK, but “technical errors” are tiny issues, right?

Wrong and double wrong.

1. These tiny “technical” issues can completely alter the meaning. Let’s say you had your game localized from English to Polish. One string is “The ship weighs 100,000 tons”-> “Statek waży 100,000 ton”. Where is the tiny “technical” error? 100,000 in Polish means one hundred tons (with three decimal places), and not one hundred thousand tons. Correct Polish number is “100 000” with a non-breaking space.
2. Another example: you have 2 different strings: “I do like this armor”, “I do not like this armor”. Your translator was in a hurry, did not pay enough attention, or was tired, and translated both of them the same: “Podoba mi się ta zbroja”. Not noticing a NOT is a more common error than you could imagine. A lot of translators do it, me included.
3. Small errors such as spaces before punctuation marks, wrong quotation marks etc. just simply look unprofessional. And make your game look bad.
4. Many small “technical” errors in one file MIGHT suggest the presence of more serious issues. This could be tempting to assume that your translator is sloppy, when he/she did not remove these pesky double spaces. Do watch out with this kind of speculations and remember that everybody makes mistakes. Yet SOMETIMES small errors are indeed an indicator that your friendly localizer is not the best specialist to choose. What is great, however, that there is software that allows you to check the files. You can also generate a clear report of potential issues, send it to your translator, ask him/her to read the text again and fix all real errors.

So, how to check localization quality of my game?

Use Xbench. Version 2.9 is still free and you can download it here: https://www.xbench.net/index.php/download. Version 3.0 is paid, but if you find Xbench useful, do buy it, because it is worth the money.* For Xbench to work and generate a list of potential errors (we will come back to this “potential” later on), you need a bilingual file. This should not be a problem for you, as a simple 2-column XLS file saved as a tab delimited text would do just fine. You do not need any of the fancy bilingual translation formats such as SDLXLIFF, TTX or SDLX. Here you can see all formats supported by Xbench. 2.9.

Let’s get into more details!

Xbench is not a complicated software, but see below how to generate an error report.

1. Run Xbench.

2. Press CTRL+N to display Project Properties dialogue.

3. Drag and drop the bilingual tab delimited text file or press Add, Next and pick the file to check. (The test file is here. Download and use it to get the same results as below. Big thank you to Enrique Pañios who allowed me to use some strings from his great game Immortal Redneck in this exercise). Remember: your files must be stripped of all additional columns (keys, IDs, character counts, etc. – they must contain only source and target language columns in one tab).

4. Uncheck Remove Duplicates (you want to check all instances of strings) and check Ongoing Translation (this will allow Xbench to recognize the file as ready to check).

5. Click OK and then OK

6. Now click QA In Check Groups and List of Check you can see all the basic checks Xbench can perform.

7. Leave all the boxes checked and click Check Ongoing Translation and see the results.

Even this basic set of check delivers some tangible results:

1. 1. Inconsistency in target: “Temporary invincibility” has two different translations. That’s odd. You do not have to understand the translation to see there are two different words (“niewidzialność” and “nietykalność”). This is inconsistent.
   2. Inconsistency in source: “Decrease” and “Increase” are opposites, yet both translations are exactly the same. This cannot be correct.
   3. Numeric mismatch: Translation contains two numbers that are not present in the source. But this might not be an error as the source contains “four” and “one”.
   4. Numeric mismatch 2: this one seems to be a real mistake – 20 and 200 are two different numbers.
   5. Double blank (1 and 2): looks like a clear mistake.
   6. Repeated word: also looks suspicious.
   7. Target same as source: an untranslated phrase. Seems to be a clear mistake again.

Regardless of what you personally think about these issues for now they are all POTENTIAL errors. They can all turn out to be false positives of course, so do not judge your translator at this stage, nor take any drastic steps based on this report. Just export the report, deliver it to your translator, and ask him or her to comment on it and indicate what is a false positive that requires no action, and what is a real error that needs fixing. And that is exactly what Xbench is for: to find potential errors, ask for clarification or assurance and tell the translator that you spotted some odd phrases which might need fixing.

8. Right click the report, press CTRL+E and save it as XLS.

Above you can see the report opened in Excel. You can ask your translator to comment in column E (and state if each issue is a FALSE POSITIVE or a REAL ERROR) and fix all real errors.

What do you gain?

Xbench allows you to:

1. Quickly analyze files received from translators
2. Check localization quality for free
3. Notice many potential errors
4. Verify translations even if you do not know the target language
5. Prepare an error report for your translators

Is that all?

Yes and no.

YES: As stated in the beginning of this article: you can scan your translation for potential errors and this method is free, takes little time and does not require foreign language knowledge. Job done.

NO: Xbench found only a few basic errors. It’s real power is hidden in the “error editor”. You can use Tools -> Manage Checklists to create your own error definitions.

In the next article I will come back to the error editor and show you some more great features of Xbench and one other linguistic assurance tool. In the meantime, you can check Xbench in action in my video and try to incorporate it within your localization workflow.

I would also like to thank Cassie for her comment and permission to quote it. Thank you, Cassie!

*I am not affiliated with ApSIC – Xbench developers. I have been a heavy user of Xbench software for several years and find it one of the most useful translation applications on the market. For translators and localizers it has tones of other functions to make their work easier, faster and better.