Dynamic Web Translation Using Petal::I18N

Andrew Rodland, Developer

http://cleverdomain.org/yapc07/

So what's with the lab coat?

But by night...

Okay, so what's this about?

• Dynamic translation.
• Not machine translation!
• But using computers to reduce work.

Why Translate?

• Because your boss said so.
• Accessibility for customers or business partners who speak other languages.

Techniques

• Duplicate the whole site.
  • Quick and easy, to start with.
  • Duplication is bad.
• Duplicate only "content".
  • A little better, but things still get out of sync.
• Dynamic translation.
  • Obviously, I picked this one.

    Just move on to the next one.

Why MVC?

"What the MVC-for-the-web crowd are really trying to achieve is a clear separation of concerns. Put your database code in one place, your application code in another, your presentation code in a third place. That way, you can chop and change different elements at will, hopefully without affecting the other parts (depending on how well your concerns are separated, of course). This is common sense and good practice."

-- Andy B. Wardley

The Tools: Catalyst

• Catalyst is a framework for writing web applications in Perl.
• Kind of like that Rails thing.
  • (You might have heard of it.)
• Uses the Model-View-Controller pattern.

Catalyst (cont.)

• Very flexible.
  • As many models, views, and controllers as you want.
  • With any code you want.
  • Power of CPAN.
• http://catalyst.perl.org.

Petal

• Petal is a templating system for Perl.
• Yes, another one.
• Based on Zope's TAL.
• Processes XML rather than text.

Petal (cont.)

• Looks like:

  <table petal:condition="true: ponies">
    <tr petal:repeat="pony ponies">
      <td>${pony/name}</td>
      <td>${pony/age}</td>
      <td>${pony/color}</td>
    </tr>
  </table>
  <div petal:condition="false: ponies">
    You don't have any ponies. How sad.
  </div>

• Comes with Petal::I18N.

Gettext

Petal::I18N uses Gnu Gettext, which is the standard for providing localized text strings.

Dynamic Translation

• Suppose you have a website.
• It needs to be usable in multiple languages.

Dynamic Translation (cont.)

Such as English.

Dynamic Translation (cont.)

And Spanish.

Dynamic Translation: Code Changes

• use Petal::I18N;
• What language does the user want?
  • Account preference / manual selection.
  • Accept-Language header.
  • Default.
• Store language in the stash.
  • Load different images.
  • Load different menus.

Dynamic Translation: Tagging

• Templates are created in English.
• Petal parses the XML. Attributes indicate text that should be translated.

<p i18n:translate="">This will be translated.</p>

<img src="pony.jpg" i18n:attributes="alt" alt="So will this." />

Dynamic Translation: String Extraction.

• I18NFool is a utility that scans HTML for these attributes and creates a gettext database.
• XML is "flattened" into translation
• This:

  <p i18n:translate="">
    <b i18n:translate="">Please wait</b>
    while we process your order.
  </p>

• Becomes two translatable strings:
  • "Please wait"
  • "${1} while we process your order."

Dynamic Translation: Human Translation

• Human translators filling in text boxes.
• Trained to work with numeric and variable substitutions.
• Can see their work live.

Dynamic Translation: Human Translation (contd.)

Dynamic Translation: Wrapping up.

• Run i18nfool-update to compile the gettext file to a .mo file.
• Test the site!

Load the site

PlatinumTel Prepaid Wireless

Dynamic Translation: The Pros

• No changes to business logic.
• No duplication of effort.
• Almost no extra work when writing templates.
  • (In fact, you can script the tagging.)

Dynamic Translation: The Cons

• It's slow.
• Method calls and hash lookups.
• XML flattening.
• Every text element, every page, every hit.
• Acceptable on a low-traffic site or a beefy box.

The Alternative

Dynamic Ahead-of-Time Translation

• Uses Petal::I18N as before.
• Doesn't translate on the fly.
• Instead, run Petal::I18N ahead of time for each language.

Ahead of Time Translation: Code Changes

• Petal::I18N isn't used in the website.
• Detecting user language and any non-template bits remain the same.
• Also, load templates from a different dir depending on language.

Ahead of Time: Still the Same

• Tag strings for translation.
• Generate a .po file.
• Hand it to the translators.
• Compile a .mo database.

Ahead of Time: Template Generation

• Petal::I18N isn't loaded into the site.
• So we have to process the templates ahead of time, with a script.

Ahead of Time: The Pros

• Much faster. Most overhead is completely eliminated.
• Can examine the generated templates.

Ahead of Time: The Cons

• Adds a step to the deployment process.
• More work for translators to view changes live.
• Petal::I18N is funny and eats variables if used offline, needs patch.

Overall Results

• A multilingual website with minimum fuss.
• I had this technique up and testing in a little over a week.
• Now running two production sites using this technique.

Thank You

Questions?

Internationalization Pitfalls

The Obvious One

• I told you how to translate templates and images.
• But text comes from other places too.
  • Databases
  • The internet
  • User content
  • Error messages
• Eek!

Character Encodings

• Considered a black art.
• Not all code passes unicode properly.
• Worse, software lies about what encodings it supports or provides.
• Double eek!

Bonus: Something Cool

I said that the cool thing about Catalyst is CPAN. Well here's a cool "Found on CPAN."

• Catalyst::Plugin::I18N::Request.
• I18Nizes request URIs and params.

• A user browsing in Spanish can go to /usario/busca?nombre=Pedro and get dispatched to the code for /user/search?name=Pedro.

• Maybe more showy than useful, but cool.

Thank You

Questions?

Sample Code

Check this location in the near future for some example code for each of the steps.

http://cleverdomain.org/yapc07/