Tools for TMX

From Apertium
Revision as of 10:28, 11 March 2009 by Francis Tyers (talk | contribs)
Jump to navigation Jump to search

As it is now quite easy to make lots of large TMXes with Bitextor and other tools, it would be good to have some tools for processing them. There are various things that it would be useful to do. For example, given a TMX file:

  • strip out translation units for any given two languages. Bitextor and other tools generate TMX files with many possible combinations of languages, it would be good to be able to say, for a file of "no is en da sv", just give me all TUs which are "en-da".
  • strip out duplicate translation units. Kind of like a uniq for TMX.
  • sort the file by: line length, language, etc. sort for TMX.
  • trim the file of dubious TUs — very short translations of long segments, very different punctuation, translations where the translation is exactly the same as the reference, translations which only consist of numbers, etc.

Examples

Good translation units


  <tu tuid="532" datatype="Text">
    <note>norden_org/start/start00a8.html_norden_org/start/start77a6.html</note>
    <tuv xml:lang="en">
      <seg>Prizes for literature, music, film and the environment.</seg>
    </tuv>
    <tuv xml:lang="da">
      <seg>Priser inden for litteratur, musik, film og miljø.</seg>
    </tuv>
  </tu>