Apertium-html-tools

Apertium Html-tools is a web application providing a fully localised interface for text/document translation, analyzation, and generation powered by Apertium. Designed to use only static resources for quick and easy deployment with any web server, it is relatively lightweight and user-friendly. It is currently located in SVN at /trunk/apertium-tools/apertium-html-tools. Html-tools relies on an Apertium HTTP API such as Apertium-apy or ScaleMT (to a lesser extent).

Usage

Getting Started

Using Html-tools is as easy as checking out from SVN, changing the configuration and running make.

   $ svn co https://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/apertium-html-tools/
   $ make #To create a blank config file
   $ vim ./config.conf #Edit the configuration, see next section for more details
   $ make

The last step is pointing your web server of choice, e.g. Apache, to serve the directory /build.

Configuration

Since Html-tools is powered entirely by static resources, a configuration file generated at apertium-html-tools/config.conf (based on config.conf.example) is maintained to allow users to modify basic environment parameters such as API location and enabled interfaces. Following is a summary of the currently supported arguments with some examples:

• APY_URL: the URL of an API instance (ScaleMT for basic functionality, APY for full) that the web application calls
  • http://www.apertium.org:8080
  • http://localhost:2737
  • https://api.apertium.org
• ALLOWED_LANGS: an array of ISO 639-3 language codes that the application displays in the interface, useful for deploying websites that have only a subset of all the languages the API being used provides (e.g. a region specific website)
  •   does not limit the languages displayed (everything the API supports will be shown).
  • tur, uzb, kaz'] limits to only language pairs between Turkish, Kazakh, and Uzbek appearing in the interface; analyzers and generators for the three languages will also be shown.
• ALLOWED_VARIANTS: an array of variant codes that the application displays in the interface, similar to the functionality of ALLOWED_LANGS
  •   does not limit the variants displayed (everything the API supports will be shown).
  • US, GB limits languages displayed in the translation interface such that pairs either have no variant or a subset of {'US', 'GB'} (en_US-es and es-en_GB are fine but en_UK-anything is not); analyzers and generators are similarly restricted.
• ENABLED_MODES: an array of the enabled interfaces, a non-empty subset of ['translation', 'analyzation', 'generation', 'sandbox']
  • translation limits the interface to only translation support.
  • translation, analyzation, generation displays the three main interfaces as tabs.
  • translation, analyzation, generation, sandbox is generally not recommended outside of a development environment since users most probably do not need an API Sandbox.
• DEFAULT_MODE: the default landing interface for a new user, an element of the ENABLED_MODES array
  • translation lets the user land on the translation interface first (recommended option).
• SHOW_NAVBAR: whether the navigation bar that allows switching between interfaces is visible.
  • True shows the navbar and is recommended when more than one mode is enabled via ENABLED_MODES.
  • False hides the navbar and is recommended when only one mode is enabled (e.g. ENABLED_MODES = translation).
• GOOGLE_ANALYTICS_PROPERTY: the Google Analytics property name
  • undefined disables Google Analytics (recommended unless you know what you're doing).
• GOOGLE_ANALYTICS_TRACKING_ID: the Google Analytics tracking code
  • undefined disables Google Analytics (recommended unless you know what you're doing).
• LIST_REQUEST_CACHE_EXPIRY: cache expiry time in hours for /list requests (/listPairs, /list?q=analyzers, /list?q=generators)
  • 24 ensures list requests are refreshed at least every day.
• LANGUAGE_NAME_CACHE_EXPIRY: cache expiry time in hours for /languageName requests
  • 24 ensures language name caches are refreshed at least very 24 hours.
• LOCALIZATION_CACHE_EXPIRY: cache expiry time in hours for localised string asset requests (e.g. /assets/strings/fra.json)
  • 24 ensures the interface strings for a locale are refreshed at least every day (browser caching may interfere).

Minification

is currently not done, since enabling gzip compression on the server gives almost-as-good results with easier debuggability, less build complexity and less dependencies. (The old minification build rules are in revision 52413.) Files are still named min.css/min.js even though they're not minified.

However, JS and CSS files are concatenated on typing make, which makes for fewer requests.

Production

In a production environment, make will generate all the necessary files including the build/index.html linking to the concatenated JS and CSS. So, see the instructions in the above section and you should be set for deployment.

Development

Since running make generates an index.html with links to concatenated content, development is a little bit more complicated. After making changes to any CSS or JS resource and before re-running make, the changes will not be reflected in / since index.html still links to the stale minified CSS or JS resources. While this could be amended by running make after every change, that's a terrible workflow unless it's automated somehow (e.g. in emacs you would have to put (add-hook 'after-save-hook 'compile) in ~/.emacs). To alleviate this problem, running make also generates index.debug.html which links to the raw CSS/JS resources that you modify. So, use /index.debug.html while debugging/adding features in the CSS or JS. Modifying the HTML does require running make every time but such changes are less frequent and usually involve less debugging. If you add a static resource (e.g. a new JS file for a spell checker interface), be sure to edit the Makefile and debug-head.html to ensure the concatenation process still runs smoothly.

Current Status

Features

Translation

Text translation is supported through an intuitive interface with support for instant translation (translation as you type) and plans to support a setting for disabling the marking of unknown tokens. Language detection is supported with CLD with plans to migrate to a solution that supports all the Apertium languages. Document translation is available for almost all modern browsers and supports less than 32MB files in DOCX, XLSX, PPTX, ODT, TXT, RTF, HTML, and LATEX formats with plans to support older Microsoft Office formats (DOC, XLS, PPT).

Morphological Analysis and Generation

Morphological analysis and generation are supported via two language selectors, one for the language desired (e.g. English) and another for the Apertium package used (e.g. English-Spanish or English-Esperanto).

Deployed Examples

There are currently two deployed instances of Apertium-apy and Apertium-html tools

Apertium

Apertium maintains html-tools at www.apertium.org and APY at apertium.org:8080 with support for most of the current trunk pairs. However, since it only contains the translation interface, it is not a good source of context for contributors looking to localise the interface.

Apertium Turkic

Apertium Turkic maintains html-tools at turkic.apertium.org and APY at turkic.apertium.org:2737 with support for several Turkic languages and the translation, analyzation, and generation modes available. As such, it is a good candidate for getting localisation string context.

Future Improvements

Localisation

The Html-tools interface is fully localisable and has support for RTL languages. The 23 currently available languages are Aragonese (arg), Avar (ava), Catalan (cat), English (eng), Basque (eus), French (fra), German (deu), Hebrew (heb), Karakalpak (kaa), Kazakh (kaz), Kyrgyz (kir), Nynorsk Norwegian (nno), Bokmål Norwegian (nob), Occitan (oci), Portuguese (por), Romanian (ron), Saami (sme), Spanish (spa), Tatar (tat), Thai (tha) and Chinese (zho).

Contributing a new translation of the interface is encouraged and a very simple process.

Creating a new translation

1. Checkout the directory with all the localisation files: svn co https://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/apertium-html-tools/assets/strings/.
2. Look at a deployed version of the html-tools interface, preferably one with all/most modes enabled. See the section on deployed sites for where to go.
3. • Starting from an existing translation (with reference)
   1. Run ./localisation-tools.py new fin -p eng, replacing fin with your language's ISO 639-3 code and eng with the language you want to use as a reference while translating.
   2. Replace all the right-hand values in each pair to match your language, use the website you found from Step 2 and the reference language translations to help with context.
   • Starting from scratch (without reference)
   1. Run ./localisation-tools.py new fin -b, replacing fin with your language's ISO 639-3 code.
   2. Replace all the right-hand values in each pair to match your language, use the website you found from Step 2 to help with context.
4. Run ./localisation-tools.py all fin, replacing fin with your language's ISO 639-3 code and see whether completion is 100 (percent). If not, keep translating (what you're missing is shown in missing) and running this step's command.
5. Optional: Add your name to the "authors" key and update the "last-updated" key. Add a line to assets/strings/locales.json with the language code mapped to the language's native name (endonym).
6. Commit to SVN: svn ci -m "Your message..."
7. If you didn't edit locales.json, let us know in IRC so that we can edit it for you and update websites. Thanks!

Updating an existing translation

1. Checkout the directory with all the localisation files: svn co https://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/apertium-html-tools/assets/strings/.
2. Look at a deployed version of the html-tools interface, preferably one with all/most modes enabled. See the section on deployed sites for where to go.
3. • Working with a reference
   1. Run ./localisation-tools.py all fin -p eng, replacing fin with your language's ISO 639-3 code and eng with the language you want to use as a reference while translating.
   2. Change all the right-hand values in each pair that are marked as %%UNAVAILABLE%% to match your language, use the website you found from Step 2 and the reference language translations to help with context.
   • Working without a reference
   1. Run ./localisation-tools-py all fin -b, replacing fin with your language's ISO 639-3 code.
   2. Change all the right-hand values in each pair that are marked as %%UNAVAILABLE%% to match your language, use the website you found from Step 2 to help with context.
4. Run ./localisation-tools.py all fin, replacing fin with your language's ISO 639-3 code and see whether completion is 100 (percent). If not, keep translating (what you're missing is shown in missing) and running this step's command.
5. Optional: Add your name to the "authors" key (if it's not there already) and update the "last-updated" key.
6. Commit to SVN: svn ci -m "Your message...". Thanks!

Progress

Updated 2 Dec 2014, r57885. Generated by progresstable.sh.

code	CBE*	CBC**
eng	100%	100.00%
nno	100%	99.89%
deu	98%	112.76%
cat	98%	109.23%
kaa	98%	107.44%
nob	98%	100.40%
tha	98%	92.96%
fin	98%	92.64%
spa	90%	36.45%
kir	87%	34.47%
fra	85%	106.99%
oci	85%	103.43%
uig	85%	95.65%
kaz	85%	94.94%
heb	85%	83.73%
tat	72%	20.49%
sme	68%	22.68%
por	64%	20.36%
eus	59%	22.81%
rus	57%	20.70%
ava	57%	19.30%
ron	53%	17.99%
arg	53%	16.75%
zho	53%	5.25%

 *CBE: completion by entries
 **CBC: completion by characters

Credits

Apertium-html-tools grew out of simple-html as features were being added by Sushain Cherivirala during GCI 2013. It's now its own beast, and continues to be maintained primarily by Sushain.

The following people have helped with localisation:

• Unhammer (Nynorsk Norwegian, Bokmål Norwegian, Saami)
• firespeaker (Kyrgyz, Kazakh, Russian, Tatar)
• spectie (Spanish, Catalan, Aragonese)
• Ilnar (Russian, Tatar)
• Sushain (Spanish)
• Aida (Kazakh)
• hbwashington (Portuguese, Spanish)
• solinus (Avar, Russian)
• shopskasalata/kikero (Romanian)
• Cédric VALMARY (French)
• Lene Antonsen (Saami)
• Tolgonay (Kyrgyz)
• wei2912 (Chinese)
• Märdan (Uyghur)
• Flammie (Finnish)
• inariksit (Finnish)
• youssef (French)
• n0nick (Hebrew)