ScaleMT

Introduction

This is the wiki page of ScaleMT, a scalable architecture to provide translation web services based on Apertium and other machine translation engines. It is based on previous works to develop an Apertium web service: Apertium_scalable_service. The web service has two different APIs: XML-RPC, a lightweight remote procedure call method using XML and HTTP, and JSON REST, that allows you to easily use the service from any website.

Architecture

ScaleMT makes the translation engines more efficient by turning them into daemons (that is, processes running in the background rather than under the interaction of a user). Besides that, it is able to run on multiple servers thanks to an algorithm which decides which daemons should run on each server and a load balancing method that decides which server should process each request. ScaleMT consists of two main Java applications:

• ScaleMTSlave runs on a machine with the translation engine installed and manages a set of running translation engine instances (daemons); it performs the requested translations by sending them to the right daemon.
• ScaleMTRouter (request router) runs on a web server; it processes the translation requests and sends them to the right ScaleMTSlave instance.

Downloading

The source code can be downloaded from our subversion repository:

svn co http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/scaleMT

Fast set up

To compile the source code you'll need:

• A Java Development Kit compatible with Java version 6. It can be Sun's implementation or any other implementation that follows the specification (see [1]).
• Maven. If you don't have Maven installed, simply download it, unzip it, and be sure that the bin directory is in your PATH.

Once you are sure you have Java JDK and Maven, you can compile the three projects you have downloaded:

cd ScaleMTRMIInterfaces
mvn install

cd ScaleMTSlave
mvn package

cd ScaleMTRouter
mvn package

Then, configure ScaleMTRouter. To do so, unzip the file ScaleMTRouter.war, present in ScaleMTRouter/target to whichever directory you want:

cd ScaleMTRouter/target
unzip ScaleMTRouter.war -d /tmp/mywar/

Go to the directory where you unzipped the file and open the file WEB-INF/classes/configuration.properties with your favourite editor. Change the value of the property requestrouter_rmi_host to the public name of the computer where you are going to run the request router. If you are going to run the router with only a slave on the same machine, you don't need to change this property.

References

• "ScaleMT: a free/open-source framework for building scalable machine translation web services". Víctor M. Sánchez-Cartagena, Juan Antonio Pérez-Ortiz. Open Source Tools for Machine Translation, MT Marathon 2010, Dublin, Ireland, 2010. The Prague Bulletin of Mathematical Linguistics 93, p. 97-106. [pdf]
• "An open-source highly scalable web service architecture for the Apertium machine translation engine". Víctor M. Sánchez-Cartagena, Juan Antonio Pérez-Ortiz. First International Workshop on Free/Open-Source Rule-Based Machine Translation, Alicante, Spain, 2009, p. 51-58. [pdf]