Apertium-apy/Translation - Revision history

Unhammer: /* servlet.py */

2014-11-06T09:38:41Z

servlet.py

Unhammer: /* servlet.py */

2014-11-06T09:38:29Z

servlet.py

Unhammer: /* translate.py */

2014-11-06T09:33:26Z

translate.py

Unhammer: /* servlet.py */

2014-04-25T16:10:34Z

servlet.py

Unhammer: /* servlet.py */

2014-04-25T16:00:11Z

servlet.py

Unhammer: /* translate.py */

2014-04-25T15:57:14Z

translate.py

Unhammer: Created page with "How translation currently works in APY: ==translate.py== The function translate() is the main entry point to translation. It expects a text string, a lock (threading.Rlock) a..."

2014-04-25T15:16:34Z

Created page with "How translation currently works in APY: ==translate.py== The function translate() is the main entry point to translation. It expects a text string, a lock (threading.Rlock) a..."

New page

How translation currently works in APY:

==translate.py==
The function translate() is the main entry point to translation. It expects a text string, a lock (threading.Rlock) and a pipeline (a pair of input/output file descriptors). The lock is there to make sure only one process can do translation on a certain pipeline at any one time. Each pipeline should have its own Rlock.

The function translateNULFlush() first deformats the text, then sends that into the input fd, then reads the output fd until it sees a NUL byte. Then it reformats and returns. It's wrapped in "with translock" so we make sure we don't read translations based on other people's input …

We could have defined translate() as just <code>return translateNULFlush(toTranslate, translock, pipeline)</code>, but then what if someone sends in a huge text? We'd lock up that pipeline for too long, and everyone else would have to wait. <smaller>Also, we'd fill up the FIFO buffers: since we don't read the output of translation until we've sent in all the input, we would be trying to push in more data into the input file descriptor, but the buffer would be full and the program would hang until someone read off the output file descriptor.</smaller> So to solve that, we split the input text up into chunks, and send one chunk at a time into translateNULflush. So translate() calls translateSplitting() once which calls translateNULFlush() a bunch of times (or only once for very short texts).

@@ Line 9: / Line 9: @@
 ==servlet.py==
-When you start servlet, you choose the number of processes with the -j switch. Each process has its own dict of pipelines and pipeline_locks. So in one servlet process, there might be an TranslateHandler.pipelines[("eng","kaz")]=(fd_in, fd_out) and a corresponding TranslateHandler.pipeline_locks[("eng","kaz")]=threading.Rlock() (this is the pipeline and translock, respectively, sent to translate.py's translate() function).
+When you start servlet, you choose the number of processes with the -j switch. Each process has its own dict of pipelines and pipeline_locks. So in one servlet process, there might be an TranslateHandler.pipelines[("eng","kaz")]=(fd_in, fd_out) and a corresponding TranslateHandler.pipeline_locks[("eng","kaz")]=toro.Lock() (this is the pipeline and translock, respectively, sent to translate.py's translate() function).
 When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started if it does flushing (the function <code>startPipeline()</code>), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started. Non-flushing pairs get the value None.

@@ Line 9: / Line 9: @@
 ==servlet.py==
-When you start servlet, you choose the number of processes with the -j switch. Each process has its own dict of pipelines and pipeline_locks. So in one servlet process, there might be an TranslateHandler.pielines[("eng","kaz")]=(fd_in, fd_out) and a corresponding TranslateHandler.pieline_locks[("eng","kaz")]=threading.Rlock() (this is the pipeline and translock, respectively, sent to translate.py's translate() function).
+When you start servlet, you choose the number of processes with the -j switch. Each process has its own dict of pipelines and pipeline_locks. So in one servlet process, there might be an TranslateHandler.pipelines[("eng","kaz")]=(fd_in, fd_out) and a corresponding TranslateHandler.pipeline_locks[("eng","kaz")]=threading.Rlock() (this is the pipeline and translock, respectively, sent to translate.py's translate() function).
-When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started (the function <code>runPipeline()</code>), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started.
+When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started if it does flushing (the function <code>startPipeline()</code>), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started. Non-flushing pairs get the value None.
+Then it starts the translation, which, like TranslateHandler.get, is a coroutine. Thus we can have several calls to get which get queued up by the ioloop, but each pipeline has a lock (per servlet process). We can at most start one pipeline per pair per servlet process, and requests for the same pipeline have to wait for their turn (which is not too long due to translateSplitting, see above), but we can handle several translation requests for different pipelines in parallell.
-Then it starts the translation in a separate thread, using <code>start_worker()</code> from ThreadableMixin pattern described at http://umumble.com/blogs/server_side_optimization/66/ with the callback <code>handleTranslation()</code>  – this pattern just ensures the servlet process doesn't block when waiting for translation results.
-So each translation request gets a separate thread, but each pipeline has a lock (per servlet process). We can at most start one pipeline per servlet process, and requests for the same pipeline have to wait for their turn (which is not too long due to translateSplitting, see above), but we can handle several translation requests for different pipelines in parallell since they all spawn off new threads.

@@ Line 12: / Line 12: @@
 When you start servlet, you choose the number of processes with the -j switch. Each process has its own dict of pipelines and pipeline_locks. So in one servlet process, there might be an TranslateHandler.pielines[("eng","kaz")]=(fd_in, fd_out) and a corresponding TranslateHandler.pieline_locks[("eng","kaz")]=threading.Rlock() (this is the pipeline and translock, respectively, sent to translate.py's translate() function).
-When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started (the function runPipeline()), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started.
+When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started (the function <code>runPipeline()</code>), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started.
-Then it starts the translation in a separate thread, using the ThreadableMixin pattern described at http://umumble.com/blogs/server_side_optimization/66/ – this just ensures the servlet process doesn't block when waiting for translation results.
+Then it starts the translation in a separate thread, using <code>start_worker()</code> from ThreadableMixin pattern described at http://umumble.com/blogs/server_side_optimization/66/ with the callback <code>handleTranslation()</code>  – this pattern just ensures the servlet process doesn't block when waiting for translation results.
+So each translation request gets a separate thread, but each pipeline has a lock (per servlet process). We can at most start one pipeline per servlet process, and requests for the same pipeline have to wait for their turn (which is not too long due to translateSplitting, see above), but we can handle several translation requests for different pipelines in parallell since they all spawn off new threads.

← Older revision		Revision as of 16:00, 25 April 2014
Line 13:		Line 13:

	When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started (the function runPipeline()), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started.		When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started (the function runPipeline()), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started.

			Then it starts the translation in a separate thread, using the ThreadableMixin pattern described at http://umumble.com/blogs/server_side_optimization/66/ – this just ensures the servlet process doesn't block when waiting for translation results.

@@ Line 7: / Line 7: @@
 We could have defined translate() as just <code>return translateNULFlush(toTranslate, translock, pipeline)</code>, but then what if someone sends in a huge text? We'd lock up that pipeline for too long, and everyone else would have to wait. <smaller>Also, we'd fill up the FIFO buffers: since we don't read the output of translation until we've sent in all the input, we would be trying to push in more data into the input file descriptor, but the buffer would be full and the program would hang until someone read off the output file descriptor.</smaller> So to solve that, we split the input text up into chunks, and send one chunk at a time into translateNULflush. So translate() calls translateSplitting() once which calls translateNULFlush() a bunch of times (or only once for very short texts).
+==servlet.py==
+When you start servlet, you choose the number of processes with the -j switch. Each process has its own dict of pipelines and pipeline_locks. So in one servlet process, there might be an TranslateHandler.pielines[("eng","kaz")]=(fd_in, fd_out) and a corresponding TranslateHandler.pieline_locks[("eng","kaz")]=threading.Rlock() (this is the pipeline and translock, respectively, sent to translate.py's translate() function).
+When TranslateHandler.get is called, it first ensures e.g. eng-kaz is started (the function runPipeline()), the first and last process of the pipeline will be assigned to self.pipelines[("eng","kaz")] when the pipeline is started.

@@ Line 2: / Line 2: @@
 ==translate.py==
-The function translate() is the main entry point to translation. It expects a text string, a lock (threading.Rlock) and a pipeline (a pair of input/output file descriptors). The lock is there to make sure only one process can do translation on a certain pipeline at any one time. Each pipeline should have its own Rlock.
+The function translate() is the main entry point to translation. It expects a text string, a lock and a pipeline (a pair of input/output file descriptors) or a command (for pairs that can't NUL flush). The lock is there to make sure only one process can do translation on a certain pipeline at any one time. Each pipeline should have its own lock.
-The function translateNULFlush() first deformats the text, then sends that into the input fd, then reads the output fd until it sees a NUL byte. Then it reformats and returns. It's wrapped in "with translock" so we make sure we don't read translations based on other people's input …
+The function translateNULFlush() first deformats the text, then sends that into the input fd, then reads the output fd until it sees a NUL byte. Then it reformats and returns. It's wrapped in "with lock" so we make sure we don't read translations based on other people's input …
 We could have defined translate() as just <code>return translateNULFlush(toTranslate, translock, pipeline)</code>, but then what if someone sends in a huge text? We'd lock up that pipeline for too long, and everyone else would have to wait. <smaller>Also, we'd fill up the FIFO buffers: since we don't read the output of translation until we've sent in all the input, we would be trying to push in more data into the input file descriptor, but the buffer would be full and the program would hang until someone read off the output file descriptor.</smaller> So to solve that, we split the input text up into chunks, and send one chunk at a time into translateNULflush. So translate() calls translateSplitting() once which calls translateNULFlush() a bunch of times (or only once for very short texts).
 ==servlet.py==