Bytecode for transfer - Revision history

Unhammer: links

2014-12-09T10:47:05Z

links

Unhammer: cat

2013-09-24T06:47:19Z

cat

Mikel at 17:51, 15 August 2012

2012-08-15T17:51:34Z

Show changes

Bech: Category:Documentation in English

2012-03-24T11:29:18Z

Category:Documentation in English

Jacob Nordfalk: /* Further work */

2012-03-05T12:01:31Z

Further work

98.210.153.3: /* Using it in a language pair */

2011-09-16T23:39:11Z

Using it in a language pair

Jacob Nordfalk at 14:57, 27 March 2010

2010-03-27T14:57:46Z

Jacob Nordfalk: /* Using it in a language pair */

2010-03-07T04:35:06Z

Using it in a language pair

Jacob Nordfalk at 21:47, 1 March 2010

2010-03-01T21:47:14Z

Jacob Nordfalk at 21:40, 28 February 2010

2010-02-28T21:40:22Z

@@ Line 15: / Line 15: @@
 == A concrete example: Esperanto-English ==
 Take a look at
-[http://apertium.svn.sourceforge.net/viewvc/apertium/trunk/lttoolbox-java/testdata/transfer/apertium-eo-en.eo-en.t1x?view=markup apertium-eo-en.eo-en.t1x] and [http://apertium.svn.sourceforge.net/viewvc/apertium/trunk/lttoolbox-java/src/org/apertium/transfer/generated/apertium_eo_en_eo_en_t1x.java?view=markup apertium_eo_en_eo_en_t1x.java] (the same file converted into Java).
+[https://sourceforge.net/p/apertium/svn/HEAD/tree/trunk/lttoolbox-java/testdata/transfer/apertium-eo-en.eo-en.t1x?view=markup apertium-eo-en.eo-en.t1x] and [https://sourceforge.net/p/apertium/svn/HEAD/tree/trunk/lttoolbox-java/src/org/apertium/transfer/generated/apertium_eo_en_eo_en_t1x.java?view=markup apertium_eo_en_eo_en_t1x.java] (the same file converted into Java).
 <s>The Java version is compiled into bytecode</s> (''An equivalent bytecode class would be directly generated now'') and executed with the Java JVM and JIT (Just-in-time) compiler which converts it into machine code during run-time.
@@ Line 66: / Line 66: @@
 Remember to do the same for $(PREFIX2).
-See http://apertium.svn.sourceforge.net/viewvc/apertium?view=rev&revision=21146 for a complete example of the changes.
+See https://sourceforge.net/p/apertium/svn/21146/log/?path=/trunk/lttoolbox-java for a complete example of the changes
 == Further work ==

@@ Line 77: / Line 77: @@
 * Considering that we have a full port lttoolbox, Apertium could be made to run purely on Java, enabling a wide range of platforms, i.a. Windows, phones (J2ME or Android), web pages, server systems. <s>Only the tagger is missing for a full system.</s>
-=== Bytecode optimizations ===
+=== Bytecode optimisations ===
 Although the achieved performance is good and certainly represents a great improvement over interpreted transfer, the generated bytecode is far from being optimal. Having this in mind, it was considered that a Java bytecode optimizer could provide a great improvement in both transfer speed and the size of the resulting bytecode classes, and [http://proguard.sourceforge.net/ ProGuard] was chosen to experimentally test it. Unfortunately, the results achieved after trying with several different configurations were far from the expected ones.
@@ Line 84: / Line 84: @@
 <code>
-java -jar proguard.jar -injars apertium-en-es.jar -outjars apertium-en-es-optimized.jar -libraryjars /home/mikel/developer/java/jdk1.6.0_33/jre/lib/rt.jar -target 6 -keepattributes InnerClasses -optimizationpasses 5 -overloadaggressively -allowaccessmodification -keepclasseswithmembers "class * {public static void main(java.lang.String[]);}" -keep "class org.apertium.interchunk.InterchunkWord" -keep "class org.apertium.transfer.TransferWord" -keep "class transfer_classes.* {public *;}" -assumenosideeffects "class org.apertium.transfer.TransferWord {public java.lang.String tl(org.apertium.transfer.ApertiumRE); public java.lang.String sl(org.apertium.transfer.ApertiumRE);}" -verbose
+java -jar proguard.jar -injars apertium-en-es.jar -outjars apertium-en-es-optimized.jar -libraryjars /home/mikel/developer/java/jdk1.6.0_33/jre/lib/rt.jar -target 6 -keepattributes InnerClasses -optimisationpasses 5 -overloadaggressively -allowaccessmodification -keepclasseswithmembers "class * {public static void main(java.lang.String[]);}" -keep "class org.apertium.interchunk.InterchunkWord" -keep "class org.apertium.transfer.TransferWord" -keep "class transfer_classes.* {public *;}" -assumenosideeffects "class org.apertium.transfer.TransferWord {public java.lang.String tl(org.apertium.transfer.ApertiumRE); public java.lang.String sl(org.apertium.transfer.ApertiumRE);}" -verbose
 </code>
-Using this configuration, [http://proguard.sourceforge.net/ ProGuard] claims to be performing over 10,000 optimizations of different types. For instance, this is the output of the first iteration, which is the most significant one (a total of 5 iterations were performed):
+Using this configuration, [http://proguard.sourceforge.net/ ProGuard] claims to be performing over 10,000 optimisations of different types. For instance, this is the output of the first iteration, which is the most significant one (a total of 5 iterations were performed):
 <pre>
@@ Line 121: / Line 121: @@
   Number of inlined tail recursion calls:      7
   Number of merged code blocks:                37
-  Number of variable peephole optimizations:   2179
+  Number of variable peephole optimisations:   2179
-  Number of arithmetic peephole optimizations: 21
+  Number of arithmetic peephole optimisations: 21
-  Number of cast peephole optimizations:       0
+  Number of cast peephole optimisations:       0
-  Number of field peephole optimizations:      9
+  Number of field peephole optimisations:      9
-  Number of branch peephole optimizations:     712
+  Number of branch peephole optimisations:     712
-  Number of string peephole optimizations:     385
+  Number of string peephole optimisations:     385
   Number of simplified instructions:           207
   Number of removed instructions:              7720
@@ Line 148: / Line 148: @@
 Not a big difference (about 130 KB, less than the 5% of the total file size), but not bad... The funny thing is that the improvement corresponds entirely to [[lttoolbox-java]], since the transfer class files are actually bigger than before! Once extracted, the transfer classes took 379.9 KB before optimizing, and 407.8 KB after it.
-What's about translation speed? Let's see how much it takes translating a single paragraph before and after the optimization:
+What's about translation speed? Let's see how much it takes translating a single paragraph before and after the optimisation:
 <pre>
@@ Line 163: / Line 163: @@
 </pre>
-So it has taken 175 ms less after the optimization... it seems that it has been about 10% faster then! Really? What will happen if we try with a longer input? Let's test it with a big text file (the text file used consists of the same paragraph as before repeated 1000 times, and it takes 378 KB):
+So it has taken 175 ms less after the optimisation... it seems that it has been about 10% faster then! Really? What will happen if we try with a longer input? Let's test it with a big text file (the text file used consists of the same paragraph as before repeated 1000 times, and it takes 378 KB):
 <pre>
@@ Line 177: / Line 177: @@
 </pre>
-So it has taken about 300 ms less after the optimization, that is, it has been almost 1% faster, which is certainly an unnoticeable difference!
+So it has taken about 300 ms less after the optimisation, that is, it has been almost 1% faster, which is certainly an unnoticeable difference!
-All in all, it seems that the time we save after the optimization is practically constant, and it doesn't seem to be too related to the transfer classes. I suspect that it greatly corresponds to the preverification step that [http://proguard.sourceforge.net/ ProGuard] carries out because we target Java 6, and not to the optimizations themselves. In fact, comparing a bytecode fragment before and after the optimization, it seems that there isn't any considerable change. The following corresponds to the non-optimized version:
+All in all, it seems that the time we save after the optimisation is practically constant, and it doesn't seem to be too related to the transfer classes. I suspect that it greatly corresponds to the preverification step that [http://proguard.sourceforge.net/ ProGuard] carries out because we target Java 6, and not to the optimisations themselves. In fact, comparing a bytecode fragment before and after the optimisation, it seems that there isn't any considerable change. The following corresponds to the non-optimized version:
   public void rule99__prep__probj(Writer arg0, TransferWord arg1, String arg2, TransferWord arg3)
@@ Line 253: / Line 253: @@
   }
-In a nutshell, although [http://proguard.sourceforge.net/ ProGuard] certainly makes things slightly better, the difference is so small that it can be considered that it is not worth using it. Later on, we could consider trying another bytecode optimizer, or implementing some optimizations ourselves during bytecode generation.
+In a nutshell, although [http://proguard.sourceforge.net/ ProGuard] certainly makes things slightly better, the difference is so small that it can be considered that it is not worth using it. Later on, we could consider trying another bytecode optimizer, or implementing some optimisations ourselves during bytecode generation.
 [[Category:Development]]
 [[Category:Documentation in English]]
+[[Category:Transfer]]

← Older revision		Revision as of 11:29, 24 March 2012
Line 81:		Line 81:

	[[Category:Development]]		[[Category:Development]]
			[[Category:Documentation in English]]

@@ Line 78: / Line 78: @@
 * A step for post-compiling to native code should be tried out.
 * With http://xmlvm.org/ there could be a way for iPhones as well
-* Considering that we have a full port lttoolbox, Apertium could be made to run purely on Java, enabling a wide range of platforms, i.a. Windows, phones (J2ME or Android), web pages, server systems. Only the tagger is missing for a full system.
+* Considering that we have a full port lttoolbox, Apertium could be made to run purely on Java, enabling a wide range of platforms, i.a. Windows, phones (J2ME or Android), web pages, server systems. <s>Only the tagger is missing for a full system.</s>
 [[Category:Development]]

@@ Line 35: / Line 35: @@
 === Using it in a language pair ===
-Add an entry to modes.xml where you replace "apertium-transfer" with "apertium-transfer-j" and use the .class file instead if the .t1x file.
+Add an entry to modes.xml where you replace "apertium-transfer" with "apertium-transfer-j" and use the .class file instead of the .t1x file.
 For example, replace
@@ Line 52: / Line 52: @@
 Now you can compile manually with
  $ apertium-preprocess-transfer-bytecode-j apertium-eo-en.eo-en.t1x eo-en.t1x.class
 === Adding it to your Makefile ===

@@ Line 45: / Line 45: @@
 with
       <program name="apertium-transfer-j">
-        <file name="apertium_eo_en_eo_en_t1x.class"/>
+        <file name="eo-en.t1x.class"/>
         <file name="eo-en.t1x.bin"/>
         <file name="eo-en.autobil.bin"/>
       </program>
+Now you can compile manually with
-Also add apertium-preprocess-transfer-bytecode-j to Makefile.am, or do it manually:
- $ apertium-preprocess-transfer-bytecode-j apertium-eo-en.eo-en.t1x apertium_eo_en_eo_en_t1x.class
+ $ apertium-preprocess-transfer-bytecode-j apertium-eo-en.eo-en.t1x eo-en.t1x.class
+You can also add optional support for bytecode compilation to Makefile.am:
+Under the lines
+ $(PREFIX1).t1x.bin: $(BASENAME).$(PREFIX1).t1x
+        apertium-validate-transfer $(BASENAME).$(PREFIX1).t1x
+        apertium-preprocess-transfer $(BASENAME).$(PREFIX1).t1x $@
+Add
+        @if [ "`which apertium-preprocess-transfer-bytecode-j`" == "" ]; then echo && echo "NOTE: lttoolbox-java (used for bytecode accelerated transfer) is missing" && echo "      Therefore the following will fail (but it's OK)" && echo; fi
+        -apertium-preprocess-transfer-bytecode-j $(BASENAME).$(PREFIX1).t1x $(PREFIX1).t1x.class
+If lttoolbox-java isnt installed a warning is emitted and compilation continues (so things still work).
+Remember to do the same for $(PREFIX2).
 == Further work ==

@@ Line 9: / Line 9: @@
 This enables
-* Faster transfer (currently factor 5). As startup times are higher this is only feasible when processing 2000 words or more
+* Faster transfer (currently factor 5). As startup times are higher by 0.33 seconds this is only feasible when processing more than 100 sentences (2000 words).
 * Debuggable transfer. Using a Java development tool, for example Netbeans, you can step thru the transfer code line-by-line, inspecting variables and see exactly what is happening
 * [[Validating transfer files]]

@@ Line 24: / Line 24: @@
 Speedup factor: 5.76
 </pre>
 == Using it ==