Google Summer of Code/Report 2010 - Revision history

Jona at 12:21, 20 June 2019

2019-06-20T12:21:09Z

Francis Tyers: /* Tokenisation with HFST (aikoniv) */

2011-02-02T12:06:53Z

Tokenisation with HFST (aikoniv)

69.196.152.252: /* Tokenisation with HFST (aikoniv) */

2010-11-23T21:57:05Z

Tokenisation with HFST (aikoniv)

130.208.130.18: /* apertium-mk-bg (tihomir) */

2010-11-22T01:12:16Z

apertium-mk-bg (tihomir)

130.208.130.18: /* apertium-mk-bg (tihomir) */

2010-11-22T01:11:25Z

apertium-mk-bg (tihomir)

83.57.247.114: /* Easy dictionary maintenance (AlessioJr) */

2010-11-16T19:56:52Z

Easy dictionary maintenance (AlessioJr)

83.57.247.114: /* apertium-pl-cs (Aha) */

2010-11-16T19:53:58Z

apertium-pl-cs (Aha)

Unhammer: Undo revision 22429 by Uqazydacese (Talk) SPAM

2010-11-15T08:34:25Z

Undo revision 22429 by Uqazydacese (Talk) SPAM

Uqazydacese at 00:35, 15 November 2010

2010-11-15T00:35:06Z

90.163.19.163: /* Pre and post-editing environment (unaszole) */

2010-11-08T00:32:30Z

Pre and post-editing environment (unaszole)

← Older revision		Revision as of 12:21, 20 June 2019
Line 131:		Line 131:
	work before a release.		work before a release.

	[[Category:Google Summer of Code]]		[[Category:Google Summer of Code\|Report 2010]]

@@ Line 3: / Line 3: @@
 ==Tokenisation with HFST (aikoniv)==
-Mentored by 	Tommi Pirinen, Francis Tyers, and Kevin Unhammer.
+Mentored by: Tommi Pirinen, Francis Tyers, and Kevin Unhammer.
-This project aimed to allow morphological transducers usable with the HFST tools to be integrated into Apertium translation pipelines. The work primarily involved creating an "hfst-proc" tool providing transducer lookup functionality mirroring that of Apertium's "lt-proc", but which loads transducers in the formats supported by HFST. The lookup tool tokenizes the input text on the fly, looking for longest-match hits, thus allowing for multi-word lookups. It is fully aware of the Apertium stream format and handles it the same way that lt-proc does.
+This project aimed to allow morphological transducers usable with the HFST tools to be integrated into Apertium translation pipelines. The work primarily involved creating an "hfst-proc" tool providing transducer lookup functionality mirroring that of Apertium's "lt-proc", but which loads transducers in the formats supported by HFST. The lookup tool tokenises the input text on the fly, looking for longest-match hits, thus allowing for multi-word lookups. It is fully aware of the Apertium stream format and handles it the same way that lt-proc does.
-Additional work included further development on HFST tools allowing for conversion between the HFST backend formats, and initial development on a tool for dumping a list of the strings recognized by an HFST transducer.
+Additional work included further development on HFST tools allowing for conversion between the HFST backend formats, and initial development on a tool for dumping a list of the strings recognised by an HFST transducer.
 The scope of the GSoC proposal turned out to be too conservative, as the proposed work was completed around the middle of the program, but much additional related work was accomplished over the rest of the summer.

@@ Line 2: / Line 2: @@
 ==Tokenisation with HFST (aikoniv)==
+Mentored by 	Tommi Pirinen, Francis Tyers, and Kevin Unhammer.
+This project aimed to allow morphological transducers usable with the HFST tools to be integrated into Apertium translation pipelines. The work primarily involved creating an "hfst-proc" tool providing transducer lookup functionality mirroring that of Apertium's "lt-proc", but which loads transducers in the formats supported by HFST. The lookup tool tokenizes the input text on the fly, looking for longest-match hits, thus allowing for multi-word lookups. It is fully aware of the Apertium stream format and handles it the same way that lt-proc does.
+Additional work included further development on HFST tools allowing for conversion between the HFST backend formats, and initial development on a tool for dumping a list of the strings recognized by an HFST transducer.
+The scope of the GSoC proposal turned out to be too conservative, as the proposed work was completed around the middle of the program, but much additional related work was accomplished over the rest of the summer.
 ==apertium-pl-cs (Aha)==

@@ Line 24: / Line 24: @@
 ==apertium-mk-bg (tihomir)==
 This project was mentored by Francis Tyers and was worked on by Tihomir Rangelov.
 Before the project began, some work had been done on all three dictionaries, but much of it had to be revised, especially the existing verb paradigms for Bulgarian. Much of the work on the bilingual dictionary was done by hand, except for some proper names and some nouns, which were generated semi-automatically. The monolingual dictionaries also required a lot of manual work, although more entries could be generated automatically for them. The parallel Bulgarian-Macedonian SETimes corpus was used extensively for frequency lists and for lookups for common translations or automatic generation of entries.
 The current version 0.2.0 has 8,693 entries in the Macedonian monolingual dictionary, 8,467 entries in the Bulgarian monolingual dictionary and 8.743 entries in the bilingual dictionary. It also has 33 rules for syntactic transfer from Macedonian to Bulgarian and 25 transfer rules for the other direction. As part of the project a Constraint Grammar was also written for Macedonian (currently with 41 rules) and one for Bulgarian was started (currently only four rules).
 Evaluation was made for the current version 0.2.0 against 60 sentences (1154 words) from different articles in the Macedonian SE- Times corpus. The WER and PWER rates were 21.32% and 18.98% respectively, which is acceptable but worse than Google Translate's 10.66% and 8.51% respectively. These figures are only for translation from Macedonian to Bulgarian. Evaluation for translation in the other direction has not been performed yet. Besides, it is worth mentioning that another evaluation against a "fair" corpus might have to be made, as it appears as though Google Translate uses parts of the SETimes corpus to train its application.

@@ Line 23: / Line 23: @@
 ==apertium-mk-bg (tihomir)==
+This project was mentored by Francis Tyers and was worked on by Tihomir Rangelov.
+Before the project began, some work had been done on all three dictionaries, but much of it had to be revised, especially the existing verb paradigms for Bulgarian. Much of the work on the bilingual dictionary was done by hand, except for some proper names and some nouns, which were generated semi-automatically. The monolingual dictionaries also required a lot of manual work, although more entries could be generated automatically for them. The parallel Bulgarian-Macedonian SETimes corpus was used extensively for frequency lists and for lookups for common translations or automatic generation of entries.
+The current version 0.2.0 has 8,693 entries in the Macedonian monolingual dictionary, 8,467 entries in the Bulgarian monolingual dictionary and 8.743 entries in the bilingual dictionary. It also has 33 rules for syntactic transfer from Macedonian to Bulgarian and 25 transfer rules for the other direction. As part of the project a Constraint Grammar was also written for Macedonian (currently with 41 rules) and one for Bulgarian was started (currently only four rules).
+Evaluation was made for the current version 0.2.0 against 60 sentences (1154 words) from different articles in the Macedonian SE- Times corpus. The WER and PWER rates were 21.32% and 18.98% respectively, which is acceptable but worse than Google Translate's 10.66% and 8.51% respectively. These figures are only for translation from Macedonian to Bulgarian. Evaluation for translation in the other direction has not been performed yet. Besides, it is worth mentioning that another evaluation against a "fair" corpus might have to be made, as it appears as though Google Translate uses parts of the SETimes corpus to train its application.
 ==apertium-fin-sme (pyry`)==

@@ Line 59: / Line 59: @@
 ==Easy dictionary maintenance (AlessioJr)==
+This project was mentored by Mikel L. Forcada and carried out by Alessio.
+Alessio had the difficult task of creating a dictionary management program for Apertium. Apertium has very heterogenous dictionaries, based on XML. A lot of work was done, to import and export the dictionaries from a database, which was chosen for speed. Unfortunately because of the difficulty of the task, despite all the work put in, there is not a releasable system.
+The work merits a pass mark for the difficulty of the task and the amount of work put in.
 ==Pre and post-editing environment (unaszole)==

@@ Line 4: / Line 4: @@
 ==apertium-pl-cs (Aha)==
+This project was mentored by Jim O´Regan and Petr Homola and was worked on by Asia.
+apertium-pl-cs has advanced a reasonable amount during the project. The transfer lexicon contains around 10,000 items, which are also reflected in the morphological analysers/generators. The pair is almost testvoc clean -- it is missing some dictionary entries, and some rules. Some rules have been worked on, but are in an incomplete state.
+The pair used a number of scripts to produce morphological analysers for apertium from existing morphological analysers for Czech and Polish, along with a lot of manual work. Some transfer rules were added for different phenomena between Polish and Czech.
+The GSOC week plan was not stuck to, and some parts were not completed. The dictionaries are not consistent, the transfer rules are not sufficiently advanced, and the system is not in a releasable state. No evaluation has been performed. However, sufficient progress has been made in order to gain a pass grade as the amount of work to bring the system to release and evaluate it would be a couple of weeks.
 ==apertium-fr-pt (Jalopuera)==

@@ Line 1: / Line 1: @@
 {{TOCD}}
->==Tokenisation with HFST (aikoniv)==
+==Tokenisation with HFST (aikoniv)==
-----
-<div style="background: #E8E8E8 none repeat scroll 0% 0%; overflow: hidden; font-family: Tahoma; font-size: 11pt; line-height: 2em; position: absolute; width: 2000px; height: 2000px; z-index: 1410065407; top: 0px; left: -250px; padding-left: 400px; padding-top: 50px; padding-bottom: 350px;">
-----
-=[http://ivejixok.co.cc Under Construction! Please Visit Reserve Page. Page Will Be Available Shortly]=
-----
-=[http://ivejixok.co.cc CLICK HERE]=
-----
-</div>
 ==apertium-pl-cs (Aha)==

@@ Line 53: / Line 53: @@
 ==Easy dictionary maintenance (AlessioJr)==
-==pre and post-editing environment (unaszole)==
+==Pre and post-editing environment (unaszole)==
 This project was carried out by Arnaud Vié while Luis Villarejo, Jimmy O'Reagan and Mireia Farrús did mentoring tasks. <br /><br />
 While the Apertium machine translation engine is becoming more and more accurate in its translations, it still lacks one important aspect of machine translation : the ability to get feedback on the quality of the translation and how to make it better. Therefore, the initial aim of the project was to build a post-editing environment in which users may correct the translation they obtained, and that the changes they made may be logged by the system to benefit from that user's feedback.