User:Irene/proposal - Revision history

Irene: work plan suggestions

2017-04-03T20:53:09Z

work plan suggestions

Irene: added category

2017-04-02T17:15:03Z

added category

Irene at 17:12, 2 April 2017

2017-04-02T17:12:19Z

Show changes

Irene: /* Which of the published tasks are you interested in? What do you plan to do? */ italics formatting

2017-03-31T07:11:32Z

Which of the published tasks are you interested in? What do you plan to do?: italics formatting

Irene: Created page with " == Contact Info == '''Name:''' Irene Tang
'''E-mail:''' itang1@swarthmore.edu
'''IRC nick:''' irene_
'''Location:''' Pennsylvania, USA / California, USA..."

2017-03-31T07:06:53Z

Created page with " == Contact Info == '''Name:''' Irene Tang '''E-mail:''' itang1@swarthmore.edu '''IRC nick:''' irene_ '''Location:''' Pennsylvania, USA / California, USA..."

New page

== Contact Info ==

'''Name:''' Irene Tang 
'''E-mail:''' itang1@swarthmore.edu 
'''IRC nick:''' irene_ 
'''Location:''' Pennsylvania, USA / California, USA 
'''Time zone:''' UTC -05:00 / UTC-08:00 

== Why are you are interested in machine translation? / Why are you are interested in Apertium? ==

I became interested in machine translation earlier in this school year when I was introduced to an organisation that works to translate the Bible for people interested in reading it—in particular, people who speak minority languages in which the text is not currently available. The representative mentioned that the translation process would be made exponentially easier and faster if only they had a computer program that could do a first-pass translation for linguists to reference, rather than starting from scratch by hand. This is a particular cause that I care about; and I’m sure there are many other groups and individuals who would appreciate machine translation as a handy supplement to their endeavors. I figured I could use my background in computer science and linguistics to contribute towards building up machine translation tools for the public to use.

I apply to Apertium because I believe in its success. Apertium is currently one of more successful translation endeavors—and while it lacks the data and traffic that is available to Google Translate, it stands out from corporate undertakings by being open-source and by catering towards uncommon, lesser-resourced languages. From my interactions on the IRC I’ve also noticed an active community of dedicated linguists/programmers, and I’ve read about how much Apertium has accomplished since its birth in 2004. I’m excited for Apertium’s mission.

== Which of the published tasks are you interested in? What do you plan to do? ==

'''Discontiguous Multiwords''' 
For an overview of Apertium’s discontiguous multiwords problem, consider the following set of sentences:

:# I take out the rubbish.
:# I take the rubbish out.
:# Saco la basura.
:# <nowiki>*</nowiki>Tomo la basura fuera.

Discontiguous multiwords are multi-word expressions that are separated by something in the middle. In the set of sentences above, “take out” is a multiword verb. When it is separated by the noun phrase “the rubbish,” it becomes a discontinuous multiword.

Apertium currently doesn’t offer support for discontinuous multiwords, and this is a source of many unfortunate translation errors. Apertium can seamlessly translate (1) into (3) from English to Spanish: in (1), the whole phrasal verb take out is together, so Apertium can easily recognize and translate it as one unit. Take out correctly becomes saco, its first-person conjugation in Spanish. However, Apertium imperfectly translates (2) into (4) from English to Spanish: in (2), the phrasal verb take out is separated by the NP the rubbish, so Apertium doesn’t recognize it as a unit and incorrectly translates it as two separate words. Take becomes tomo and out becomes fuera, independently, which is not what we want. This demonstrates that discontiguous multiwords produce significant wrinkles in the translation process.

My plan is to improve the multiwords processor into being able to recognize when sentences contain discontiguous multiwords, and then reorder the sentence structure so that the whole verb phrase is placed together before bilingual dictionary lookup occurs. As noted in the wiki page for this project, this involves (1) creating a typology of discontinuous multiword expressions in some Germanic, Celtic, Romance, Turkic, and Uralic languages; (2) creating a module for recognising and reordering discontiguous multiword expressions; and (3) supporting discontiguous multiwords for specifically the English-Spanish pair.

== How and who will it benefit in society? ==

Discontiguous multiwords are common in Germanic, Celtic, Romance, Turkic, and Uralic languages. These groups make up the majority of Apertium’s language database. All Apertium users of these five language groups stand to benefit from this project.

== Why should Google and Apertium sponsor it? ==

This issue is rather large, but the solution is within close reach and it provides generous rewards. Discontinuous multiwords are quite common in everyday speech (for those languages that they appear in), so fixing the problem will generously improve translation quality across the board. The discontiguous multiwords problem should be addressed the sooner the better; but this project has been sitting in the GSoC ideas tank on the wiki since 2010.

== Work Plan ==

'''Community bonding period''' – (begin typology) 

'''Part I: preparing data''' – create a typology of different types of discontinuous multiword expressions in Germanic, Celtic, Romance, Turkic, and Uralic languages. This helps with getting an idea of how to build a module in part II. I estimate that it would take 2-5 days to investigate multiword expressions in each language, depending on how familiar I am with the language. I chose the following languages for their significance in Apertium’s database and for my accessibility to them. I’m more familiar with the Romance languages than the others. Creating typologies for roughly 10-12 languages would take up at least a hefty month’s worth of time; I plan to start on Part I during the community-bonding period.

*Week 1 (5/22): Germanic- English, Swedish | Celtic- Welsh
*Week 2 (5/29): Romance- Portuguese, Spanish, French
*Week 3 (6/5): Romance- Italian, Romanian
*Week 4 (6/12): Turkic- | Uralic- Finnish
''Deliverable #1:'' typologies of discontiguous multiword expressions for 10-12 languages currently supported by Apertium, with at least one from each of the five language categories.

'''Part II: building the module''' – create a module/script for recognising and reordering discontiguous multiword expressions

*Week 5 (6/19):
*Week 6 (6/26):
*Week 7 (7/3):
*Week 8 (7/10):
''Deliverable #2:'' functioning discontiguous multiword processor, not yet integrated into Apertium

'''Part III:''' - integrating the module into Apertium (insert between Apertium-pretransfer and lt-proc-b)

*Week 9 (7/17):
*Week 10 (7/24):
*Week 11 (7/31): include support for discontiguous multiwords in specific pairs
*Week 12 (8/7): include support for discontiguous multiwords in specific pairs
''Project completed:'' typologies and fully-integrated module for processing discontiguous multiwords

*Week 13 (8/14): testing
*Week 14 (8/21): pencils down

== List your skills and give evidence of your qualifications. ==

I’m a second-year Computer Science major and Linguistics minor at Swarthmore College (United States). English is my native language and Spanish is a language that I studied for four years in high school.
*Relevant coursework: Data Structures/Algorithms, Computer Systems, Algorithm Analysis, Artificial Intelligence/Machine Learning, Syntax
*Technical skills: Python, C++, C, Java
*Coding challenges: https://github.com/irene-tang/discontiguous-multiwords (information is in the README)

== List any non-Summer of Code plans you have for the summer. ==

If my project is accepted, then my plan is to complete GSoC and take some light elective course somewhere, either online or at a community college.

@@ Line 26: / Line 26: @@
 Discontiguous multiwords are multi-word expressions that are separated by something in the middle. In the set of sentences above, ''take out'' is a multiword verb. When it is separated by the noun phrase ''the rubbish'', it becomes a discontinuous multiword.
-Apertium currently doesn’t offer support for discontinuous multiwords, and this is a source of many unfortunate translation errors. ''Take out'' is a multiword in English, but its Spanish translation ''sacar'' is not. Apertium can seamlessly translate (1) into (3) from English to Spanish: in (1), the whole phrasal verb ''take out'' is together, so Apertium can easily recognise and translate it as one unit. ''Take out'' correctly becomes ''saco'', its first-person conjugation in Spanish. However, Apertium imperfectly translates (2) into (4) from English to Spanish: in (2), the phrasal verb ''take out'' is separated by the NP ''the rubbish'', so Apertium doesn’t recognise it as a unit and incorrectly translates it as two separate words. ''Take'' becomes ''tomo'' and ''out'' becomes ''fuera'', independently, which is not what we want; ''tomar fuera'' cannot be used interchangeably with ''sacar''. This demonstrates that discontiguous multiwords produce significant wrinkles in the translation process.
+Apertium currently doesn’t offer support for discontinuous multiwords, and this is a source of many unfortunate translation errors. ''Take out'' is a multiword in English, but its Spanish translation ''sacar'' is not. Apertium can seamlessly translate (1) into (3) from English to Spanish: in (1), the whole phrasal verb ''take out'' is together, so Apertium can easily recognise and translate it as one unit. ''Take out'' correctly becomes ''saco'', its first-person conjugation in Spanish. However, Apertium imperfectly translates (2) into (4) from English to Spanish: in (2), the phrasal verb ''take out'' is separated by the NP ''the rubbish'', so Apertium doesn’t recognise it as a unit and incorrectly translates it as two separate words. ''Take'' becomes ''tomo'' and ''out'' becomes ''fuera'', independently, which is not what we want; ''tomar fuera'' and''sacar'' cannot be used interchangeably. This demonstrates that discontiguous multiwords produce significant wrinkles in the translation process.
 My plan is to eliminate such errors by improving the multiwords processor into being able to recognise when sentences contain discontiguous multiwords, and then reorder the sentence structure so that the whole verb phrase is placed together before bilingual dictionary lookup occurs. For the set of sentences above, the processor should be able to recognise the discontinuous ''take___out'' in (2) and rearrange the sentence to look like the ''take out___'' in (1).
@@ Line 38: / Line 38: @@
 </pre>
-As noted in the [http://wiki.apertium.org/wiki/Ideas_for_Google_Summer_of_Code/Discontiguous_multiwords#Tasks wiki page for this project], this involves (1) creating a typology of discontinuous multiword expressions in some Germanic, Celtic, Romance, Turkic, and Uralic languages; (2) creating a module for recognising and reordering discontiguous multiword expressions; and (3) providing support for discontiguous multiwords in some existing language pairs.
+As noted in the [http://wiki.apertium.org/wiki/Ideas_for_Google_Summer_of_Code/Discontiguous_multiwords#Tasks wiki page for this project], this involves (1) creating a typology of discontinuous multiword expressions in some Germanic, Celtic, Romance, Turkic, and Uralic languages; (2) creating a module for recognising and reordering discontiguous multiword expressions; and (3) providing support for discontiguous multiwords in some existing language pairs. See work plan for details.
 == How and who will it benefit in society? ==
@@ Line 51: / Line 51: @@
 '''Community bonding period''' <br />
+*Understand dix formats
 *Understand the current multiwords processor module
 *Understand the features of lt-toolbox
 *Devise a typology format, create a typology for types of English phrasal verbs / discontiguous multiword expressions
+*Devise a method for coding typologies into Apertium files
 '''Part I: preparing data''' <br />
-Create a typology of different types of discontinuous multiword expressions in some Germanic, Celtic, Romance, Turkic, and Uralic languages. This is necessary for getting an idea of how to build a module in part II. I estimate that it would take 2-4 days to complete full investigations of multiword expressions in each language, depending on how familiar I am with the language. I chose the following languages for their significance in Apertium’s database and for my accessibility to/familiarity with them.
+Create a typology of different types of discontinuous multiword expressions in some Germanic, Celtic, Romance, Turkic, and Uralic languages. This is necessary for getting an idea of how to build a module in part II. The typologies should include an analysis of ones that cannot be created by the multiwords processor. Each week's typologies should be coded into Apertium files by the end of the weekend. I estimate that it would take 2-4 days to complete full investigations of multiword expressions in each language, depending on how familiar I am with the language. I chose the following languages for their significance in Apertium’s database and for my accessibility to/familiarity with them.
 *Week 1 (5/22): Romance- Spanish, Portuguese
 *Week 2 (5/29): Romance- Italian, Romanian
@@ Line 64: / Line 66: @@
 '''Part II: building the module'''
+The module should respect discontiguous multiwords that may remain discontiguous in both languages. If we are trying to translate a discontiguous multiword from xxx —> yyy, and it is well-formed in language yyy for the word to be discontiguous, then the output translation should allow it to remain discontiguous. Otherwise, the sentence should be reordered and the word should be translated as a single unit.
 *Week 5 (6/19): devise a module for recognising multiword expressions in each of the languages that I created typologies for, write unit tests to make sure it is functioning correctly
+**Devise a compatible format for integrating the expressions into monodix: figure out how to annotate the words with respect to the current multiwords processor
 *Week 6 (6/26): (cont.)
 *Week 7 (7/3): write a script to have the module reorder sentences to unify discontiguous multiwords, write unit tests to make sure it is functioning correctly
+**Devise a method for integrating the script with Apertium
 *Week 8 (7/10): (cont.)
 ''Deliverable #2:'' functioning discontiguous multiword processor, not yet integrated into Apertium
 '''Part III: integrating the module into Apertium'''
-*Week 9 (7/17):  insert the module between Apertium-pretransfer and lt-proc-b, testing
+*Week 9 (7/17): "insert the module between Apertium-pretransfer and lt-proc-b, testing", is what the wiki says
 *Week 10 (7/24): (cont.)
 *Week 11 (7/31): include support for discontiguous multiwords in specific pairs
 *Week 12 (8/7): (cont.)
-''Project completed:'' typologies and fully-integrated module for processing discontiguous multiwords
+''Project completed:'' fully-integrated typologies and  module for processing discontiguous multiwords
 *Week 13 (8/14): testing

← Older revision		Revision as of 17:15, 2 April 2017
Line 90:		Line 90:

	If my project is accepted, then my plan is to complete GSoC and take some light elective course somewhere, either online or at a community college.		If my project is accepted, then my plan is to complete GSoC and take some light elective course somewhere, either online or at a community college.


			[[Category:GSoC_2017_Student_Proposals]]