User:JCentelles/GSoCapplication - Revision history

JCentelles: /* Coding Challenge */

2013-05-13T05:22:04Z

Coding Challenge

JCentelles: /* Coding Challenge */

2013-05-13T05:21:30Z

Coding Challenge

JCentelles at 07:41, 8 May 2013

2013-05-08T07:41:02Z

JCentelles at 05:14, 3 May 2013

2013-05-03T05:14:49Z

JCentelles at 14:10, 28 April 2013

2013-04-28T14:10:21Z

JCentelles at 13:47, 28 April 2013

2013-04-28T13:47:39Z

JCentelles at 13:44, 28 April 2013

2013-04-28T13:44:17Z

JCentelles at 11:35, 27 April 2013

2013-04-27T11:35:43Z

JCentelles at 06:30, 24 April 2013

2013-04-24T06:30:57Z

JCentelles: Created page with ''''TITLE: Chinese-to-Spanish Apertium System''' •'''Why is it you are interested in machine translation?''' The quantity of information in our society is increasing incredi…'

2013-04-24T06:17:45Z

Created page with ''''TITLE: Chinese-to-Spanish Apertium System''' •'''Why is it you are interested in machine translation?''' The quantity of information in our society is increasing incredi…'

New page

'''TITLE: Chinese-to-Spanish Apertium System'''

•'''Why is it you are interested in machine translation?'''
The quantity of information in our society is increasing incredibly quickly, and most of this information is multilingual, which means that translation plays a very important role to deal with this information. Companies, governments and most people have to face the challenge of communicating in other language that their own. So the question is, who is not interested in having access to a great machine translation system? So, I think developing machine translation systems is a challenge which should have a big impact on society. I am interested in participating on it.

•'''Why is it that you are interested in the Apertium project?'''
I am interested in Apertium mainly because it is a popular, well-known and active open source project and it uses a rule-based translation engine which I want to further familiarize with.

•'''Which of the published tasks are you interested in? What do you plan to do?'''
Introducing Chinese to Spanish translation. About my work plan, you can see further information in the description of the project. Here I just summarize the main ideas. I will start to familiarize and incorporate into the Apertium architecture the given list of GPL resources (I provide) to analyze Chinese and generate Spanish (well, this language is already in Apertium). Then, I will work with an statistical system + postedition to complete the transfer approach.

'''Project description'''

'''MOTIVATION'''

Chinese and Spanish are two of the most spoken languages in the world and there are many economical reasons to pursue this language pair translations (i.e. many China companies interested in expanding to Latin America).

There are not many Chinese-to-Spanish available in the website: google translator is one of them. However, the translation quality is not extremely good. It seems they have to produce translation through English pivoting to compensate the lack of Chinese-Spanish parallel corpora.

Therefore, corpus-based approaches are possible with this language pair, but these approaches do not have extremely good results as other pairs such as English-Spanish because there is not huge parallel data for this language pair. Furthermore, the differences between the two languages increases the difficulty of the translation.

Our expectations of the Chinese-Spanish rule-based system in comparison to other machine translation approaches are the following ones:

1)It will be able to better manage the difference in morphology from Chinese to Spanish. Chinese is an isolating language, which means that there is a one-to-one correspondance between words and morphemes. Whereas, Spanish is a fusional language, which means that words and morphemes are mixed together without clear limits. Making analysis and generation of morphology may be nice to approach Chinese and Spanish.

2)I will experiment if reordering from Chinese and Spanish can benefit from reordering transfer rules.

3)The rule-based can exploite the use of linguistic tools which are available separately for Chinese and Spanish.

'''DESCRIPTION'''

We will introduce the Chinese-to-Spanish translation in the Apertium open source project. We will do this integration using the GPL available tools for the pair of languages. This tools will be used for the analysis and generation steps. Some of the resources that we have include parallel corpus (from which we can extract bilingual dictionaries in case we do not find dictionaries), statistical postaggers and segmenter (from Stanford, for Chinese, see references later in the Apertium questions).

For the transfer phase, we propose to generate rules automatically from a parallel corpus using [3]. We will experimentally validate and post-edit before introducing them into the transfer-based approach. We have already a baseline phrase-based system. As parallel corpus, we will use the United Nations corpus and the Holy Bible corpus [2] in addition to an automatically created parallel corpus from the EPPS (which contains 250 thousand sentences). Further corpus may include some manuals provided by the TAUS company (here, we have to ask if the corpus can be used for this project). Finally, we will put this new language pair available on-line.

'''WHO ARE THE BENEFICIARIES AND HOW WILL THEY BENEFIT?'''

We are focusing on making a first approach from Chinese to Spanish rule-based translation.
Also, as the translator will be available in the web, tourists traveling across China and Latin America can use it. Spanish students from Chinese (or the other way round) can use the translator to test their knowledge.

'''WORK PLAN'''
[[File:grantt2.jpg]]

[[File:grantt.jpg]]

As follows we name the resources that we will be using:

(1) Parallel corpora:
Bible corpus, UN corpus [2] and EPPS corpus translated automatically (no references for this, but it contains 250k parallel sentences).
(2) Stanford segmenter (http://nlp.stanford.edu/software/segmenter.shtml)
(3) Stanford postagger (http://nlp.stanford.edu/software/tagger.shtm)
(4) For Spanish, we will be using reources from Apertium.
(5) Transfer-based rules extractor [3].

APERTIUM QUESTIONS

•'''Can I do a pair with language x and language y ?'''
— Yes, there are no restrictions. But you should take the following into consideration:
(a) Are there existing machine translation (MT) systems for this pair?
Yes there are. The most popular are the Google translator (statistical), Yahoo Babelfish (rule-based) and Microsoft Bing (hybrid).
(b) If there are existing systems, how good are they? -- Could you do better in three months?
Even the existing machine translation systems are well-known, their Chinese to Spanish translations are quite poor.
However, I may not build a better translator in three months, but what I can do is to settle down a consistent Apertium baseline of (Simplified) Chinese to Spanish rule-based translation system.
(c) How closely related is the pair?
In terms of morphology and syntax, the languages involved are quite far away.
(d) How many resources already exist for the pair?
We have at our disposal many Chinese and Spanish data corpus, see [2], additional corpus may be added if we get the authorization, as the ones provided by TAUS (more than 3 million sentences). Furthermore, we also have Chinese and Spanish segmenters and postaggers (Stanford for Chinese [1]). We have other resources in http://www.mandarintools.com/ The project will be based in results from [3]
(e) Are there any mentors who can evaluate your work?
Francis Tyers and Felipe Sánchez-Martínez.
In addition, Marta Ruiz Costa-jussà will support me with the resources and the statistical part of the work.

'''REFERENCES'''

[1] Pi-Chuan Chang, Huihsin Tseng, Dan Jurafsky, and Christopher D. Manning. 2009. Discriminative Reordering with Chinese Grammatical Relations Features. In Proceedings of the Third Workshop on Syntax and Structure in Statistical Translation.
[2] Marta R. Costa-jussà, Carlos Henríquez, and Rafael E. Banchs, Evaluating Indirect Strategies for Chinese-Spanish statistical machine translation JAIR Journal of Artificial Intelligence Research, 2012, vol 45, pages 761-780
[3] Felipe Sánchez-Martínez. (2008). Using unsupervised corpus-based methods to build rule-based machine translation systems. PhD thesis, Universitat d’Alacant.

'''email: Jordi.Centelles.Sabater@gmail.com'''

@@ Line 79: / Line 79: @@
 '''nosotros compramos una manzana'''
-kid's story:
+'''kid's story:'''
 Jaime ~y María en jardín interior . hoy clima agradable , muy cálido . pero ayer frío oh ！ellos no pueden en exterior #jugar . Jaime ~y María muy #gustar #jugar , ellos siempre en grande casa en frente ~de ~de jardín #juntos #jugar

@@ Line 79: / Line 79: @@
 '''nosotros compramos una manzana'''
+kid's story:
+Jaime ~y María en jardín interior . hoy clima agradable , muy cálido . pero ayer frío oh ！ellos no pueden en exterior #jugar . Jaime ~y María muy #gustar #jugar , ellos siempre en grande casa en frente ~de ~de jardín #juntos #jugar
+Jaime es un niño pequeño . él ahora #seis años . niña pequeña #ser su  hermana . ella ahora #cinco años . Jaime tiene un perro , el perro ahora en jardín interior . perro muy #gustar  ~con Jaime ~y María juega . perro ahora muy feliz .
+María tiene perro ? no , ella no tiene perro , ella #tener gato . pero , gato en casa interior #dormir
+su madre ~y gato en casa interior. ella desde ventana #ver ~a Jaime ~y María #jugar .  Jaime rápidamente corre ~a gran viejo árbol detrás ，porque él no quiere María vea ~a él
-The bilingual dictionary and the transfer rules are in: https://github.com/jCentelles/Apertium-zh-es
-Also here there are the transfer rules and the bilingual dictionary to translate the first paragraph of the kid's history (in the Readme file there is the command): https://sourceforge.net/projects/gsocapertium/files/?
 Note: the Chinese Segmenter and Postagger will be integrated in Apertium during the development of the project.
 == Resources ==

@@ Line 82: / Line 82: @@
 The bilingual dictionary and the transfer rules are in: https://github.com/jCentelles/Apertium-zh-es
+Also here there are the transfer rules and the bilingual dictionary to translate the first paragraph of the kid's history (in the Readme file there is the command): https://sourceforge.net/projects/gsocapertium/files/?
 Note: the Chinese Segmenter and Postagger will be integrated in Apertium during the development of the project.

@@ Line 49: / Line 49: @@
 [[File:grantt.jpg]]
+== Coding Challenge ==
+For the coding challenge I have built a small Rule-based Machine Translation system (based on Apertium) from Chinese to Spanish.  The analysis part consists of a  Chinese Segmenter and PoStagger, both open source. I have written some transfer rules and a small bilingual dictionary. Finally, for the generation step, I have used the Spanish dictionary from the Apertium English-Spanish translation .
+As follows, the command lines to translate some Chinese sentences:
+$ "^我<PN>$ ^买<VV>$ ^一个<CD>$  ^车<NN>$ " | apertium-transfer apertium-zh-es.zh-es.t1x zh-es.t1x.bin zh-es.autobil.bin
+'''^prpers<prn><tn><p1><mf><sg>$ ^comprar<vblex><pri><p1><sg>$ ^un<num><m><sg>$ ^coche<n><m><sg>$'''
+$ echo "^我<PN>$ ^买<VV>$ ^一辆<CD>$  ^车<NN>$ " | apertium-transfer apertium-zh-es.zh-es.t1x zh-es.t1x.bin zh-es.autobil.bin | lt-proc -g zh-es.autogen.bin
+'''yo compro un coche'''
+$ echo "^我<PN>$ ^买<VV>$ ^一个<CD>$  ^苹果<NN>$ " | apertium-transfer apertium-zh-es.zh-es.t1x zh-es.t1x.bin zh-es.autobil.bin | lt-proc -g zh-es.autogen.bin
+'''yo compro una manzana'''
+$ echo "^我们<PN>$ ^买<VV>$ ^一个<CD>$  ^苹果<NN>$ " | apertium-transfer apertium-zh-es.zh-es.t1x zh-es.t1x.bin zh-es.autobil.bin | lt-proc -g zh-es.autogen.bin
+'''nosotros compramos una manzana'''
+The bilingual dictionary and the transfer rules are in: https://github.com/jCentelles/Apertium-zh-es
+Note: the Chinese Segmenter and Postagger will be integrated in Apertium during the development of the project.
 == Resources ==

@@ Line 3: / Line 3: @@
 == General Questions ==
-=== Why is it you are interested in machine translation? ==='
+=== Why is it you are interested in machine translation? ===
 The quantity of information in our society is increasing incredibly quickly, and most of this information is multilingual, which means that translation plays a very important role to deal with this information. Companies, governments and most people have to face the challenge of communicating in other language that their own. So the question is, who is not interested in having access to a great machine translation system? So, I think developing machine translation systems is a challenge which should have a big impact on society. I am interested in participating on it.
@@ Line 42: / Line 42: @@
 Also, as the translator will be available in the web, tourists traveling across China and Latin America can use it. Spanish students from Chinese (or the other way round) can use the translator to test their knowledge.
-== Work Plan =='
+== Work Plan ==
@@ Line 94: / Line 94: @@
 '''email: Jordi.Centelles.Sabater@gmail.com'''
-''' nick IRC: jCentelles """
+''' nick IRC: jCentelles'''
+=== Personal skills ===
+I am a Telecommunication engineer by the UPC (Universitat Politècnica de Catalunya) and I am
+currently pursuing economics in the UB (Universitat de Barcelona).
+'''Programming skills:''' Knowledge in C++,Python, Perl, Java, PHP and Xml. Also Knowledge in
+Statistical Machine Translation (Moses), and in building apps and webs (Html and Javascript).
+'''Language skills:''' Native in Spanish. Very basic knowledge of Chinese, but I will develop most of the
+project in Singapore, where I have accessible Chinese colleagues.
+=== Open Source projects ===
+I have experience in working with Moses. I have been in charge of programming the Spanish-Chinese
+web-translator and its applications for android and Apple's iOS for the Chinese company Baidu and the
+Institute for Infocomm Research in Singapore.
+=== list any non-Summer-of-Code plans you have for the Summer ===
+I am working on my final project from my Telecom's bachelor at Institute for Infocomm Research in
+Singapore. My project goal is to work on Chinese-Spanish machine translation. I will be full-time
+dedicated to the Google Summer of Code, and it will be a big part of my final project.
 [[Category:GSoC_2013_Student_proposals|JCentelles]]

@@ Line 1: / Line 1: @@
 == Title ==
-'''TITLE: Chinese-to-Spanish Apertium System'''
+Chinese-to-Spanish Apertium System'''
-== Apertium questions ==
+== General Questions ==
-•'''Why is it you are interested in machine translation?'''
+=== Why is it you are interested in machine translation? ==='
 The quantity of information in our society is increasing incredibly quickly, and most of this information is multilingual, which means that translation plays a very important role to deal with this information. Companies, governments and most people have to face the challenge of communicating in other language that their own. So the question is, who is not interested in having access to a great machine translation system? So, I think developing machine translation systems is a challenge which should have a big impact on society. I am interested in participating on it.
-•'''Why is it that you are interested in the Apertium project?'''
+=== 'Why is it that you are interested in the Apertium project? ===
 I am interested in Apertium mainly because it is a popular, well-known and active open source project and it uses a rule-based translation engine which I want to further familiarize with.
-•'''Which of the published tasks are you interested in? What do you plan to do?'''
+=== Which of the published tasks are you interested in? What do you plan to do? ===
 Introducing Chinese to Spanish translation. About my work plan, you can see further information in the description of the project. Here I just summarize the main ideas. I will start to familiarize and incorporate into the Apertium architecture the given list of GPL resources (I provide) to analyze Chinese and generate Spanish (well, this language is already in Apertium). Then, I will work with an statistical system + postedition to complete the transfer approach.
@@ Line 64: / Line 64: @@
-•'''Can I do a pair with language x and language y ?'''
+=== Can I do a pair with language x and language y ? ===
 — Yes, there are no restrictions. But you should take the following into consideration:
 (a) Are there existing machine translation (MT) systems for this pair?
 Yes there are. The most popular are the Google translator (statistical), Yahoo Babelfish (rule-based) and Microsoft Bing (hybrid).
 (b) If there are existing systems, how good are they? -- Could you do better in three months?
 Even the existing machine translation systems are well-known, their Chinese to Spanish translations are quite poor.
 However, I may not build a better translator in three months, but what I can do is to settle down a consistent Apertium baseline of (Simplified) Chinese to Spanish rule-based translation system.
 (c) How closely related is the pair?
 In terms of morphology and syntax, the languages involved are quite far away.
 (d) How many resources already exist for the pair?
 We have at our disposal many Chinese and Spanish data corpus, see [2], additional corpus may be added if we get the authorization, as the ones provided by TAUS (more than 3 million sentences). Furthermore, we also have Chinese and Spanish segmenters and postaggers (Stanford for Chinese [1]). We have other resources in http://www.mandarintools.com/  The project will be based in results from [3]
 (e) Are there any mentors who can evaluate your work?
 Francis Tyers and Felipe Sánchez-Martínez.

@@ Line 1: / Line 1: @@
+== Title ==
 '''TITLE: Chinese-to-Spanish Apertium System'''
+== Apertium questions ==
 •'''Why is it you are interested in machine translation?'''
 The quantity of information in our society is increasing incredibly quickly, and most of this information is multilingual, which means that translation plays a very important role to deal with this information. Companies, governments and most people have to face the challenge of communicating in other language that their own. So the question is, who is not interested in having access to a great machine translation system? So, I think developing machine translation systems is a challenge which should have a big impact on society. I am interested in participating on it.
@@ Line 12: / Line 12: @@
 Introducing Chinese to Spanish translation. About my work plan, you can see further information in the description of the project. Here I just summarize the main ideas. I will start to familiarize and incorporate into the Apertium architecture the given list of GPL resources (I provide) to analyze Chinese and generate Spanish (well, this language is already in Apertium). Then, I will work with an statistical system + postedition to complete the transfer approach.
+== Project description ==
+=== Motivation ===
-'''Project description'''
-'''MOTIVATION'''
 Chinese and Spanish are two of the most spoken languages in the world and there are many economical reasons to pursue this language pair translations (i.e. many China companies interested in expanding to Latin America).
@@ Line 31: / Line 30: @@
 )The rule-based can exploite the use of linguistic tools which are available separately for Chinese and Spanish.
+=== Description ===
-'''DESCRIPTION'''
 We will introduce the Chinese-to-Spanish translation in the Apertium open source project. We will do this integration using the GPL available tools for the pair of languages. This tools will be used for the analysis and generation steps. Some of the resources that we have include parallel corpus (from which we can extract bilingual dictionaries in case we do not find dictionaries), statistical postaggers and segmenter (from Stanford, for Chinese, see references later in the Apertium questions).
@@ Line 43: / Line 42: @@
 Also, as the translator will be available in the web, tourists traveling across China and Latin America can use it. Spanish students from Chinese (or the other way round) can use the translator to test their knowledge.
+== Work Plan =='
-'''WORK PLAN'''
@@ Line 50: / Line 49: @@
 [[File:grantt.jpg]]
+== Resources ==
 As follows we name the resources that we will be using:
@@ Line 61: / Line 61: @@
+== Apertium Questions ==
-APERTIUM QUESTIONS
 •'''Can I do a pair with language x and language y ?'''
@@ Line 79: / Line 79: @@
 In addition, Marta Ruiz Costa-jussà will support me with the resources and the statistical part of the work.
+== References ==
-'''REFERENCES'''
 	[1] Pi-Chuan Chang, Huihsin Tseng, Dan Jurafsky, and Christopher D. Manning. 2009. 	Discriminative Reordering with Chinese Grammatical Relations Features. In 	Proceedings of the Third Workshop on Syntax and Structure in Statistical Translation.
@@ Line 87: / Line 87: @@
+== About me ==
 '''email: Jordi.Centelles.Sabater@gmail.com'''
+''' nick IRC: jCentelles """
 [[Category:GSoC_2013_Student_proposals|JCentelles]]

← Older revision		Revision as of 11:35, 27 April 2013
Line 90:		Line 90:

	'''email: Jordi.Centelles.Sabater@gmail.com'''		'''email: Jordi.Centelles.Sabater@gmail.com'''

			[[Category:GSoC_2013_Student_proposals\|JCentelles]]

← Older revision		Revision as of 06:30, 24 April 2013
Line 44:		Line 44:

	'''WORK PLAN'''		'''WORK PLAN'''


	[[File:grantt2.jpg]]		[[File:grantt2.jpg]]