User:Srj31/GSOC 2020 proposal:Bengali-Hindi pair - Revision history

Srj31: /* Coding Challenge */

2020-03-31T12:53:39Z

Coding Challenge

Srj31: /* Skill */

2020-03-30T21:39:44Z

Skill

Srj31: /* Second Phase */

2020-03-30T20:24:13Z

Second Phase

Srj31: /* Second Phase */

2020-03-30T20:16:17Z

Second Phase

Srj31: /* Second Phase */

2020-03-30T20:15:46Z

Second Phase

Srj31: /* Work Plan */

2020-03-30T20:08:32Z

Work Plan

Srj31: /* How it will benefit */

2020-03-30T19:52:27Z

How it will benefit

Srj31: /* Why is it that you are interested in Machine Translation and Apertium? */

2020-03-30T19:50:51Z

Why is it that you are interested in Machine Translation and Apertium?

Srj31: /* Which of the published tasks are you interested in? What do you plan to do? */

2020-03-30T19:34:26Z

Which of the published tasks are you interested in? What do you plan to do?

Srj31: /* First Phase */

2020-03-30T17:14:16Z

First Phase

@@ Line 167: / Line 167: @@
              জেমস '''লিখতে''' ভাল '''ছিল'''  - जेम्स '''लिखने''' में अच्छा '''था'''  - (James was good at '''writing''')
-- Disambiguation is required for the genders, bengali verbs do not mark gender but hindi does.  (solution: need the transfer rules to identify the gender from the subject of the sentence , else from the context of the text)
+- Disambiguation is required for the genders, bengali verbs do not mark gender but hindi does.  (solution:  need the transfer rules to identify the gender from the subject of the sentence , else
+from the context of the text. Both the languages follow SOV word order, thus different cases
+have to be taken for intransitive and transitive verbs, so I was suggested that anaphora
+resolution could be helpful here in identifying the subject so that the correct marker proceeds
+the verb)
 e.g.

@@ Line 283: / Line 283: @@
 '''Experience :'''
-I have studied many languages and linguistics in general while preparing for the International Linguistics Olympiad 2019 and creating and testing problems for the National Linguistic Olympiad , has allowed me to formalise rules which are followed while translating from one language to another and to notice certain patterns that are followed while translating from source language to target language.
+I have studied many languages and linguistics in general while preparing for the International
+Linguistics Olympiad 2019 and Asian Pacific Linguistic Olympiad, and creating and testing
-I have done online courses on NLP on the online platform coursera and am proficient in Data Structures and Algorithms, through competitive programming.
+problems for the National Linguistic Olympiad , has allowed me to learn about the rules that a
+language follows and to formalise rules which are followed while translating from one language
+to another and to notice certain patterns that are followed while translating from source
+language to target language. I have also been involved in programming since high school.
+Taking part in competitive programming contests, preparing problems for school contests and
+have worked on IoT projects such as home automation. I have done online courses on NLP on
+the online platform coursera and am proficient in Data Structures and Algorithms on account of
+competitive programming.
 =='''Non Summer of Code plans'''==

@@ Line 92: / Line 92: @@
 '''Week 8:'''
+Transfer rules for hin->ben
 Test translations, make some improvements, fix some bugs and prepare for the evaluation

← Older revision		Revision as of 20:16, 30 March 2020
Line 79:		Line 79:
	Expand the bilingual dictionary		Expand the bilingual dictionary

	Write transfer rules for ~~hin-~~ben transfer		Write transfer rules for ben->hin transfer

@@ Line 86: / Line 86: @@
 Expand the bilingual dictionary, lexical selection rules
-Transfer rules for hin-ben
+Transfer rules for ben->hin
 Manual Disambiguation(ben)

@@ Line 34: / Line 34: @@
 '''Post-application period:'''
-Find language resources for both and ben-hin
+Find language resources for  ben-hin(Mainly wikipedia with articles available in both languages)
-Learn more regarding the Apertium dictionaries and tools
+Learn more regarding the Apertium dictionaries and tools(transfer rules, CG, anaphora resolution) and look at other language pairs and their rules, to further understand how to create efficient rules
 '''Community Bonding:'''
 Getting familiar with all the Apertium modules and it’s working. Discussion with mentors and clearing doubts.

← Older revision		Revision as of 19:52, 30 March 2020
Line 26:		Line 26:


	Bengali is the official and most widely spoken language of Bangladesh and second most widely spoken of the 22 scheduled languages of India, behind Hindi. Translation allows ideas and information to spread across cultures. In the process, translation changes history. With Hindi and bengali being the most widespread language in India, creating one for languages in India will provide the spread of culture and literature of India. The current Bengali-Hindi pair has only a bilingual dictionary having some of the noun words and not much. By doing this project I aim to create a working language pair which performs much better and can create a correct translation.		Bengali is the official and most widely spoken language of Bangladesh and second most widely spoken of the 22 scheduled languages of India, behind Hindi. Translation allows ideas and information to spread across cultures. In the process, translation changes history. With Hindi and bengali being the most widespread language in India, creating one for languages in India will provide the spread of culture and literature of India. The current Bengali-Hindi pair has only a bilingual dictionary having some of the noun words and not much. By doing this project I aim to create a working language pair which performs much better and can create a correct translation, thus creating translations for two of the most spoken languages in India.



	== '''Work Plan''' ==		== '''Work Plan''' ==

@@ Line 17: / Line 17: @@
 == '''Why is it that you are interested in Machine Translation and Apertium?''' ==
-I had been wanted to work on a project which involved Machine Translation and was intrigued by how computers could understand what we said.This curiosity got me into studying linguistics in high school. I came fifth at the National Linguistics Olympiad and had the honor of representing Team India at the International Linguistics Olympiad 2019 at South Korea. It was this experience and the 3 years of indulging in Linguistics along with my interest in NLP, that got me interested in a Machine Translation project. Apertium being an open source platform for developing rule-based machine translation systems, has intrigued me to contribute to this platform and I will have the opportunity to further create more language pairs for the various languages of India.
+I have always been intrigued by how computers could understand what we said and had been wanting to work on a project which involved Machine Translation This curiosity got me into studying linguistics in high school. I came fifth at the National Linguistics Olympiad and had the honor of representing Team India at the International Linguistics Olympiad 2019 at South Korea. It was this experience and the 3 years of indulging in Linguistics along with my interest in NLP, that got me interested in a Machine Translation project. Apertium being an open source platform for developing rule-based machine translation systems, has intrigued me to contribute to this platform and I will have the opportunity to further create more language pairs for the various languages of India.
 == '''Which of the published tasks are you interested in? What do you plan to do?''' ==

@@ Line 22: / Line 22: @@
 == '''Which of the published tasks are you interested in? What do you plan to do?''' ==
-I plan on working on Adopting the unreleased language pair Hindi-Bengali and get the pair released in both directions having a WER < 20. I plan on improving the pair to be able to translate sentences coherently. As Bengali and Hindi are similar in various linguistic aspects, they will have similar rules thus machine translation would give correct sentences if done correctly.  Hindi and Bengali are very similar languages and are an "Apertium-ish" pair.
+I plan on working on Adopting the unreleased language pair Hindi-Bengali and get the pair released in both directions having a WER < 20. I plan on improving the pair to be able to translate sentences coherently. As Bengali and Hindi are similar in various linguistic aspects, they will have similar rules thus machine translation would give correct sentences if done correctly.  Hindi and Bengali are very similar languages and are an "Apertium-ish" pair, thus this pair would work with the Apertium architecture.
 == '''How it will benefit''' ==

@@ Line 62: / Line 62: @@
 '''Week 4:'''
+Learn more about the Constraint Grammar and implement it
 Update the documentation and prepare for the evaluation