User:Rroychoudhury/GSoC 2020 Proposal

From Apertium
Jump to navigation Jump to search

Personal Details

Name : Rajarshi Roychoudhury
Email address : rroychoudhury2@gmail.com
IRC: Rajarshi
Github: https://github.com/RajarshiRoychoudhury
Timezone : GMT+5.30hrs
Current Designation : Undergraduate Researcher at Jadavpur University specialising in Natural Language Processing


About me

Open source softwares I use : Apertium , Tensorflow , Ubuntu.
Professional Interests : Natural language Processing ,Computational Linguistics , Sentiment Analysis , Statistical and Rule-based Machine Translation.

Why is it that I am interested in Apertium and Machine Translation

Being a Natural Language Processing researcher , Machine Translation intrigues me with its diverse nature.Finding a bridge between two languages is a hard task , and Machine Translation builds that bridge efficiently. Machine translation is beneficial for society for overcoming the language barrier . It also helps to gather resources on languages that are slowly becoming extinct.

Most of the current translation systems use statistical machine translation,Google being one of them. Apertium on the other hand uses rule-based machine translation , which is particularly useful for closely related language pairs. Moreover I am interested in Apertium because it deals with low-resource languages , which is a challenge for neural network based systems due to lack of data. Working with Apertium would let me explore the morphology and lexicons of the two languages I am most familiar with , and it would be interesting to develop transfer rules for building a bridge between the two so that people can utilise this for free.

Which of the published tasks are you interested in?

The published task I am interested in is a modified version of Adopt an unreleased language pair (http://wiki.apertium.org/wiki/Ideas_for_Google_Summer_of_Code). I plan to work on the Bengali-Hindi pair and. Bengali is my mother-tongue and I am well conversant in Hindi and its grammar. However in addition to this I wish to incorporate some changes in the disambiguation and lexical selection step. Below are the details of what I propose to introduce.

My Proposal

Title: IMPROVING MACHINE TRANSLATION WITH SENTIMENT PRESERVATION IN HINDI-BANGLA PAIR

Some problems with Translation pair for Bangla-Hindi:

Existing models for Bangla-Hindi are far from accurate. For example, when the following sentence written in Bengali is given in Google Translate, it gives a completely incorrect translation.

                                       Bengali:”মেয়েটি দুধ জ্বাল দিতে দিতে বাবার সাথে কথা বলছে”
Hindi: “लड़की दूध देने के लिए अपने पिता से बात कर रही है”

Where as it should give

                                       “दूध उबालते समय लड़की अपने पिता से बात कर रही है”.

In this case it fails to recognise “জ্বাল দিতে দিতে” as a present participle. This is because there is an intermediate translation in English, which loses a part of the source information.In such scenarios where we deal with closely related language pairs like Bangla-Hindi( which share the Sanskrit roots), RBMT is more effective.Talking specifically about Apertium, this can easily be solved in the 3 stage shunking step in the Apertium workflow.

Some problems with Apertium:

The current framework for Apertium has some disadvantages. One of them being the source information is lost during the target generation step. This may result in incorrect translation. Considering the released pairs of Apertium (in this case English-Esperanto) ,the sentence

                                “Its funny how thieves try to break into a house  and get arrested.”

Gets translated to “Ties amuza kiel ŝtelistoj provas rompi en domo kaj akiri arestita.” , which takes the literal translation of the sentence and means

                                 “the thieves try to {break into the house and get arrested}”
                           instead of   “the thieves try to {break into the house }and{ get arrested}”

To incorporate this information of the source language , we need a sense of what the sentence tries to convey, which is impossible considering that source information is lost during the target step generation, for only the grammatical inflections of the lexicons are stored. Some additional information on the words needs to be preserved.

Solution

The technique I will use is sentiment analysis on word level. The sentiment polarity of the words( out of 3 polarity-positive, negative , neutral) will be stored as an information(tag) on the lexicons in the monolingual dictionary.
This sentiment tag will be used in

  • POS disambiguation- Lexical selection will have additional selection criteria of sentiment, which will help to disambiguate lexicons by generating rules based on patterns incorporating sentiment of words. (.lrx file)
  • Chunking: The sentiment of the words will be mainly used in the.t1x file , which identifies words, and groups of words, which may need to have their order altered, or tags adding. Order altering with sentiment analysis can solve the problem of the translation specified in the above section. Patterns can be generated by combining grammatical annotations and sentiment analysis of the words, and suitable reordering/ tag removal can be done.

How to do sentiment analysis on word level? https://papers.nips.cc/paper/5782-character-level-convolutional-networks-for-text-classification.pdf is an example of a paper whuch used neural networks for text classification on word levels. Recent studies show that this method is very effective in predicting the sentiment polarity of a word. SInce apertium deals with low resource languages , a huge exhaustive corpus is not available. However for any language, the number of unique characters is well defined (it is kept inside the <alphabet> tag of monolingual dictionary). We can determine character embedding weights for each of these characters , and treat words as a sequence of characters . These sequences of vectors can be fed inside a Recurrent-Neural-Network , and we can classify the words based on sentiments. The result can be stored in a file , so the end result will be independent of neural networks. Entries in the monolingual dictionaries will be modified by incorporating the sentiment determined as a tag . This method achieves a good accuracy even for a small corpus of 8000 words. Particularly for this project, I will use SentiWordnet Hindi (https://amitavadas.com/sentiwordnet.php), where sentiment annotated data is already present. Also it has ~8000 words for Bengali, Hindi each and hence can be a good resource. The code for sentiment prediction using character embedding with neural networks is given below.

Workplan

The ultimate goal is to have multi-purposable transducers for a variety of Indic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y.
Current Scenario:

  • Hindi
    • Number of stems: 37,833
    • Paradigms:101
    • Coverage: ~83.1%
  • Bengali
    • Number of stems: 8230
    • Paradigms: 137
    • Coverage: ~74%
  • Resources:
    • For Bengali and Hindi we have Sentiwordnet which will give sentiment annotated data of ~9000 words on each, which will act as training data for sentiment classification. These words will also be incorporated in the dictionaries. Besides, wikipedia dumps, online text resources are also available. In apertium, we have
      • Monolingual Bengali dictionary from apertium-bn-en
      • Monolingual Hindi dictionary from apertium-hin
  • Calculation of word error rate(WER):
    • This will be done by calculated using random Wikipedia texts and online text available from various sources. My plan is to collect a good amount of resources on both languages and build dictionaries on the basis of that, and coverage/WER will be calculated against random Wikipedia tests.


Detailed Work Plan

Week Dates Goals Bidix WER Coverage
Post-application period 10 March - 26 May
  • Revising the grammar of Bengali and Hindi
  • Studying in detail the documentation and apertium workflow
  • Study the existing paradigms and lexicons in monolingual dictionaries, make paradigms to combine noun and postposition for accurate translation from Bengali to Hindi( for example, proper noun paradigm have nominative , objective and generative in Bengali, but no such inflection in the paradigm in existing Hindi dictionary )
  • Collect resources on both languages
Current
situation


~9,000 (cat-ita)
~7,500 (cat-por)
Current
situation


~30% (cat > ita)
~30% (cat > por)
~30% (por > cat)
Current
situation


~88% (cat > ita)
~82% (ita > cat)
~88% (cat > por)
~84% (por > cat)
1 27 May - 2 June
  • Determine sentiment of existing lexicons
  • Making paradigms to incorporate words like “রয়েছিল”.”খেয়েছিল”
  • Incorporating sentiment tags in the existing and new lexicons
  • Expanding the bidix(~600)
~11,000 (cat-ita) ~85.5% (ita > cat)
2 3 June- 9 June
  • Start making lexical selection rules(bn)
  • Start making transfer rules(bn-hi)
  • Expanding bidix(~1400)
~13,000 (cat-ita) ~87.5% (ita > cat)
3 10 June - 16 June
  • Expanding lexical selection rules(bn)
  • Expanding transfer rules(bn-hi),work on .t1x file
  • Expanding bidix(~3000)
  • Documentation
~14,000 (cat-ita) <20% (ita > cat) ~89% (ita > cat)
4 17 June - 23 June
  • Expanding lexical selection rules(bn)
  • Expanding transfer rules(bn-hi) work on .t2x file
  • Expanding bidix(~4500)
  • Testing on resources(bn-hi translation)
~15,000 (cat-ita) ~90% (cat > ita)
~90% (ita > cat)
5 24 June - 30 June
  • Expand bilingual dictionary cat-ita
  • Transfer and lexical selection rules (cat > ita)
  • Testvoc cat-ita, ita-cat: vblex

First evaluation (28 June)

~16,000 (cat-ita) ~90.5% (cat > ita)
~90.5% (ita > cat)
6 1 July - 7 July
  • Expand bilingual dictionary cat-ita
  • Transfer and lexical selection rules (cat > ita)
  • Testvoc cat-ita, ita-cat: adj, adv, np
~17,000 (cat-ita) ~91% (cat > ita)
~91% (ita > cat)
7 8 June - 14 July
  • Transfer and lexical selection rules (cat > ita)
  • Testvoc cat-ita, ita-cat: n
  • Write documentation
~18,000 (cat-ita) <15% (cat > ita)
<15% (ita > cat)
~91.5% (cat > ita)
~91.5% (ita > cat)
8 15 July - 21 July
  • por > cat
  • Expand bilingual dictionary cat-por
  • Disambiguation rules (por > cat)
  • Work on Portuguese proper names
~9,500 (cat-por) ~87% (por > cat)
9 22 July - 28 July
  • Expand bilingual dictionary
  • Disambiguation rules (por > cat)
  • Work on Portuguese proper names
  • Transfer and lexical selection rules (por > cat)
  • Testvoc cat-por, por-cat: np

Second evaluation (26 July)

~11,500 (cat-por) ~89% (por > cat)
10 29 July - 4 August
  • Expand bilingual dictionary
  • Disambiguation rules (por > cat)
  • Transfer and lexical selection rules (por > cat)
~13,000 (cat-por) <20% (por > cat) ~89.5% (por > cat)
11 5 August - 11 August
  • cat > por
  • Expand bilingual dictionary
  • Transfer and lexical selection rules (cat > por)
  • Testvoc cat-por, por-cat: closed categories, vblex
~14,500 (cat-por) ~90% (cat > por)
~90% (por > cat)
12 12 August - 18 August
  • Expand bilingual dictionary
  • Transfer and lexical selection rules (cat > por)
  • Testvoc cat-por, por-cat: adj, adv
~16,000 (cat-por) ~90.5% (cat > por)
~90.5% (por > cat)
13 18 August - 25 August
  • Expand bilingual dictionary
  • Transfer and lexical selection rules (cat > por)
  • Testvoc cat-por, por-cat: n

Final evaluation (26 August)

~17,000 (cat-por) <15% (cat > por)
<15% (por > cat)
~91.0% (cat > por)
~91.0% (por > cat)


== List your skills and give evidence of your qualifications == I was a part of the research team that worked on “Sentiment Analysis on word level based on character embedding” in my university and have done 2-3 projects on Statistical Machine Translation. I have taught Bengali as a teaching assistant in my high school , which is my mother tongue. I have learnt Hindi in high school for 6 years and I am very familiar with the grammatical nuances of both the languages.Besides that I am familiar with Python, XML. Attached is my CV (https://drive.google.com/open?id=0BwLpcVkeJcn4Tmx6UjNKTmRqWF9Td0hGVGRKNjJ1TUtmaUFB)