User:Rroychoudhury/GSoC 2020 Proposal
Contents
Personal Details
Name : Rajarshi Roychoudhury
Email address : rroychoudhury2@gmail.com
IRC: Rajarshi
Github: https://github.com/RajarshiRoychoudhury
Timezone : GMT+5.30hrs
Current Designation : Undergraduate Researcher at Jadavpur University specialising in Natural Language Processing
About me
Open source softwares I use : Apertium , Tensorflow , Ubuntu.
Professional Interests : Natural language Processing ,Computational Linguistics , Sentiment Analysis , Statistical and Rule-based Machine Translation.
Why is it that I am interested in Apertium and Machine Translation
Being a Natural Language Processing researcher , Machine Translation intrigues me with its diverse nature.Finding a bridge between two languages is a hard task , and Machine Translation builds that bridge efficiently. Machine translation is beneficial for society for overcoming the language barrier . It also helps to gather resources on languages that are slowly becoming extinct.
Most of the current translation systems use statistical machine translation,Google being one of them. Apertium on the other hand uses rule-based machine translation , which is particularly useful for closely related language pairs. Moreover I am interested in Apertium because it deals with low-resource languages , which is a challenge for neural network based systems due to lack of data. Working with Apertium would let me explore the morphology and lexicons of the two languages I am most familiar with , and it would be interesting to develop transfer rules for building a bridge between the two so that people can utilise this for free.
Which of the published tasks are you interested in?
The published task I am interested in is a modified version of Adopt an unreleased language pair (http://wiki.apertium.org/wiki/Ideas_for_Google_Summer_of_Code). I plan to work on the Bengali-Hindi pair and. Bengali is my mother-tongue and I am well conversant in Hindi and its grammar. However in addition to this I wish to incorporate some changes in the disambiguation and lexical selection step. Below are the details of what I propose to introduce.
My Proposal
Title: IMPROVING MACHINE TRANSLATION WITH SENTIMENT PRESERVATION IN HINDI-BANGLA PAIR
Some problems with Translation pair for Bangla-Hindi:
Existing models for Bangla-Hindi are far from accurate. For example, when the following sentence written in Bengali is given in Google Translate, it gives a completely incorrect translation.
Bengali:”মেয়েটি দুধ জ্বাল দিতে দিতে বাবার সাথে কথা বলছে”
Hindi: “लड़की दूध देने के लिए अपने पिता से बात कर रही है”
Where as it should give
“दूध उबालते समय लड़की अपने पिता से बात कर रही है”.
In this case it fails to recognise “জ্বাল দিতে দিতে” as a present participle. This is because there is an intermediate translation in English, which loses a part of the source information.In such scenarios where we deal with closely related language pairs like Bangla-Hindi( which share the Sanskrit roots), RBMT is more effective.Talking specifically about Apertium, this can easily be solved in the 3 stage shunking step in the Apertium workflow.
Some problems with Apertium:
The current framework for Apertium has some disadvantages. One of them being the source information is lost during the target generation step. This may result in incorrect translation. Considering the released pairs of Apertium (in this case English-Esperanto) ,the sentence
“Its funny how thieves try to break into a house and get arrested.”
Gets translated to “Ties amuza kiel ŝtelistoj provas rompi en domo kaj akiri arestita.” , which takes the literal translation of the sentence and means
“the thieves try to {break into the house and get arrested}”
instead of “the thieves try to {break into the house }and{ get arrested}”
To incorporate this information of the source language , we need a sense of what the sentence tries to convey, which is impossible considering that source information is lost during the target step generation, for only the grammatical inflections of the lexicons are stored. Some additional information on the words needs to be preserved.
Solution
The technique I will use is sentiment analysis on word level. The sentiment polarity of the words( out of 3 polarity-positive, negative , neutral) will be stored as an information(tag) on the lexicons in the monolingual dictionary.
This sentiment tag will be used in
- POS disambiguation- Lexical selection will have additional selection criteria of sentiment, which will help to disambiguate lexicons by generating rules based on patterns incorporating sentiment of words. (.lrx file)
- Chunking: The sentiment of the words will be mainly used in the.t1x file , which identifies words, and groups of words, which may need to have their order altered, or tags adding. Order altering with sentiment analysis can solve the problem of the translation specified in the above section. Patterns can be generated by combining grammatical annotations and sentiment analysis of the words, and suitable reordering/ tag removal can be done.
How to do sentiment analysis on word level?
https://papers.nips.cc/paper/5782-character-level-convolutional-networks-for-text-classification.pdf is an example of a paper whuch used neural networks for text classification on word levels. Recent studies show that this method is very effective in predicting the sentiment polarity of a word.
SInce apertium deals with low resource languages , a huge exhaustive corpus is not available. However for any language, the number of unique characters is well defined (it is kept inside the <alphabet> tag of monolingual dictionary). We can determine character embedding weights for each of these characters , and treat words as a sequence of characters . These sequences of vectors can be fed inside a Recurrent-Neural-Network , and we can classify the words based on sentiments. The result can be stored in a file , so the end result will be independent of neural networks. Entries in the monolingual dictionaries will be modified by incorporating the sentiment determined as a tag . This method achieves a good accuracy even for a small corpus of 8000 words. Particularly for this project, I will use SentiWordnet Hindi (https://amitavadas.com/sentiwordnet.php), where sentiment annotated data is already present. Also it has ~8000 words for Bengali, Hindi each and hence can be a good resource. The code for sentiment prediction using character embedding with neural networks is given below.
Workplan
The ultimate goal is to have multi-purposable transducers for a variety of Indic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y.
Current Scenario:
- Hindi
- Number of stems: 37,833
- Paradigms:101
- Coverage: ~83.1%
- Bengali
- Number of stems: 8230
- Paradigms: 137
- Coverage: ~74%
- Resources:
- For Bengali and Hindi we have Sentiwordnet which will give sentiment annotated data of ~9000 words on each, which will act as training data for sentiment classification. These words will also be incorporated in the dictionaries. Besides, wikipedia dumps, online text resources are also available. In apertium, we have
- Monolingual Bengali dictionary from apertium-bn-en
- Monolingual Hindi dictionary from apertium-hin
- For Bengali and Hindi we have Sentiwordnet which will give sentiment annotated data of ~9000 words on each, which will act as training data for sentiment classification. These words will also be incorporated in the dictionaries. Besides, wikipedia dumps, online text resources are also available. In apertium, we have
- Calculation of word error rate(WER):
- This will be done by calculated using random Wikipedia texts and online text available from various sources. My plan is to collect a good amount of resources on both languages and build dictionaries on the basis of that, and coverage/WER will be calculated against random Wikipedia tests.
Detailed Work Plan
| Week | Dates | Goals | Bidix | WER | Coverage |
|---|---|---|---|---|---|
| Post-application period | May 4-May 31 |
|
Current situation |
Current situation |
Current situation |
| 1 | June 1-June 7 |
|
~600 | ||
| 2 | 8 June- 15 June |
|
~1,400 | ||
| 3 | 16 June - 23 June |
|
~3,000 | ||
| 4 | 24 June - 30 June |
|
~4,500 | ||
| 5 | 1 July - 8 July |
First evaluation (3 July) |
~6,000 | <33% | ~78% (bn) ~87% (hi) |
| 6 | 8 July - 15 July |
|
~7,000 | ||
| 7 | 16 June - 23 July |
|
~9,000 (cat-ita) | ||
| 8 | 24 July - 31 July |
|
~11,000 | ||
| 9 | 1 August July - 8 August |
Second evaluation (31 July) |
~13,000 | <27% | ~82%(bn-hi) ~90%(hi-bn) |
| 10 | 9 August - 16 August |
|
~14,500 | ||
| 11 | 17 August - 23 August |
|
~16,000 | ||
| 12 | 24 August - 31 August |
|
~16,000 | <22% | ~87% (bn-hi) ~90% (hi-bn) |
List your skills and give evidence of your qualifications
I was a part of the research team that worked on “Sentiment Analysis on word level based on character embedding” in my university and have done 2-3 projects on Statistical Machine Translation. I have taught Bengali as a teaching assistant in my high school , which is my mother tongue. I have learnt Hindi in high school for 6 years and I am very familiar with the grammatical nuances of both the languages.Besides that I am familiar with Python, XML. Attached is my CV [[1]]
Coding Challenge
- Install Apertium (see Minimal installation from SVN)
- Go through the HOWTO
- Rewrote paradigms(bn monolingual) ,added words to monolingual dictionary, Built a bilingual dictionary, wrote transfer files(.t1x), Constraint grammar (.rlx files)
- Wrote a code for sentiment prediction
