User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi

From Apertium
Jump to navigation Jump to search

Contact Information

Name: Priyank Modi
Email: priyankmodi99@gmail.com
Current Designation: Undergraduate Researcher in the LTRC Lab, IIIT Hyderabad (completing 6th semester/3rd year in April '20) and a Teaching Assistant for Linguistics courses
IRC: pmodi
Timezone: GMT +0530 hrs
Linkedin: https://www.linkedin.com/in/priyank-modi-81584b175/
Github: https://github.com/priyankmodiPM

Why I am interested in Apertium

Apertium is an Open Source Rule-based machine translation system. Being an undergrad researcher at the LTRC lab in IIIT-H currently working on understanding the nuances of Indian languages and developing systems which improve our analysis of the same, Machine Translation interests me because it’s a complex problem which tries to achieve something most people believe is only achievable by humans.

Translating data to other languages, and especially low - resource languages gives the speakers of those languages access to valuable data and can help in several domains, such as education, news, judiciary, etc. The dictionaries made in the process are crucial for low resource languages and can even help making spell checkers etc.

The most striking factor for me is the fact that while recent trends to find a solution to MT lean towards Neural Networks and Deep Learning, which fall short when it comes to resource-poor languages, Apertium looks to tackle this using a rule based approach. Not only is this beneficial because of the level of understanding it provides instead of simply blaming data for poor results, it actually shows that it can perform better for low resource languages(even for the pair I present in my proposal).

A tool which is rule-based and open source really helps the community with language pairs that are resource - poor and gives them free translations for their needs and that is why I want to work on improving on it. I want to work with Apertium and GSoC so I can contribute to an important Open Source Tool while also honing my own skills, and I hope to become a part of this amazing community of developers!

Which of the published tasks are you interested in? What do you plan to do?

Adopt an unreleased language pair. I plan on developing the Hindi-Punjabi language pair in both directions i.e. hin-pan and pan-hin. This'll involve improving the monolingual dictionaries for both languages, the hin-pan bilingual dictionary and writing suitable transfer rules to bring this pair to a releasable state.

My Proposal

Why Google and Apertium should sponsor it

  • Both Hindi and Punjabi are widely spoken languages, both by number of speakers and geographic spread. Despite that, Punjabi especially has very limited online resources.
  • Services like Google Translate give unsatisfactory results when it comes to translation of this pair(see Section 2.1) On the contrary, I was able to achieve close to human translation for some sentences using minimal rules and time(see Section 3 : Coding Challenge).
  • I believe the Apertium architecture is suited perfectly for this pair and can replace the current state-of-art translator for this pair.
  • This is an important project(since it adds diversity to Apertium and translation systems in general) which requires at least 2-3 months of dedicated work and can be an important resource.

How and who it will benefit in society

As mentioned above, the Apertium community is strongly committed to under-resourced and minoritised/marginalised languages and Google helps its own way via programs like GSoC and GCI. There exists a good amount of vernacular literature and scriptures which could be circulated to a larger group of people if this project is successful. It'll be an important open source dictionary resource for both languages. My larger aim from this project is to develop a chain of pairs covering Indian languages. Since Urdu and Punjabi share their roots, at least one more pair can be developed with minimum effort. I plan of working towards the Hindi-English pair next year since by then I'll have a good understanding of the architecture a cross-language-family pair.

Google Translate : Analysis and comparison

Google Translate provides an interface to translate the pair in question. I have analysed the results of the translation into Punjabi from Google. The numerical results(computed on a small set of sentences from the coding challenge. The human translation which has been reviewed by 3 annotators is also available in the repo) are given below(source-target):

  • hin-pan: 79.23% WER
  • hin-pan: 56.56% PER
  • pan-hin: 97.89% WER
  • pan-hin: 72.89% WER

The results are simply poor, especially when it comes to longer sentences with less frequently used words. It is rather easy to see that Google Translate doesn't try to capture the case or tense in sentences, rather picks the most commonly used form of that particular root. NER is very limited, in contrast to the Apertium module which captures it well(because of it's 3 stage transfer mechanism I believe). The use of English as intermediate between Romance languages causes problems known to all users, such as the translation of p2.pl verb forms with elided subject to p2.sg, the incorrect choice of past times in the verbs and the disappearance of some pronouns. Here is an example of the last case of the Italian test text (randomly obtained):

Original text (bold mine):

altri invece ne hanno apprezzato la spontaneità, la tenacia e l'affettuosità

Google translation:

altres han apreciat la seva espontaneïtat, tenacitat i afecte

Post-edited translation:

altres n'han apreciat l'espontaneïtat, tenacitat i afecte

It should be added that, although Google translations tend to be more phraseological than the ones obtained by rules, they are also much more difficult to post-edit. The reason is that, while the translation by rules often makes evident and even expected errors, the neuronal translation significantly changes the text, reordering parts of the sentence, removing or putting words, changing singular to plural or plural to singular (!), and modifying expressions. The evaluation of whether the meaning is the same as the original requires a lot more time. This has been quite clear when I have made the post-edition of both the Apertium and Google translations for the Italian and Portuguese texts.

Current state of dictionaries

A released module already exists for Hindi(as part of the urd-hin pair). However there still exist a lot of anomalies in the Hindi mono-dictionary. I've compiled a preliminary version of list of some these here[insert link]. Apart from these, the existing state of the hin-pan bi-dictionary also needs massive improvement. The first step of this project will be to revise these lists of issues and come up with a sustainable solution. It'll be crucial that the changes made, especially to the Hindi mono-dictionary do no affect the urd-hin pair(and the hindi-begali, hindi-marathi and hindi-gujarati pairs which also have little but some work done) in a negative way.

Resources

[to be added - under confirmation for public use]
Hindi-Punjabi Dictionary - wiktionary
Punjabi-Hindi dictionary - Glosbe (awaiting confirmation)
Punjabi Articles - Wikipedia
Punjabi Dictionary - Wiktionary

Workplan

PHASE DURATION GOALS OF THE WEEK BIDIX WER Coverage
COMMUNITY BONDING PERIOD
  • START:April 26th
  • END:May 17th
  • List and discuss implementation choices of hin-pan bidix and urd-hin pair
  • Reading up on the details of Transfer rules(whether or not a 3-stage transfer is the best way for this pair) and assigning weights
  • Finding Language Resources
  • Making Frequency lists
Week ONE : CLOSED CATEGORIES
  • START:May 18th
  • END:May 24th
  • Function words(voc prn, cnj, det, prn, post, gen_endings)
  • Transfer rules for post-positions
Week TWO : Adjectives
  • START:May 25th
  • END:May 31st
  • Punjabi mono-dictionary : adj and adv
  • Expanding bilingual dictionary
  • Lexical selection rules for adj and adv
WEEK THREE: Verbal Paradigms
  • START:June 1st
  • END:June 7th
  • Punjabi mono-dictionary : Verbal paradigms(vblex, vbser, vaux)
  • Expanding bilingual dictionary
  • Lexical selection rules for verbs
  • test adj, adv
~ 1,000
  • Third version of the wrapper with all functions importable from python.
Week FOUR: Dictionary Expansion
  • START:June 8th
  • END:June 14th
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 3,500
Week FIVE
  • START:June 15th
  • END:June 21st
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 5,500
Week SIX
  • START:June 15th
  • END:June 21st
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 7,500
Week SEVEN
  • START:June 22nd
  • END:June 28th
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 9,000
Week EIGHT
  • START:June 29th
  • END:July 5th
  • Expanding bilingual dictionary
  • Lexical selection rules

First Evaluation(June 29th - July 3rd)

~ 7,500
Week NINE
  • START:July 6th
  • END:July 12th
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 7,500
Week TEN
  • START:July 13th
  • END:July 19th
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 7,500
Week ELEVEN
  • START:July 20th
  • END:July 26th
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 7,500
Week TWELVE
  • START:July 27th
  • END:August 2nd
  • Expanding bilingual dictionary
  • Lexical selection rules

Second Evaluation(July 27th - Jult 31st)

~ 7,500
Week THIRTEEN
  • START:August 3rd
  • END:August 9th
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 7,500
Week FOURTEEN
  • START:August 10th
  • END:August 16th
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 7,500
Week FIFTEEN
  • START:August 17th
  • END:August 23rd
  • Expanding bilingual dictionary
  • Lexical selection rules
~ 7,500
Week FIFTEEN
  • START:August 24th
  • END:August 30th
  • Expanding bilingual dictionary
  • Lexical selection rules

Final evaluation(August 24th - August 31st)

~ 7,500

Skills

I'm currently a third year(commencing start of April '20 hopefully :D ) student at IIIT Hyderabad where I'm studying Computational Linguistics. It is a dual degree course where we study Computer Science, Linguistics, NLP and more.

I've been interested in linguistics from the very beginning and due to the rigorous programming courses, I'm also adept at several programming languages like Python, C++, XML, Bash Scripting, etc. I'm skilled in writing Algorithms. Data Structures, and Machine Learning Algorithms as well.

I also have a lot of experience studying and generating data which I feel is essential in solving any problem, especially the one mentioned in this proposal. My paper on 'Hindi TimeBank: An ISO-TimeML Annotated Reference Corpus' recently got accepted in 16th Joint ACL - ISO Workshop on Interoperable Semantic Annotation at LREC 2020. I am working on extending the same for Punjabi using Transfer learning.

I am also closely involved with the committee conducting Asia-Pacific Linguistics Olympiad(which holds a camp, mentors and prepares students for the International Linguistics Olympiad) and help with the organisation and judging for the same.

Due to the focused nature of our courses, I have worked in several projects, such as building Anaphora Resolution systems, Abstractive Summarizers(using Pointer-generators, hierarchical attention and transformers), POS Taggers, Named Entity Recognisers, simple Q-A systems, a fLinux based shell etc. all of which required a working understanding of Natural Language Processing scripting. Some of these projects aren't available on GitHub because of the privacy settings but can be provided if required.

I am fluent in English, Hindi and Punjabi.

Coding challenge

I've completed the coding challenge for translation from Hindi into Punjabi. You can find my work here : Coding challenge repository
Original corpus : source lang-hin
Translated output : target lang-pan
Human Translation : target lang-pan(human)

Non-Summer-of-Code plans for the Summer

Since I'll be having my college summer vacations for almost the entire duration of the project, I can easily spend 30-40 hours per week on the project. Since, the academic schedule might vary a little bit due to lock downs for prevention of COVID-19, I'll be starting work early and cover the problems in the post-application period. I've also kept workload slightly heavier in the first 2 weeks to cover up any unlikely, uncertain extensions in academics that might show up. Even then, I can spend around 20 hours a week in any case(note that this is a very unlikely situation and even then this period won't last more than a couple of weeks since the coursework is already underway online and is expected to be over before start of the project).