User:Iamas/GSoC13 Application: "Improved Bilingual Dictionary Induction"
Name
Arnav Sharma
Contact Information
- E-mail : arnavsharma93@gmail.com
- GitHub : arnavsharma93
- IRC : iamas, arnavsharma93
- SourceForge : iamas
Why am I interested in Machine Translation?
Machine Translation is an important technology for localization, and is particularly relevant in a linguistically diverse country like India. Machine Translation can help reduce the language barrier. That motivated me to study Computational Linguistics in IIIT-H. I am currently working in the Machine Translation Department of IIIT-H.
Why am I interested in the Apertium Project?
I have been fascinated by FOSS and open source software since the time I heard about it. As mentioned above, Machine Translation and computational linguisitics interests me a lot. Apertium combines both of these factors. Plus, I really like Begiak.
Which of the published tasks am I interested in? What do I plan to do?
I am interested in the project Improved Bilingual Dictionary Induction. The aim is to write a set of scripts that generate valid entries for a bidix from the word aligned parallel corpus and also to evaluate the reliability of the extracted translations.
Proposal Title
Improved bilingual dictionary induction
Why Apertium and Google should sponsor it?
Bilingual Dictionary is one of the five main dictionaries used in Apertium. This project involves generating valid and consistent Apertium bilingual dictionary entries from a word-aligned parallel corpus. There exist such tools but most of the generated entries have to be checked, which can greatly increase the amount of time it takes to make a new translation system. This will greatly benefit the lexicographers and other contributors and will help in reducing the effort and time taken to make new translation system.
Work Plan
Coding Challenge
The coding challenge involved:
- Install Apertium
- Install GIZA++
- Generate a word alignment model for a parallel corpus of your choice.
- Rewrite the script generate-bidix-templates.py to use python3/ElementTree.
I have finished the coding challenge.
Interim period and community bonding period
- Get to know the community better
- Habituate myself with the Apertium platform and project
- Make preparations and gain necessary information that will help me in the coding period.
- Contribute by solving bugs, rewriting scripts and contributing to the language pairs Hindi-Punjabi and Hindi-Urdu.
Week Plan
| WEEK | DATE | PLANS |
|---|---|---|
| Week 01 | 06.17-06.23 | Make word aligned data after running morph analyzer on a parallel corpus of at least three language pairs. |
| Week 02 | 06.24-06.30 | Write a script to generate the list of word alignments. |
| Week 03 | 07.01-07.07 | Write a script to extract the most frequent combinations of paradigms in Source Language - Transfer Language. |
| Week 04 | 07.08-07.14 | Complete the script which generates templates for user's selection. |
| Deliverable #1 | Developed templating system | |
| Week 05 & 06 | 07.15-07.28 | Write a script which checks to see whether the source language paradigm has a template with the transfer language paradigm. |
| Week 07 & 08 | 07.29-08.11 | Write a script to make bidix entries in an incremental fashion. |
| Deliverable #2 | Script that makes bidix entries in an incremental fashion. | |
| Week 09 | 08.12-08.18 | EVALUATION: MANUAL: Check a few dictionaries by randomly sampling a few hundred words. |
| Week 10 & 11 | 08.19-09.01 | EVALUATION: AUTOMATIC: Compare the bidix dictionary against some online web dictionaries |
| Week 11 | 08.25-09.01 | Combine all of the scripts above to automate all the tasks. |
| Week 12 | 09.02-09.08 | Improve code, add lots of comments and write wiki for all of the scripts usage. |
| Deliverable #3 | Final project |
Biography
I am currently pursuing Bachelor of Technology in Computer Science and MS by Research in Computational Linguistics at IIIT-H. I have just finished my second year in that. I have been studying the various fields of Computational Linguistics for the past two years and I can not wait to study more. I am proficient in Python, C/C++, Bash, SQL and HTML5. I have developed an Urdu-Hindi transliterator using NLP tools. It gave an accuracy of 75%.
Non-Summer-of-Code plans for the summer
I might have to go for a social entrepreneurship trip for 3 days in July. Also, I plan on improving my programming skills by taking part in algorithmic coding competitions. Otherwise, I have nothing else planned for the summer. This project will be my main priority.
