User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi/progress
Jump to navigation
Jump to search
Progress on [1]
Contents
Current task(Community bonding Week 2)
DONE
- DONE - Trying to collect parallel texts to calculate WER.
- DONE - Added bidirectional dictionary(33k paradigms)
- DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied)
- DONE - Gathered a sufficiently long parallel corpus. Currently cleaning up the xml files and generating aligned, text files for the two languages.
- DONE - Scraped all Wikipedia texts and made a combined frequency list.
- DONE - Frequency lists using WikiExtractor on latest dump.
IN PROGRESS
- IN PROGRESS - Practicing basic understanding of Persian script.
- IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists.
TODO(Next week - Community Bonding Week 3)
- Fix transfer rules for postpositions(fixing some issues).
- Start work on function words addition to dictionary.
Literature(Apertium Wiki) Covered
- DONE - Calculating Coverage.
- DONE - A long introduction on Transfer Rules.
- DONE - Transfer Rules examples
- DONE - Wikipedia Dumps.
- DONE - Generating Frequency lists.
- DONE - Building Dictionaries#Monolingual.
- DONE - Evaluation.
Status table
| Week | Stems | native coverage | WER,PER | Progress | |||||
|---|---|---|---|---|---|---|---|---|---|
| № | dates | uzb | uzb-kaa | uzb | uzb-kaa | uzb→kaa | uzb→kaa | Evaluation | Notes |
