User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi/progress
< User:Pmodi | GSOC 2020 proposal: Hindi-PunjabiJump to navigation Jump to search
Current tasks(Week 5)
- Fix सब__prn for खुद
- Fix determiners in Hindi (Add काफी to determiners)
- Add post-positions to definitions - ਹਸਪਤਾਲੋਂ = ਹਸਪਤਾਲ ਤੋਂ.
- What is the analysis for ਦੋਸਤੋਂ.
- ਮੈਂ is the translation for मैंने and मैं. Add transfer rule and check if tagger correctly identifies which one is being picked.
- Fix alternate spellings analyses..
- कि v/s की punjabi ki can also be kya.
- Fix gender for adj - ਬਹੁਤੀ
- Complete manual analysis of bidix entries for adjectives and adverbs.
- Add more verbs.
- Run testvoc on Adjectives and Adverbs.
|1||May 1-24||-||-||-||-||47%||-||-||-||-||Original coverage - 12.8%|
|2||May 24-31||-||+1400||-||-||52%||-||36.83%, 33.84%||40.03%, 36.93%||-||WER, PER On a set of 25 sentences(612 words)|
|3||June 1-7||-||+500||+250||-||58%||-||49.52%, 42.71%||48.65%, 44.03%||-||WER, PER On a set of 50 sentences(1038 words)|
|4||June 8-14||+50||+1500||+1000||-||63%||-||41.36%, 35.82%||41.47%, 37.03%||-||WER, PER On a set of 50 sentences(1038 words)|
|5||June 15-21||-||+50||+200||-||66%||52.8%||41.36%, 35.82%||41.47%, 37.03%||-||WER, PER On a set of 50 sentences(1038 words)|
|6||June 22-28||-||+350||+700||-||70%||59%||39.76%, 34.22%||40.03%, 36.23%||-||WER, PER On a set of 50 sentences(1038 words)|
- DONE - Bidix updates.
- DONE - Fixed errors with postposition transfer.
- DONE - Added proper nouns.
- DONE - Added noun paradigms.
- DONE - Added Adverbs.
- DONE - Added about 1400 adjective stems
- DONE - Function words(cnj, det, prn, post, gen_endings), Coverage > 47%
- DONE - Collected parallel texts to calculate WER,PER etc..
- DONE - Added bidirectional dictionary(33k paradigms)
- DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied)
- DONE - Scraped all Wikipedia texts and made a combined frequency list.
- DONE - Frequency lists using WikiExtractor on latest dump.
- IN PROGRESS - Adding Verbs.
- IN PROGRESS - Fix inconsistencies with parallel texts.
- IN PROGRESS - Practicing basic understanding of Persian script.
TODO(Next week - Week 3)
- Add verbs and nouns.
Literature(Apertium Wiki) Covered
- DONE - Calculating Coverage.
- DONE - A long introduction on Transfer Rules.
- DONE - Transfer Rules examples
- DONE - Wikipedia Dumps.
- DONE - Generating Frequency lists.
- DONE - Building Dictionaries#Monolingual.
- DONE - Evaluation.
- DONE - Extract.
- DONE - Monodix Basics
- DONE - Improved Corpus Based Paradigm Matching.
- DONE - Transliteration.
- DONE - Workflow reference.
- DONE - Tagger Tranining.
- DONE - Modes introduction.
- DONE - Apertium-viewer.