User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi/progress
Jump to navigation Jump to search
Current tasks(Week 1)
- Fix some discrepancies due to postpositions(posttransfer)
- Add Adverbs.
- Clean up parallel text and compute WER.
|2||May 24-31||-||+1400||-||-||52%||-||36.83%, 33.84%||40.03%, 36.93%||-||WER, PER On a set of 25 sentences(612 words)|
- DONE - Added about 1400 adjective stems
- DONE - Function words(cnj, det, prn, post, gen_endings), Coverage > 47%
- DONE - Collected parallel texts to calculate WER,PER etc..
- DONE - Added bidirectional dictionary(33k paradigms)
- DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied)
- DONE - Scraped all Wikipedia texts and made a combined frequency list.
- DONE - Frequency lists using WikiExtractor on latest dump.
- IN PROGRESS - Adding Adverbs.
- IN PROGRESS - Fix inconsistencies with parallel texts.
- IN PROGRESS - Practicing basic understanding of Persian script.
- IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists(might not be so beneficial since Punajbi and Hindi share inflections for different word forms)
TODO(Next week - Community Bonding Week 3)
- Fix transfer rules for postpositions(fixing some issues).
- Start work on function words addition to dictionary.
Literature(Apertium Wiki) Covered
- DONE - Calculating Coverage.
- DONE - A long introduction on Transfer Rules.
- DONE - Transfer Rules examples
- DONE - Wikipedia Dumps.
- DONE - Generating Frequency lists.
- DONE - Building Dictionaries#Monolingual.
- DONE - Evaluation.
- DONE - Extract.
- DONE - Monodix Basics
- DONE - Improved Corpus Based Paradigm Matching.
- DONE - Transliteration.