Difference between revisions of "User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi/progress"
Jump to navigation
Jump to search
| Line 1: | Line 1: | ||
==Current tasks(Week |
==Current tasks(Week 2)== |
||
* Fix some discrepancies due to postpositions(posttransfer) |
* Fix some discrepancies due to postpositions(posttransfer) |
||
* Complete manual analysis of bidix entries for adjectives and adverbs. |
|||
| ⚫ | |||
* Add more verbs. |
|||
* Clean up parallel text and compute WER. |
|||
* Run testvoc on Adjectives and Adverbs. |
|||
== Progress == |
== Progress == |
||
| Line 73: | Line 74: | ||
===DONE=== |
===DONE=== |
||
| ⚫ | |||
* DONE - Added about 1400 adjective stems |
* DONE - Added about 1400 adjective stems |
||
* DONE - Function words(cnj, det, prn, post, gen_endings), Coverage > 47% |
* DONE - Function words(cnj, det, prn, post, gen_endings), Coverage > 47% |
||
| Line 87: | Line 89: | ||
* IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists(might not be so beneficial since Punajbi and Hindi share inflections for different word forms) |
* IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists(might not be so beneficial since Punajbi and Hindi share inflections for different word forms) |
||
==TODO(Next week - Week |
==TODO(Next week - Week 3)== |
||
* Fix transfer rules for postpositions(fixing some issues). |
* Fix transfer rules for postpositions(fixing some issues). |
||
* Add verbs. |
* Add verbs and nouns. |
||
* Get bidixes up-to-date with stems added to monodixes. |
* Get bidixes up-to-date with stems added to monodixes. |
||
Revision as of 21:39, 8 June 2020
Contents
Current tasks(Week 2)
- Fix some discrepancies due to postpositions(posttransfer)
- Complete manual analysis of bidix entries for adjectives and adverbs.
- Add more verbs.
- Run testvoc on Adjectives and Adverbs.
Progress
Progress table
| Week | Stems | Coverage | WER,PER | Progress | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| № | dates | hin | pan | hin-pan | hin | pan | hin-pan | hin→pan | pan→hin | Evaluation | Notes |
| 1 | May 1-24 | - | - | - | - | 47% | - | - | - | - | - |
| 2 | May 24-31 | - | +1400 | - | - | 52% | - | 36.83%, 33.84% | 40.03%, 36.93% | - | WER, PER On a set of 25 sentences(612 words) |
| 3 | June 1-7 | - | +500 | +2500 | - | 58% | - | 49.52%, 42.71% | 48.65%, 44.03% | - | WER, PER On a set of 50 sentences(1038 words) |
DONE
- DONE - Added Adverbs.
- DONE - Added about 1400 adjective stems
- DONE - Function words(cnj, det, prn, post, gen_endings), Coverage > 47%
- DONE - Collected parallel texts to calculate WER,PER etc..
- DONE - Added bidirectional dictionary(33k paradigms)
- DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied)
- DONE - Scraped all Wikipedia texts and made a combined frequency list.
- DONE - Frequency lists using WikiExtractor on latest dump.
IN PROGRESS
- IN PROGRESS - Adding Adverbs.
- IN PROGRESS - Fix inconsistencies with parallel texts.
- IN PROGRESS - Practicing basic understanding of Persian script.
- IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists(might not be so beneficial since Punajbi and Hindi share inflections for different word forms)
TODO(Next week - Week 3)
- Fix transfer rules for postpositions(fixing some issues).
- Add verbs and nouns.
- Get bidixes up-to-date with stems added to monodixes.
Literature(Apertium Wiki) Covered
- DONE - Calculating Coverage.
- DONE - A long introduction on Transfer Rules.
- DONE - Transfer Rules examples
- DONE - Wikipedia Dumps.
- DONE - Generating Frequency lists.
- DONE - Building Dictionaries#Monolingual.
- DONE - Evaluation.
- DONE - Extract.
- DONE - Monodix Basics
- DONE - Improved Corpus Based Paradigm Matching.
- DONE - Transliteration.
- DONE - Workflow reference.
- DONE - Tagger Tranining.
- DONE - Modes introduction.
- DONE - Apertium-viewer.
