Difference between revisions of "User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi/progress"
Jump to navigation
Jump to search
(5 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
Progress on [https://wiki.apertium.org/wiki/User:Pmodi/GSOC_2020_proposal:_Hindi-Punjabi] |
Progress on [https://wiki.apertium.org/wiki/User:Pmodi/GSOC_2020_proposal:_Hindi-Punjabi] |
||
− | ==Current task(Community bonding Week |
+ | ==Current task(Community bonding Week 2)== |
+ | ===DONE=== |
||
− | ===Finalising Frequency lists & Learning Persian script=== |
||
⚫ | |||
+ | * DONE - Added bidirectional dictionary(33k paradigms) |
||
* DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied) |
* DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied) |
||
− | * DONE - Gathered a sufficiently long parallel corpus. Currently cleaning up the xml files and generating aligned, text files for the two languages. |
||
* DONE - Scraped all Wikipedia texts and made a combined frequency list. |
* DONE - Scraped all Wikipedia texts and made a combined frequency list. |
||
* DONE - Frequency lists using WikiExtractor on latest dump. |
* DONE - Frequency lists using WikiExtractor on latest dump. |
||
+ | |||
+ | ===IN PROGRESS=== |
||
* IN PROGRESS - Practicing basic understanding of Persian script. |
* IN PROGRESS - Practicing basic understanding of Persian script. |
||
⚫ | |||
* IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists. |
* IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists. |
||
− | ==TODO(Next week - Community Bonding Week |
+ | ==TODO(Next week - Community Bonding Week 3)== |
− | * Fix transfer rules for postpositions. |
+ | * Fix transfer rules for postpositions(fixing some issues). |
* Start work on function words addition to dictionary. |
* Start work on function words addition to dictionary. |
||
==Literature(Apertium Wiki) Covered== |
==Literature(Apertium Wiki) Covered== |
||
+ | * DONE - Calculating Coverage. |
||
+ | * DONE - A long introduction on Transfer Rules. |
||
+ | * DONE - Transfer Rules examples |
||
* DONE - Wikipedia Dumps. |
* DONE - Wikipedia Dumps. |
||
* DONE - Generating Frequency lists. |
* DONE - Generating Frequency lists. |
||
* DONE - Building Dictionaries#Monolingual. |
* DONE - Building Dictionaries#Monolingual. |
||
* DONE - Evaluation. |
* DONE - Evaluation. |
||
+ | |||
+ | == Progress table == |
||
+ | |||
+ | {|class=wikitable |
||
+ | |- |
||
+ | !colspan="2"|Week |
||
+ | !colspan="3"|Stems |
||
+ | !colspan="3"|Coverage |
||
+ | !colspan="2"|WER,PER |
||
+ | !colspan="2"|Progress |
||
+ | |- |
||
+ | ! № |
||
+ | ! dates |
||
+ | ! hin |
||
+ | ! pan |
||
+ | ! hin-pan |
||
+ | ! hin |
||
+ | ! pan |
||
+ | ! hin-pan |
||
+ | ! hin→pan |
||
+ | ! pan→hin |
||
+ | !Evaluation |
||
+ | !Notes |
||
+ | |} |
Revision as of 15:37, 19 May 2020
Progress on [1]
Contents
Current task(Community bonding Week 2)
DONE
- DONE - Collected parallel texts to calculate WER,PER etc..
- DONE - Added bidirectional dictionary(33k paradigms)
- DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied)
- DONE - Scraped all Wikipedia texts and made a combined frequency list.
- DONE - Frequency lists using WikiExtractor on latest dump.
IN PROGRESS
- IN PROGRESS - Practicing basic understanding of Persian script.
- IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists.
TODO(Next week - Community Bonding Week 3)
- Fix transfer rules for postpositions(fixing some issues).
- Start work on function words addition to dictionary.
Literature(Apertium Wiki) Covered
- DONE - Calculating Coverage.
- DONE - A long introduction on Transfer Rules.
- DONE - Transfer Rules examples
- DONE - Wikipedia Dumps.
- DONE - Generating Frequency lists.
- DONE - Building Dictionaries#Monolingual.
- DONE - Evaluation.
Progress table
Week | Stems | Coverage | WER,PER | Progress | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
№ | dates | hin | pan | hin-pan | hin | pan | hin-pan | hin→pan | pan→hin | Evaluation | Notes |