Difference between revisions of "User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi/progress"

From Apertium
Jump to navigation Jump to search
Line 32: Line 32:
 
|-
 
|-
 
!colspan="2"|Week
 
!colspan="2"|Week
!colspan="2"|Stems
+
!colspan="3"|Stems
!colspan="2"|native coverage
+
!colspan="3"|Coverage
 
!colspan="2"|WER,PER
 
!colspan="2"|WER,PER
 
!colspan="2"|Progress
 
!colspan="2"|Progress
Line 39: Line 39:
 
! №
 
! №
 
! dates
 
! dates
! uzb
+
! hin
! uzb-kaa
+
! pan
! uzb
+
! hin-pan
! uzb-kaa
+
! hin
  +
! pan
! uzb→kaa
 
  +
! hin-pan
! uzb→kaa
 
  +
! hin→pan
  +
! pan→hin
 
!Evaluation
 
!Evaluation
 
!Notes
 
!Notes

Revision as of 15:33, 19 May 2020

Progress on [1]

Current task(Community bonding Week 2)

DONE

  • DONE - Trying to collect parallel texts to calculate WER.
  • DONE - Added bidirectional dictionary(33k paradigms)
  • DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied)
  • DONE - Gathered a sufficiently long parallel corpus. Currently cleaning up the xml files and generating aligned, text files for the two languages.
  • DONE - Scraped all Wikipedia texts and made a combined frequency list.
  • DONE - Frequency lists using WikiExtractor on latest dump.

IN PROGRESS

  • IN PROGRESS - Practicing basic understanding of Persian script.
  • IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists.

TODO(Next week - Community Bonding Week 3)

  • Fix transfer rules for postpositions(fixing some issues).
  • Start work on function words addition to dictionary.

Literature(Apertium Wiki) Covered

  • DONE - Calculating Coverage.
  • DONE - A long introduction on Transfer Rules.
  • DONE - Transfer Rules examples
  • DONE - Wikipedia Dumps.
  • DONE - Generating Frequency lists.
  • DONE - Building Dictionaries#Monolingual.
  • DONE - Evaluation.

Status table

Week Stems Coverage WER,PER Progress
dates hin pan hin-pan hin pan hin-pan hin→pan pan→hin Evaluation Notes