Difference between revisions of "User:Pmodi/GSOC 2020 proposal: Hindi-Punjabi/progress"

Revision as of 21:31, 8 June 2020

Week		Stems			Coverage			WER,PER		Progress
№	dates	hin	pan	hin-pan	hin	pan	hin-pan	hin→pan	pan→hin	Evaluation	Notes
1	May 1-24	-	-	-	-	47%	-	-	-	-	-
2	May 24-31	-	+1400	-	-	52%	-	36.83%, 33.84%	40.03%, 36.93%	-	WER, PER On a set of 25 sentences(612 words)
3	June 1-7	-	+500	-	-	58%	-	49.52%, 42.71%	48.65%, 44.03%	-	WER, PER On a set of 50 sentences(1038 words)

DONE - Added about 1400 adjective stems
DONE - Function words(cnj, det, prn, post, gen_endings), Coverage > 47%
DONE - Collected parallel texts to calculate WER,PER etc..
DONE - Added bidirectional dictionary(33k paradigms)
DONE - Fixed bidirectional translation i.e. pan->hin(gave close to human translation for small test set, even though similar transfer rules were copied)
DONE - Scraped all Wikipedia texts and made a combined frequency list.
DONE - Frequency lists using WikiExtractor on latest dump.

IN PROGRESS - Adding Adverbs.
IN PROGRESS - Fix inconsistencies with parallel texts.
IN PROGRESS - Practicing basic understanding of Persian script.
IN PROGRESS - Extracting stems and suffix/prefix patterns using the frequency lists(might not be so beneficial since Punajbi and Hindi share inflections for different word forms)

@@ Line 55: / Line 55: @@
 ! -
 ! WER, PER On a set of 25 sentences(612 words)
-|
+|-