View source for User:Eden/GSOC2020 Swahili-Lingala

== Goal ==
Create a usable ‘Swahili-Lingala’ language pair. <br/>

== Swahili and Lingala resources ==
Here is a list of ''open'' and ''public domain'' resources(dictionaries, grammar books, texts, etc) for Swahili:<br/>

*Corpus/frequency list/bigram
- [https://github.com/thefreezer/apertium-swa/blob/master/dev/wikipedia_corpus.txt ~7m word corpus](needs a little bit more work) <br/>
- [http://crubadan.org/writingsystems An Crúbadán] <br />
*Dictionary
- [https://kamusi.org/swahili-english-wordlist-2008 Swa-Eng] and [https://kamusi.org/english-swahili-wordlist-2008 Eng-Swa]<br/>
- [https://archive.org/details/swahilienglishdi00mada/page/n15/mode/2up Madan A.C.,1846], [https://archive.org/details/englishswahilid00madagoog/page/n13/mode/2up Madan A. C.,1902][https://archive.org/details/Swahili-englishDictionary/mode/2up Charles, W. R.]<br/>
- [https://github.com/freedict/fd-dictionaries/tree/master/swh-eng Freedict] <br/>
*Grammar rules
- [https://en.wikipedia.org/wiki/Swahili_grammar Wikipedia's Grammar Rules]<br/>
- [https://archive.org/details/swahiligrammarvo00burtiala/page/96/mode/2up Burt, A. E,1910] <br/>
- [https://archive.org/details/ERIC_ED012888/page/n123/mode/2up Follome]<br/>
- [https://archive.org/details/ERIC_ED012888/page/n123/mode/2up Steerie, Edward]<br/>
*Other
- [http://www.language-archives.org/language/swh Language Archive] <br/>
- [https://wals.info/languoid/lect/wals_code_swa WALS] <br/>

== Work plan ==
Community bonding period(May 4-June 1)
 - See [[User:Eden/GSoC_progress]]

 Week 1(June 1-7): 
 - adding nouns(from frequency list) in the lin transducer
 - Add nouns (from frequency list) in the swa transducer
 - Work on vowels
 - Constraint grammar for nouns
 - Add verbs

 Week 2(June 8-14):
 - adding pronouns and adjectives in the swa transducer 
 - Continue work on verbs
 - Reference: kaz and lin transducers
 - Add prepositions and pronouns, conjunctions
 - Work on numerals
 - CG for all the above

 Week 3(June 15-21):  
 - Regression testing
 - Test and polish transducer(work on bi-grams)
 - Finish adding adverbs, conjunctions, prepositions, etc
 - Start work on bilingual dictionary

 Week 4(June 22-28):  
 - Add nouns and adjectives in bidix
 - Transfer rules for nouns and adjectives(both directions)
 - Disambiguation rules

* '''Deliverable #1(June 29): Advanced Swahili transducer(>10k entries) with basic bilingual dictionary''' 

 Week 5(June 29-July 5):  
 - Continue work on bidix: add nouns and verbs 
 - Focus on verbs
 - Transfer rules from eng-lin, kaz-eng, and eng-fre
 - Transfer rules for verbs in both directions

 Week 6(July 6-12):  
 - Add pronouns and transfer rules for them
 - Add adverbs
 - Wok on compound Swahili words
 - Transfer rules for pronouns, adverbs and compound nouns(both directions)

 Week 7(July 13-19): 
 - Goal: well defined macros for verbs and pronouns
 - WER < 35% on 500 word story
 - add/polish rules for concordance between verbs and pronouns

 Week 8(July 20-26): 
 - Continue work on transfer rules
 - Work on disambiguation rules
 - Lots of testing and improvements
 - WER < 30% in both directions on a 1,000-word story

* '''Deliverable #2(July 3): Advanced bilingual dictionary(~15,000 words) and transfer rules''' ...

 Week 9(July 27-August 2) :
 - Continue work on disambiguation(both directions)
 - Testvoc and improvements
 - Filling bidix

 Week 10(August 3-9):
 - Work on transfer rules
 - goal is WER ~30% on a story greater > 1000 words

 Week 11(August 10-16):
 - Continue work on transfer rules and testing
 - Wikipedia article translations
 - Continue filling bidix

 Week 12(August 17-23):
 - Continue filling bidix with miscellaneous words
 - Detailed analysis of work completed(wiki)
 - (if work done well, start working on new pairs)
 - Evaluation of results and documentation

* '''Submit Code and Final Evaluations(August 24-31): WER < 30%(with ~20,000 words in bidix) in both directions on most texts
'''