User:Eden - Revision history

Eden at 05:02, 19 May 2020

2020-05-19T05:02:57Z

Eden at 10:59, 28 March 2020

2020-03-28T10:59:50Z

Show changes

Eden: /* Coding challenge */

2019-04-08T07:10:35Z

Coding challenge

Eden: /* Skills and qualifications */

2019-04-08T06:48:27Z

Skills and qualifications

Eden at 06:44, 8 April 2019

2019-04-08T06:44:20Z

Eden: /* Lingala resources */

2019-04-08T06:05:51Z

Lingala resources

Eden: /* Coding challenge */

2019-04-08T06:02:51Z

Coding challenge

Firespeaker: /* Coding challenge */

2019-04-08T01:50:19Z

Coding challenge

Eden at 03:03, 4 April 2019

2019-04-04T03:03:36Z

Show changes

Francis Tyers: /* Non-Summer-of-Code plans */

2019-03-21T08:23:56Z

Non-Summer-of-Code plans

@@ Line 1: / Line 1: @@
 == Contact Information ==
 Name: Eden-Grace Muamba<br />
@@ Line 13: / Line 11: @@
 My main focus is on bringing Bantu languages to Apertium.
-== Work Done ==
+== Things I work on ==
+=== Transducers ===
-'''English-Lingala ''':  [[User:Eden/GSOC2019_English-Lingala]]  <br/>
-'''English-Swahili(Proposal) ''':  [[User:Eden/GSOC2020Proposal_English-Swahili]]
+{|class="wikitable sortable"
+|-
+! project !! language !! stems !! coverage
+|-
+! [https://github.com/thefreezer/apertium-swa apertium-swa]
+| Swahili ||align="right"|  ||align="center"|
+|-
+! [https://github.com/apertium/apertium-lin apertium-lin]
+| Lingala ||align="right"|  ||align="center"|
+|}
+=== MT-pair ===
+{|class="wikitable sortable"
+|-
+! project !! languages !! stems !! coverage
+|-
+! [https://github.com/apertium/apertium-eng-lin apertium-eng-lin]
+| English-Lingala
+|align="right"| ||
+|-
+! [https://github.com/thefreezer/apertium-eng-swa apertium-swa-eng]
+| Swahili-English
+|align="right"|  ||
+|-
+! [apertium-swa-lin]
+| Swahili-Lingala
+|align="right"|  ||
+|}

@@ Line 37: / Line 37: @@
 '''Update 1: Apr/1/19'''<br/>
 . Added ~95% of all words from this [https://sourceforge.net/p/apertium/svn/HEAD/tree/branches/xupaixkar/rasskaz/ story]. <br/>
-. From the ''493''-word story, my final translation has ''74'' unknown words(*) and ''63'' words with the wrong final form(#). Most of them are verbs, adj and adv. Original story is [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/story.txt here] and [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/output.txt here] is the final output.<br/>(eng-lin)
+. From the ''493''-word story, my final translation has ''74'' unknown words(*) and ''63'' words with the wrong final form(#). Most of them are verbs, adj and adv. Original story is [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/story_eng.txt here] and [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/eng-lin-output.txt here] is the final output.<br/>(eng-lin)
 . Added 8 rules which give me correct translations for:<br/>
 * prn/np vblex/vbhaver/vbser det n (eg. I see a house) with correct present and past(saw) verb tenses

@@ Line 126: / Line 126: @@
 == Skills and qualifications ==
 Ongoing major: first year Computer Science students with a minor in Statistics<br />
-Relevant technical skills: python(online data mining, inferential statistics, numpy, pandas, matplotlib), c++(proficient), sql(elementary), git(proficient), bash(proficient), html5/css3(advanced)<br />
+Relevant technical skills: python(online data mining, inferential statistics, numpy, pandas, matplotlib), c++(elementary), sql(intermediate), git(intermediate), bash(intermediate), html5/css3(advanced)<br />
 Work experience: as an intern created static and dynamic websites<br />
 Languages: French(native), Lingala(native), English(Fluent) , Swahili(proficient), Tshiluba(proficient), Twi(elementary)<br />

@@ Line 11: / Line 11: @@
 == My goal ==
-I’m planning to start the ‘English-Lingala’ language pair.
+I’m planning to start the ‘English-Lingala’ language pair.<br/>
+At first I was only planning to work in one direction(eng-lin), but following ''firespeaker'' 's suggestion, I will also work in the other direction to make my mentor's job easier.
 == Why am I interested in Apertium? ==
@@ Line 36: / Line 37: @@
 '''Update 1: Apr/1/19'''<br/>
 . Added ~95% of all words from this [https://sourceforge.net/p/apertium/svn/HEAD/tree/branches/xupaixkar/rasskaz/ story]. <br/>
-. From the ''493''-word story, my final translation has ''74'' unknown words(*) and ''63'' words with the wrong final form(#). Most of them are verbs, adj and adv. Original story is [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/story.txt here] and [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/output.txt here] is the final output.<br/>
+. From the ''493''-word story, my final translation has ''74'' unknown words(*) and ''63'' words with the wrong final form(#). Most of them are verbs, adj and adv. Original story is [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/story.txt here] and [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/output.txt here] is the final output.<br/>(eng-lin)
 . Added 8 rules which give me correct translations for:<br/>
 * prn/np vblex/vbhaver/vbser det n (eg. I see a house) with correct present and past(saw) verb tenses
@@ Line 52: / Line 53: @@
 == Work plan ==
-(this page will frequently change as I get more familiar with Apertium)
  community bonding period
- - reading more about transfer-rules and creating a doc for Lingala rules
+ - reading more about transfer-rules and creating a doc for eng-lin lin-eng rules
  - build a better frequency list of Lingala words
  - reading more about the HFST
+ - continue work and achieve a WER<50% from at least one direction
  Week 1:
  - adding nouns(from frequency list) in the lin transducer
- - adding verbs in the lin transducer
+ - adding verbs with correct tenses in the lin transducer
  - constraint grammar
  Week 2:
  - adding pronouns and adjectives in the lin transducer
- - adding adverbs, conjunctions, prepositions, etc
+ - also adding adverbs, conjunctions, prepositions, etc
  - constraint grammar for prn and adj
  Week 3:
- - polishing the transducer to give better analyses(eg.
+ - polishing the transducer to give better analyses
  - filling nouns and adjectives in bilingual dictionary,
  - regression testing
  Week 4:
- - transfer rules for nouns and adjectives
+ - transfer rules for nouns and adjectives(both directions)
  - disambiguation rules
@@ Line 82: / Line 83: @@
  - continue work on bilingual dictionary,
  - main work will be on verbs
- - transfer rules for verbs
+ - transfer rules for verbs in both directions
  Week 6:
  - filling pronouns, adverbs, and others in the bidix
  - work on compound Lingala words
+ - transfer rules for pronouns, adverbs and compound nouns(both directions)
  Week 7:
- - adding pronouns, determinants, adverbs, and others in the bidix
+ - adding determinants and more adjectives in the bidix
- - WER < 25% on a 500 word story
+ - WER < 35% on a 500 word story
  - add/polish rules for concordance between verbs and pronouns
  Week 8:
  - continue work on transfer rules in .t2x and t3x files
- - work on disambiguation,
+ - work on disambiguation(eng-lin, lin-eng)
  - lots of testing and improvement of bilingual dictionary
- - WER < 25% on a 1000 word story
+ - WER < 30% in both directions on a 1000 word story
-* '''Deliverable #2''' Advanced bilingual dictionary and transfer rules
+* '''Deliverable #2''' Advanced bilingual dictionary(~5,000 words) and transfer rules
  Week 9 :
- - continue work on disambiguation
+ - continue work on disambiguation(both directions)
  - testvoc and improvements
+ - filling bidix(common nouns)
  Week 10:
  - work on transfer rules,
- - goal is WER < 20% on a story greater > 1000 words(is this achievable?)
+ - goal is WER < 30% on a story greater > 1000 words(is this achievable?)
  Week 11:
  - continue work on transfer rules and testing,
  - Wikipedia article translations
+ - continue filling bidix
  Week 12:
+ - filling bidix with miscellaneous words
  - detailed analysis of work completed(wiki),
  - evaluation of results and documentation
-* '''Project completed''' Goal is to have a WER < ~25% on most texts
+* '''Project completed''' WER~30%(with ~7,000 words in bidix) in both directions on most texts
 == Skills and qualifications ==
@@ Line 126: / Line 131: @@
 == Non-Summer-of-Code plans ==
-Traveling to Ontario for 5 days from June 29, but that will not affect my work. I’m committed to put it at least 40+ hours a week for the duration of the project.
+Traveling to Ontario for 5 days from June 29, but that will not affect my work. No other commitments which will allow me to put it at least 40+ hours a week for the duration of the project.

@@ Line 24: / Line 24: @@
 == Lingala resources ==
-Given that Lingala is mostly a spoken language, there isn't al
 Here is a list of ''open'' and ''public domain'' resources(dictionaries, grammar books, texts, etc) for the Lingala language:<br/>
 - [http://crubadan.org/languages/ln Crubadan text corpus] A text corpus sorted by word frequency<br/>

@@ Line 35: / Line 35: @@
 == Coding challenge ==
 All my work is in my repo: https://github.com/thefreezer/GSOC-apertium-eng-lin <br />
-'''Update 1'''<br/>
+'''Update 1: Apr/1/19'''<br/>
 . Added ~95% of all words from this [https://sourceforge.net/p/apertium/svn/HEAD/tree/branches/xupaixkar/rasskaz/ story]. <br/>
 . From the ''493''-word story, my final translation has ''74'' unknown words(*) and ''63'' words with the wrong final form(#). Most of them are verbs, adj and adv. Original story is [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/story.txt here] and [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/output.txt here] is the final output.<br/>
@@ Line 46: / Line 46: @@
 {{comment|It would be good to be able to evaluate WER, so a correct Lingala version of the story would be very useful —[[User:Firespeaker|Firespeaker]] ([[User talk:Firespeaker|talk]]) 03:50, 8 April 2019 (CEST).}}
+'''Update 2: Apr/7/19'''<br/>
+. Added the full Lingala translation [https://github.com/thefreezer/GSOC-apertium-eng-lin/blob/master/story_lin.txt here].<br/>
+. lin-eng: ''75.27% WER''<br/>
+. eng-lin: ''85.65% WER'' (I mostly focused on lin-eng, which explains why this direction is higher)<br/>
 == Work plan ==

@@ Line 44: / Line 44: @@
 I will try to implement a rule for dealing with the future tense(eg. I will play ...)<br/>
 ''Note: a lot of these rules are inspired from the eng-fra pair''
+{{comment|It would be good to be able to evaluate WER, so a correct Lingala version of the story would be very useful —[[User:Firespeaker|Firespeaker]] ([[User talk:Firespeaker|talk]]) 03:50, 8 April 2019 (CEST).}}
 == Work plan ==