Apertium-kaz-kir/Workplan

Major goals

• Good WER
• Clean testvoc
• 12'000 stems in bidix (~1000 stems per week, or ~200 per day)
• Sort Adjective and Noun stems in kir.lexc into appropriate categories
• Trimmed coverage approaching 90%

Schedule

Timeline

See GSoC 2013 Timeline for complete timeline. Important coding dates follow:

• June 17th: coding begins
• July 29th - August 2nd: midterm evaluations
• September 16th - September 23rd: pencils down
• September 27th: final evaluation

Workplan

week	dates	goals	eval	accomplishments	notes
post-application period 3 - 24 May		finish coding challenge with WER ~10% trimmed coverage 45% total 250 stems in dix	4/5 good	coding challenge: WER ~9% trimmed coverage: 52%,48% stems in dix: 380	Demonstrated ability to add stems to dix and lexc. A couple easy lexical selection rules are still not written. Needs to learn more about other aspects of apertium and evaluation. —Firespeaker 06:45, 20 May 2013 (UTC)
community bonding period 27 May - 16 June		run first testvoc run coverage scripts get first frequency lists write ≥4 lexical selection rules write ≥2 transfer rules write ≥3 disambig rules note: should be in IRC every day	3/5 okay	— ran trimmed coverage script on a corpus took a look at frequency lists wrote 4 pairs of lexical selection rules wrote 4 variants of 1 transfer rule —	demonstrated ability to work with lexical selection rules demonstrated ability to work with transfer rules got only some experience with coverage scripts did not get experience with testvoc did not get experience with disambig rules was not around IRC frequently worked in bursts, did not spend a single long period of time —Firespeaker 02:28, 2 July 2013 (UTC)
1	17 - 22 June	total 1500 stems in dix clean testvoc for <postadv> <ij> 500-word evaluation, WER ~10% trimmed coverage 51%	0/5 fail		did not show up —Firespeaker 02:28, 2 July 2013 (UTC)
2	23 - 29 June	total 2400 stems in dix clean testvoc for <num> <post> trimmed coverage 53%	0/5 fail	stems in dix: 408	did not show up —Firespeaker 02:28, 2 July 2013 (UTC)
3	30 - 6 July	total 3200 stems in dix clean testvoc for <cnjcoo> <cnjadv> <cnjsub> trimmed coverage 55%	2/5 lacking	stems in dix: 508 trimmed coverage: 59.5%,51.5%	trimmed coverage good too narrow a focus on a single corpus number of stems too low no new WER text no testvoc —Firespeaker 20:43, 8 July 2013 (UTC)
4	7 - 13 July	total 4000 stems in dix clean testvoc for <adv> trimmed coverage 59%	2/5 lacking	stems in dix: 2574 trimmed coverage: 69.3%,63.8% azattyq_24455849 WER: 14.78% completed most of TODO-list	good progress on adding stems fixed little things as directed good progress on post-editing process didn't make good progress on reducing WER still no testvoc committed once every 3 or 4 days; should be committing every day poor communication with mentors; needs to be around more often —Firespeaker 22:16, 22 July 2013 (UTC)
5	14 - 20 July	total 4800 stems in dix clean testvoc for <prn> <det> trimmed coverage 63%	3/5 okay
6	21 - 27 July	total 5600 stems in dix clean testvoc for <adj> <adj><advl> trimmed coverage 68%	3/5 okay	stems in dix: 5552 trimmed coverage: 72%,67% azattyq_24455849 WER: 18.01%	good improvement in dix should be checking for errors (e.g., extra spaces) not much progress with WER text simple lrx and t1x should be enough here still no indication of progress with testvoc better communication and commit frequency, but could still improve —Firespeaker 18:21, 1 August 2013 (UTC)
7	28 - 3 August	total 6400 stems in dix trimmed coverage 70%	2/5 lacking
midterm eval 2 August		total 6500 stems in dix 500-word evaluation, WER ~10% trimmed coverage 72%	2/5 lacking		midterm TODO list goals only partially attained overall progress has been mediocre among the lowest-performing students noticeable improvement in the last few weeks needs to improve more to pass the final —Firespeaker 18:26, 1 August 2013 (UTC)
8	4 - 10 August	total 7200 stems in dix clean testvoc for <n> <num><subst> <np> <adj><subst> trimmed coverage 75%	2/5 lacking	stems in dix: 6493 trimmed coverage: 79.6%,74.1%
9	11 - 17 August	total 8000 stems in dix trimmed coverage 78%	2/5 lacking
10	18 - 24 August	total 8800 stems in dix trimmed coverage 81%	3/5 okay
11	25 - 31 August	total 9600 stems in dix clean testvoc for <v> trimmed coverage 83%	3/5 okay	stems in dix: 6730 trimmed coverage: 82.5%,78.4% azattyq_24455849 WER: 6.62%
12	1 - 7 September	total 10400 stems in dix trimmed coverage 85%	3/5 okay	stems in dix: 7007 trimmed coverage: 84.2%,79.8%	Good adjective typology Decent progress on coverage Not around much later in the week Still no testvoc... —Firespeaker 07:29, 10 September 2013 (UTC)
13	8 - 15 September	total 11200 stems in dix trimmed coverage 87%	1/5 bad	stems in dix: 7454 trimmed coverage: 85.2%,80.4%	Decent increase in coverage Still no testvoc Still ~600 unsorted ADJ Not around much —Firespeaker 20:06, 22 September 2013 (UTC)
pencils-down week final evaluation 16 - 23 September		total 12000 stems in dix 500-word evaluation, WER ~10% clean testvoc for all categories trimmed coverage 88% release 0.1.0 and move to trunk		stems in dix: 7546 trimmed coverage: 85.8%,81.6%	Good coverage "Good" WER results But lots of # and * errors :( No work on testvoc Some ADJ sorted; still >500 unsorted only 2 sets of LRX rules since early in GSoC only 1 transfer rule since early in GSoC
Final evaluation					Has improved coverage a certain amount Has not done anything else Mentors have had to nag to get him to work Has not been around enough Among the lowest-performing students Has not improved since midterm Last-ditch efforts not at all impressive

Tips and Tricks

Adding stems quickly

• Add top stems from frequency lists of unknown forms
• Use spectie's dix-entries-to-be-checked script