User:Gang Chen/GSoC 2013 Progress - Revision history

Gang Chen: /* 3.3 Graph */

2013-10-07T08:43:26Z

3.3 Graph

Gang Chen: /* 3.2 System-precision */

2013-10-07T08:42:45Z

3.2 System-precision

Gang Chen: /* 4.2.2 Catalan */

2013-10-06T14:14:57Z

4.2.2 Catalan

Gang Chen: /* 4.2 With or without CG */

2013-10-06T04:35:26Z

4.2 With or without CG

Gang Chen: /* 4.2 With or without CG */

2013-10-06T03:57:14Z

4.2 With or without CG

Gang Chen: /* 4.2 With or without CG */

2013-09-26T06:10:48Z

4.2 With or without CG

Gang Chen: /* 4. Rules and CG */

2013-09-26T06:05:47Z

4. Rules and CG

Gang Chen: /* 4. With or Without Rules and CG */

2013-09-26T05:06:29Z

4. With or Without Rules and CG

Gang Chen: /* 4.1 data */

2013-09-26T05:04:11Z

4.1 data

Gang Chen: /* GSOC 2013 */

2013-09-25T07:02:00Z

GSOC 2013

@@ Line 67: / Line 67: @@
 |}
-===== 3.3 Graph =====
+===== 3.2 System Precision =====
 [[File:LSW_tagger_lines_vs_system-precision_Spanish.png]]
 [[File:LSW_tagger_lines_vs_system-precision_Catalan.png]]

@@ Line 66: / Line 66: @@
 | 1000+ lines, 80% automatically mapped from [http://www.coli.uni-saarland.de/~thorsten/tnt/ the TnT tagger], 20% by hand-tagging
 |}
-===== 3.2 System-precision =====
-<pre>
-Spanish Euroarl Lines	HMM_Bigram	LSW_L1R1	LSW_L1	LSW_R1	LSW_L12	LSW_R12	LSW_L12R1	LSW_L1R12
-			0.8171		0.8486		0.8506	0.8496	0.8485	0.8484	0.8484		0.8484
-			0.7876		0.8508		0.8479	0.8529	0.8494	0.8504	0.8484		0.8487
-			0.8289		0.8553		0.8252	0.8517	0.8394	0.8484	0.8484		0.85
-			0.8582		0.8574		0.8153	0.8549	0.8331	0.849	0.8487		0.8515
-			0.868		0.8593		0.8134	0.8573	0.8302	0.8515	0.8494		0.8542
-			0.8756		0.863		0.8182	0.8578	0.8245	0.8527	0.8494		0.856
-			0.8818		0.8654		0.81	0.8547	0.8237	0.8549	0.8496		0.8605
-			0.8852		0.8662		0.8084	0.8546	0.8194	0.8545	0.8503		0.8617
-			0.886		0.8665		0.8083	0.8548	0.8199	0.8552	0.8512		0.8637
-			0.886		0.8664		0.8087	0.8549	0.8215	0.8552	0.8522		0.8648
-			0.8861		0.8664		0.8089	0.8553	0.8212	0.8552	0.852		0.8657
-													0.8659
-1000000													0.8667
-</pre>
-<pre>
-Catalan Wikipedia Lines	HMM_Bigram	LSW_L1R1	LSW_L1	LSW_R1	LSW_L12	LSW_R12	LSW_L12R1	LSW_L1R12
-			0.8619		0.8907		0.8907	0.8907	0.8907	0.8907	0.8907		0.8907
-			0.8513		0.8969		0.8812	0.8949	0.889	0.8939	0.8907		0.8912
-			0.8674		0.905		0.8256	0.9002	0.8606	0.8975	0.8921		0.8944
-			0.8856		0.9086		0.8048	0.9022	0.8386	0.8999	0.8911		0.8966
-			0.9041		0.9124		0.8018	0.9053	0.8212	0.9033	0.8925		0.9038
-			0.9143		0.9147		0.7989	0.906	0.8123	0.9034	0.8923		0.9085
-			0.9189		0.9202		0.7985	0.9081	0.8067	0.9071	0.8926		0.9153
-			0.9274		0.9232		0.7986	0.908	0.8041	0.9105	0.8916		0.9202
-			0.9277		0.9249		0.7974	0.9065	0.8034	0.9092	0.8914		0.9217
-			0.9285		0.926		0.7975	0.9072	0.8016	0.9094	0.8921		0.9239
-			0.9295		0.9254		0.7969	0.9074	0.8015	0.9084	0.8931		0.9248
-</pre>
-<pre>
-English Europarl Lines	HMM_Bigram	LSW_L1R1	LSW_L1	LSW_R1	LSW_L12	LSW_R12	LSW_L12R1	LSW_L1R12
-			0.8258		0.8268		0.8234	0.8256	0.8263	0.8266	0.8277		0.8271
-			0.7991		0.8312		0.8329	0.8182	0.8311	0.8244	0.8274		0.8279
-			0.8294		0.852		0.8653	0.8342	0.8441	0.8343	0.829		0.8344
-			0.8494		0.8622		0.8659	0.8475	0.8516	0.8433	0.831		0.8415
-			0.8771		0.8742		0.8679	0.8449	0.8628	0.8498	0.8319		0.8586
-			0.8886		0.8807		0.8741	0.847	0.8732	0.8515	0.8325		0.8717
-			0.8948		0.8843		0.8711	0.8486	0.8753	0.8524	0.8325		0.8783
-			0.8942		0.8867		0.8708	0.8499	0.8781	0.8524	0.8328		0.8852
-			0.8934		0.8874		0.8705	0.8499	0.8774	0.853	0.8323		0.8872
-			0.8929		0.8886		0.8719	0.8497	0.8787	0.8539	0.8323		0.8888
-			0.8945		0.8887		0.8736	0.8496	0.8787	0.854	0.8326		0.8902
-													0.8906
-1000000													0.891
-</pre>
 ===== 3.3 Graph =====

@@ Line 142: / Line 142: @@
 [[File:CA-HMM-CG.png]]
 [[File:CA-LSW-CG.png]]
+====== 4.2.3 English ======
+[[File:EN-HMM-CG.png]]
+[[File:EN-LSW-CG.png]]
 === Week Plan and Progress===

@@ Line 130: / Line 130: @@
 ===== 4.2 With or without CG =====
+* '''cgTrain''' means using CG for processing the training corpus (to reduce ambiguity).
+* '''cgTag''' means using CG before the PoS tagger.
+====== 4.2.1 Spanish ======
 [[File:ES-HMM-CG.png]]
 [[File:ES-LSW-CG.png]]
+====== 4.2.2 Catalan ======
 [[File:CA-HMM-CG.png]]
 [[File:CA-LSW-CG.png]]
-* Note:
-'''cgTrain''' means using CG for processing the training corpus (to reduce ambiguity).
-'''cgTag''' means using CG before the PoS tagger.
 === Week Plan and Progress===

@@ Line 130: / Line 130: @@
 ===== 4.2 With or without CG =====
-[[File:Spanish_HMM_CG.png]]
+[[File:ES-HMM-CG.png]]
-[[File:Spanish_LSW_CG.png]]
+[[File:ES-LSW-CG.png]]
-[[File:Catalan_HMM_CG.png]]
+[[File:CA-HMM-CG.png]]
-[[File:Catalan_LSW_CG.png]]
+[[File:CA-LSW-CG.png]]
+* Note:
-* Note: ''HMM_CG-Train'' means only use CG in the training step (to process training data), but not in the tagging step.
-* Note2: in the graph “Catalan HMM-Bigram with CG”, the two lines for HMM-Bigram and HMM-Bigram-CG-TrainOnly are too close to clarify.
+'''cgTrain''' means using CG for processing the training corpus (to reduce ambiguity).
+'''cgTag''' means using CG before the PoS tagger.
 === Week Plan and Progress===

@@ Line 125: / Line 125: @@
 ===== 4.1 With or without Rules =====
+[[File:Spanish_SW_LSW_LSW-noRules.png]]
-[[]]
+[[File:Catalan_SW_LSW_LSW-noRules.png]]
-[[]]
+[[File:English_SW_LSW_LSW-noRules.png]]
-[[]]
 ===== 4.2 With or without CG =====
-[[File:Es_HMM_Rules_and_CG.png]]
+[[File:Spanish_HMM_CG.png]]
-[[File:Es_LSW_Rules_and_CG.png]]
+[[File:Spanish_LSW_CG.png]]
-[[File:Ca_HMM_Rules_and_CG.png]]
+[[File:Catalan_HMM_CG.png]]
-[[File:Ca_LSW_Rules_and_CG.png]]
+[[File:Catalan_LSW_CG.png]]
 * Note: ''HMM_CG-Train'' means only use CG in the training step (to process training data), but not in the tagging step.

@@ Line 123: / Line 123: @@
 ==== 4. With or Without Rules and CG ====
-===== 4.1 data =====
-'''es: HMM & LSW'''
-<pre>
-Lines	HMM	HMM_No-Rules	HMM_CG	HMM_CG-Train	|	LSW(-1, +1)	LSW(-1, +1)_No-Rules	LSW(-1, +1)_CG	LSW(-1, +1)_CG-Train
-	0.8194	0.7462		0.8473	0.8173		|	0.8508		0.8486			0.8458		0.8486
-	0.8311	0.8243		0.8526	0.8425		|	0.8576		0.8507			0.8496		0.856
-	0.8605	0.8342		0.8587	0.8548		|	0.8596		0.8528			0.8523		0.8589
-	0.8703	0.8525		0.8631	0.8634		|	0.8615		0.8534			0.8549		0.8623
-	0.8778	0.8559		0.8659	0.871		|	0.8652		0.8562			0.8587		0.8671
-	0.8841	0.8568		0.8687	0.8753		|	0.8677		0.8585			0.8608		0.8702
-	0.8875	0.8606		0.8711	0.879		|	0.8685		0.8589			0.862		0.8717
-	0.8883	0.8612		0.8712	0.8786		|	0.8688		0.8592			0.862		0.8721
-	0.8883	0.862		0.8712	0.8783		|	0.8686		0.8589			0.8622		0.8728
-	0.8884	0.8623		0.8712	0.8779		|	0.8687		0.859			0.8622		0.8728
-</pre>
-'''ca: HMM & LSW'''
-<pre>
-Lines	HMM	HMM_No-Rules	HMM_CG	HMM_CG-Train	|	LSW(-1, +1)	LSW(-1, +1)_No-Rules	LSW(-1, +1)_CG	LSW(-1, +1)_CG-Train
-	0.8237	0.7954		0.863	0.8234		|	0.8521		0.8528			0.8492		0.8528
-	0.8301	0.823		0.8519	0.832		|	0.8665		0.8345			0.8642		0.8675
-	0.8484	0.8303		0.8694	0.8568		|	0.8701		0.8245			0.8685		0.872
-	0.8656	0.8421		0.8775	0.8689		|	0.8739		0.8362			0.8743		0.8778
-	0.8757	0.8567		0.8808	0.8741		|	0.8763		0.8327			0.8762		0.8806
-	0.8804	0.8642		0.8811	0.8753		|	0.8816		0.8312			0.8808		0.8855
-	0.8887	0.8747		0.8868	0.8812		|	0.8847		0.823			0.8832		0.8885
-	0.8892	0.8752		0.8869	0.8825		|	0.8863		0.8373			0.885		0.8903
-	0.8897	0.8771		0.8887	0.8819		|	0.8873		0.8422			0.8855		0.8908
-	0.8909	0.8785		0.8893	0.8827		|	0.8866		0.8422			0.8853		0.8906
-</pre>
 * Note: ''HMM_CG-Train'' means only use CG in the training step (to process training data), but not in the tagging step.

@@ Line 1: / Line 1: @@
 == GSOC 2013 ==
-I'm working with Apertium for the GSoC 2013, on the project "Sliding Window Part of Speech Tagger for Apertium".
+I was working with Apertium for the GSoC 2013 (2013.06-2013.09), on the project "Sliding Window Part of Speech Tagger for Apertium".
 my proposal is here: [http://wiki.apertium.org/wiki/User:Gang_Chen/GSoC_2013_Application:_%22Sliding_Window_PoS_Tagger%22 Proposal]