User:Weizhe/GSOC 2020 proposal - Revision history

Weizhe at 02:22, 12 April 2020

2020-04-12T02:22:35Z

Weizhe: /* Self Introduction */

2020-04-12T02:20:21Z

Self Introduction

Weizhe at 07:44, 27 March 2020

2020-03-27T07:44:20Z

Show changes

Weizhe: Blanked the page

2020-03-26T08:53:07Z

Blanked the page

Show changes

Weizhe: /* Abstract */

2020-03-26T08:51:11Z

Abstract

Weizhe: /* Proposed Plan */

2020-03-25T13:49:15Z

Proposed Plan

Weizhe: /* Abstract */

2020-03-25T13:24:03Z

Abstract

ScoopGracie: ScoopGracie moved page Weizhe/GSOC 2020 proposal to User:Weizhe/GSOC 2020 proposal: Move to userspace

2020-03-24T19:10:39Z

ScoopGracie moved page Weizhe/GSOC 2020 proposal to User:Weizhe/GSOC 2020 proposal: Move to userspace

Weizhe: /* Proposed Plan */

2020-03-24T14:38:02Z

Proposed Plan

Weizhe: /* Proposed Plan */

2020-03-24T14:34:48Z

Proposed Plan

← Older revision		Revision as of 02:22, 12 April 2020
Line 37:		Line 37:


	I found the project I was interested in: "Robust tokenization in lttoolbox". I started this project a month ago and completed the Coding Challenge. The source code and documents for the Coding Challenge are kept in my Repository. At the same time I also configured the Apertium kernel and lttoolbox environment on my PC, carried out some operations, and had a general understanding of their works. For now I can understand the target project better and deeper so that lexical analysis and tokenization could be handled better.		I found the project I was interested in: '''"Robust tokenization in lttoolbox"'''[http://wiki.apertium.org/wiki/Ideas_for_Google_Summer_of_Code/Robust_tokenisation]. I started this project a month ago and completed the '''Coding Challenge'''. The source code and documents for the Coding Challenge are kept in my '''Repository'''[https://github.com/GavinWz/Apertium]. At the same time I also configured the Apertium kernel and lttoolbox environment on my PC, carried out some operations, and had a general understanding of their works. For now I can understand the target project better and deeper so that lexical analysis and tokenization could be handled better.

@@ Line 23: / Line 23: @@
-In my first year in university, I joined the high-performance computing center of the university as a research assistant. Through research and learning during the period, I have a deep understanding of software architecture and open source projects. In March 2019, I participated in the "LanQiao" national programming competition - a programming skill and algorithm contest - and won the second prize.
+In my first year in university, I joined the '''high-performance computing center'''[http://cs.wfu.edu.cn/2014/0603/c1227a33048/page.htm] of the university as a research assistant. Through research and learning during the period, I have a deep understanding of software architecture and open source projects. In March 2019, I participated in the "LanQiao" national programming competition - a programming skill and algorithm contest - and won the second prize.
 I am committed to improving my skills through project practice, so I think I will learn more professional knowledge through this exercise in Apertium.
 == The reason of why I am interested in Apertium ==

@@ Line 51: / Line 51: @@
 My task will be to write algorithms so that the character reading and recognition process can proceed smoothly, regardless of the character type. And after reading, the input text can be separated into some lexemes by a given class of delimiters. Then, a new tokenization operation is carried out to assign the matching token to the obtained lexemes.
-The tokenization process should be divided into four separate processes: read, delimiter handling, identify, and tokenization. For each process, the algorithm should be designed or modified separately from the original code. Before the algorithm design of each process, there will be a ready-made test case, the algorithm design goal is to pass the test. After passing the basic test, it is necessary to propose an in-depth test case, and then update the algorithm to get the latest test results. Through this loop, the program will gradually become more robust and satisfactory. Eventually, the test documentation and source code for each unit will be generated and progressively enhanced. After all the unit designs are completed, splicing of the units may be necessary, and before that there should be a basic test case. After the same iterative approach, the splicing process will be completed quickly. The new tokenization design will then be completed successfully.
+The tokenization process should be divided into four separate processes: read, delimiter handling, recognition, and tokenization. For each process, the algorithm should be designed or modified separately from the original code. Before the algorithm design of each process, there will be a ready-made test case, the algorithm design goal is to pass the test. After passing the basic test, it is necessary to propose an in-depth test case, and then update the algorithm to get the latest test results. Through this loop, the program will gradually become more robust and satisfactory. Eventually, the test documentation and source code for each unit will be generated and progressively enhanced. After all the unit designs are completed, splicing of the units may be necessary, and before that there should be a basic test case. After the same iterative approach, the splicing process will be completed quickly. The new tokenization design will then be completed successfully.
 === Benefits to society(Deliverable) ===

@@ Line 129: / Line 129: @@
 * Discuss with the mentors how to improve the compatibility of lttoolbox with Unicode by combining the ICU with the original code
-* Divide tokenization process into four separate processes: reading, partition, identify, and tokenization
+* Divide tokenization process into four separate processes: reading, identify, delimiter handling, and tokenization
@@ Line 135: / Line 135: @@
-* Following the TDD pattern, complete the reading and partition processes
+* Following the TDD pattern, complete the reading and identify processes
@@ Line 141: / Line 141: @@
-* Complete the identify processes
+* Complete the delimiter handling processes

@@ Line 51: / Line 51: @@
 My task will be to write algorithms so that the character reading and recognition process can proceed smoothly, regardless of the character type. And after reading, the input text can be separated into some lexemes by a given class of delimiters. Then, a new tokenization operation is carried out to assign the matching token to the obtained lexemes.
-The tokenization process should be divided into four separate processes: read, partition, identify, and tag. For each process, the algorithm should be designed or modified separately from the original code. Before the algorithm design of each process, there will be a ready-made test case, the algorithm design goal is to pass the test. After passing the basic test, it is necessary to propose an in-depth test case, and then update the algorithm to get the latest test results. Through this loop, the program will gradually become more robust and satisfactory. Eventually, the test documentation and source code for each unit will be generated and progressively enhanced. After all the unit designs are completed, splicing of the units may be necessary, and before that there should be a basic test case. After the same iterative approach, the splicing process will be completed quickly. The new tokenization design will then be completed successfully.
+The tokenization process should be divided into four separate processes: read, delimiter handling, identify, and tokenization. For each process, the algorithm should be designed or modified separately from the original code. Before the algorithm design of each process, there will be a ready-made test case, the algorithm design goal is to pass the test. After passing the basic test, it is necessary to propose an in-depth test case, and then update the algorithm to get the latest test results. Through this loop, the program will gradually become more robust and satisfactory. Eventually, the test documentation and source code for each unit will be generated and progressively enhanced. After all the unit designs are completed, splicing of the units may be necessary, and before that there should be a basic test case. After the same iterative approach, the splicing process will be completed quickly. The new tokenization design will then be completed successfully.
 === Benefits to society(Deliverable) ===

← Older revision	Revision as of 19:10, 24 March 2020
(No difference)

@@ Line 143: / Line 143: @@
-* Complete the reading and identify processes
+* Complete the identify processes
@@ Line 149: / Line 149: @@
-* Complete the reading and tokenization processes
+* Complete the tokenization processes
 * First Evaluation

@@ Line 120: / Line 120: @@
 * Start with the FSTProcessor::analysis() function and read the lttoolbox source code again
 * Understand how lttoolbox implements tokenization
-* Deliverable: Familiar with the way apertium works and the use of lttoolbox
+Deliverable: Familiar with the way apertium works and the use of lttoolbox
@@ Line 150: / Line 151: @@
 * Complete the reading and tokenization processes
 * First Evaluation
-* Deliverable: The algorithms to realize these four functions are obtained
+Deliverable: The algorithms to realize these four functions are obtained
@@ Line 175: / Line 177: @@
 * Optimize the code
 * Second Evaluation
-* Deliverable: Unit splicing is completed and project functions are all realized
+Deliverable: Unit splicing is completed and project functions are all realized
@@ Line 193: / Line 196: @@
 * Complete all the unfinished tasks above
 * Delivery of the project
-* Deliverable:  Complete the update of the compiler and documentation
+Deliverable:  Complete the update of the compiler and documentation
 === Time Commitment ===