User:Eiji - Revision history

Eiji at 11:48, 20 March 2023

2023-03-20T11:48:17Z

Eiji at 22:11, 19 March 2023

2023-03-19T22:11:16Z

Eiji at 12:19, 19 March 2023

2023-03-19T12:19:35Z

Eiji at 23:22, 18 March 2023

2023-03-18T23:22:40Z

Eiji at 18:52, 18 March 2023

2023-03-18T18:52:06Z

Eiji at 18:39, 18 March 2023

2023-03-18T18:39:37Z

Eiji at 18:38, 18 March 2023

2023-03-18T18:38:25Z

Eiji at 18:33, 18 March 2023

2023-03-18T18:33:19Z

Eiji at 00:05, 18 March 2023

2023-03-18T00:05:59Z

Eiji at 19:24, 17 March 2023

2023-03-17T19:24:31Z

@@ Line 33: / Line 33: @@
 May 29 - June 4
 |Investigating word segmentation and tokenization from paper, and summarising useful findings into a report
-|Looking up NLP conference papers on word segmentation or tokenization, and checking popular Japanese tokenizers such as MeCab and Juman. Summarising the findings in two A4 papers.
+|Looking up NLP conference papers on word segmentation or tokenization, and checking popular Japanese tokenizers such as MeCab and Juman. Summarising the findings in three A4 papers.
 |-
 |-

@@ Line 101: / Line 101: @@
 Aug7 - Aug13
 |Testing and fixing bugs
-|Testing accuracy and speed with texts and refactoring the code. Modifying Japanese related files too.
+|Testing accuracy and speed with texts and refactoring the code. Modifying Japanese-related documents too.
 |-
 |-

@@ Line 39: / Line 39: @@
 June 5 - June 11
 |Testing possible algorithms for tokenization and becoming aware of the pros and cons of them
 |N-gram, Maximal matching, Viterbi, SentensePiece, etc... Discussing the algorithm with mentors.
 |-
 |-
@@ Line 45: / Line 45: @@
 June 12 - June 18
 |Analyzing drawbacks of current tokenization for apertium-iii and apertium-jpn
-|Drawing up a report and thinking of what the hybrid model needs to cover. Making a pseudocode for the model.
+|Drawing up a report and thinking of what the hybrid model needs to cover. Making a pseudocode for the model. Getting feedback from mentors about the pseudocode.
 |-
 |-
@@ Line 51: / Line 51: @@
 June 19 - June 25
 |Producing a hybrid model
-|Trying to minimize the drawbacks of the current tokenizer and make the model more efficiently. Coding the model according to the pseudocode in python.
+|Trying to minimize the drawbacks of the current tokenizer and make the model more efficient. Coding the model according to the pseudocode in python. Getting feedback from mentors about the code.
 |-
@@ Line 87: / Line 87: @@
 July 24 - July 30
 |Improving twol file and lexicon file
-|Adding more kanji, hiragana, and katakana in it and organizing the file for better use. Including roman numeric characters. Evaluation of coverage improvement.
+|Adding more kanji, hiragana, and katakana in it and organizing the file for better use. Including roman numeric characters. Evaluation of coverage improvement. Sending pull request on jpn repository.
 |-
@@ Line 94: / Line 94: @@
 July 31 - Aug 6
 |Improving disambiguation for Japanese
-|Adding grammar to lexical selection, determiner, and remove reading noun. Evaluation of Coverage improvement.
+|Adding grammar to lexical selection, determiner, and remove reading noun. Evaluation of Coverage improvement. Sending pull request on jpn repository.
 |-
@@ Line 101: / Line 101: @@
 Aug7 - Aug13
 |Testing and fixing bugs
-|Testing accuracy and speed with texts and refactoring the code.
+|Testing accuracy and speed with texts and refactoring the code. Modifying Japanese related files too.
 |-
 |-

@@ Line 2: / Line 2: @@
     Name:Eiji Miyamoto
     E-mail address:motopon57@gmail.com
-    University: University of Manchester
+    University: University of Manchester(BSc.Computer Science)
     IRC:thelounge72
     github:https://github.com/yypy22

@@ Line 33: / Line 33: @@
 May 29 - June 4
 |Investigating word segmentation and tokenization from paper, and summarising useful findings into a report
-|Looking up NLP conference papers on word segmentation or tokenization, and checking popular Japanese tokenizers such as MeCab and Juman.
+|Looking up NLP conference papers on word segmentation or tokenization, and checking popular Japanese tokenizers such as MeCab and Juman. Summarising the findings in two A4 papers.
 |-
 |-
@@ Line 39: / Line 39: @@
 June 5 - June 11
 |Testing possible algorithms for tokenization and becoming aware of the pros and cons of them
 |N-gram, Maximal matching, Viterbi, SentensePiece, etc... Discussing the algorithm with mentors.
 |-
 |-

@@ Line 45: / Line 45: @@
 June 12 - June 18
 |Analyzing drawbacks of current tokenization for apertium-iii and apertium-jpn
-|Drawing up a report and thinking of what hybrid model needs to cover.
+|Drawing up a report and thinking of what the hybrid model needs to cover.
 |-
 |-
@@ Line 58: / Line 58: @@
 June 26 - July 2
 |Analyzing the model and improving it
-|Checking efficiency of memory usage and speed of the model. If it has room for improvement, improve it.
+|Checking efficiency of memory usage and speed of the model. If it has room for improvement, improve it. Testing speed and accuracy with texts, comparing manually tokenized sentences and the one tokenized with the hybrid model.
 |-
@@ Line 64: / Line 64: @@
 | Week 6:
 July 3 - July 9
-|Testing the hybrid model
+|Converting the model into a faster language
+|Python model to C++ or faster language.
-|Testing speed and accuracy with texts, comparing manually tokenized sentences and the one tokenized with the hybrid model.
 |-
 |-
@@ Line 73: / Line 73: @@
 | Week 7:
 July 10 - July 16
+|Continue to work on converting
-|Converting the model into a faster language
+|Fixing bugs and checking logic and function. Testing the model with simple sentences.
-|Python model to C++ or faster language.
 |-
 |-

@@ Line 115: / Line 115: @@
 ==Coding Challenge==
 Integration apertium-iii transducer into apertium-jpn.  https://github.com/yypy22/apertium-jpn
+I modified MakeFile.am and modes.xml to use tokenized.py from apertium-iii. I added tokenizer-related code such as $(LANG1).autotok.hfst into makefile and program tag in modes.xml, dropping tokenized.py file into jpn repository.
 ==Skills==
 Python, Java, C++(intermediate), JavaScript(intermediate), HTML, CSS, Docker, Django, XML, Git, jQuery