User:Venkat1997/proposal - Revision history

Venkat1997: /* Coding Phase */

2017-04-03T10:37:34Z

Coding Phase

Venkat1997: /* Coding Phase */

2017-04-03T10:37:00Z

Coding Phase

Venkat1997: /* Which of the published tasks am I interested in? What do I plan to do? */

2017-04-03T10:34:19Z

Which of the published tasks am I interested in? What do I plan to do?

Venkat1997: /* Which of the published tasks am I interested in? What do I plan to do? */

2017-04-03T10:32:08Z

Which of the published tasks am I interested in? What do I plan to do?

Venkat1997: /* Which of the published tasks am I interested in? What do I plan to do? */

2017-04-03T10:31:09Z

Which of the published tasks am I interested in? What do I plan to do?

Venkat1997: /* Which of the published tasks am I interested in? What do I plan to do? */

2017-04-03T10:30:07Z

Which of the published tasks am I interested in? What do I plan to do?

Venkat1997: /* Coding Phase */

2017-04-02T16:08:31Z

Coding Phase

Venkat1997: /* Coding Phase */

2017-04-02T08:25:37Z

Coding Phase

Venkat1997: /* Coding Phase */

2017-04-02T08:25:00Z

Coding Phase

Venkat1997 at 08:24, 2 April 2017

2017-04-02T08:24:10Z

@@ Line 41: / Line 41: @@
 * Finalize what third-party tools are going to used for the task.
 ** Test the performance of using KenLM in place of IRSTLM.
-** Test the performance of using fast_align/Poor Man's Alignment in place of the powerful Giza++.
+** Test the performance of using [https://github.com/clab/fast_align fast_align]/Poor Man's Alignment in place of the powerful Giza++.
-* Write a script that takes care of building the third-party tools (IRSTLM, KenLM, Giza++, Moses).
+* Write a script that takes care of building the third-party tools.
 * If the user has already installed the third party tools, ensure that they are installed properly by checking if the required binaries exist in the user specified directory.
 '''Week-4'''

@@ Line 39: / Line 39: @@
 * Write unit tests to ensure that the validation module is working properly.
 '''Week-3'''
+* Finalize what third-party tools are going to used for the task.
-* Write a script that takes care of building the third-party tools (IRSTLM, Giza++, Moses).
+** Test the performance of using KenLM in place of IRSTLM.
+** Test the performance of using fast_align/Poor Man's Alignment in place of the powerful Giza++.
+* Write a script that takes care of building the third-party tools (IRSTLM, KenLM, Giza++, Moses).
 * If the user has already installed the third party tools, ensure that they are installed properly by checking if the required binaries exist in the user specified directory.
 '''Week-4'''

@@ Line 14: / Line 14: @@
 == Which of the published tasks am I interested in? What do I plan to do? ==
-I am interested in the task User-friendly lexical selection training. I plan to extend [https://github.com/tiefling-cat/apertium-flst/ Nikita Medyankin's work] on the driver script. The config file he uses could be made more user-friendly and his work does not have an installer script currently and some scripts that he uses could be factored out. Also, some hard-to-install dependencies can be replaced. Hence, what I plan to do is refactor his code and remove unnecessary scripts, adopt a more user friendly yaml config file, make the installation of third-party tools easier (maybe even removing some of them that are currently used), provide regression tests to this driver script and finally test the work by running on some language pairs that don't have many rules and add those rules to the pair if they improve quality.
+I am interested in the task User-friendly lexical selection training. I plan to extend [https://github.com/tiefling-cat/apertium-flst/ Nikita Medyankin's work] on the driver script. The config file he uses could be made more user-friendly. His work also does not have an installer script that takes care of installing third-party tools currently and some scripts that he uses could be factored out. Some hard-to-install dependencies that are currently used can also be replaced. Hence, what I plan to do is refactor his code and remove unnecessary scripts, adopt a more user friendly yaml config file, make the installation of third-party tools easier (maybe even removing some of them that are currently used), provide regression tests to this driver script and finally test the work by running on some language pairs that don't have many rules and add those rules to the pair if they improve quality.
 == Reasons why Google and Apertium should sponsor it ==

@@ Line 14: / Line 14: @@
 == Which of the published tasks am I interested in? What do I plan to do? ==
-I am interested in the task User-friendly lexical selection training. I plan to extend [https://github.com/tiefling-cat/apertium-flst/ Nikita Medyankin's work] on the driver script. The config file he uses could be made more user-friendly, his work does not have an installer script currently and some scripts that he uses could be factored out. Also, some hard-to-install dependencies can be replaced. Hence, what I plan to do is refactor his code and remove unnecessary scripts, adopt a more user friendly yaml config file, make the installation of third-party tools easier (maybe even removing some of them that are currently used), provide regression tests to this driver script and finally test the work by running on some language pairs that don't have many rules and add those rules to the pair if they improve quality.
+I am interested in the task User-friendly lexical selection training. I plan to extend [https://github.com/tiefling-cat/apertium-flst/ Nikita Medyankin's work] on the driver script. The config file he uses could be made more user-friendly and his work does not have an installer script currently and some scripts that he uses could be factored out. Also, some hard-to-install dependencies can be replaced. Hence, what I plan to do is refactor his code and remove unnecessary scripts, adopt a more user friendly yaml config file, make the installation of third-party tools easier (maybe even removing some of them that are currently used), provide regression tests to this driver script and finally test the work by running on some language pairs that don't have many rules and add those rules to the pair if they improve quality.
 == Reasons why Google and Apertium should sponsor it ==

@@ Line 14: / Line 14: @@
 == Which of the published tasks am I interested in? What do I plan to do? ==
-I am interested in the task User-friendly lexical selection training. I plan to extend [https://github.com/tiefling-cat/apertium-flst/ Nikita Medyankin's work] on the driver script. His config file could be made more user-friendly, it does not have an installer script currently and some scripts that he uses could be factored out. Also, some hard-to-install dependencies can also be replaced. Hence, what I plan to do is refactor his code and remove unnecessary scripts, adopt a more user friendly yaml config file, make the installation of third-party tools easier (maybe even removing some of them that are currently used), provide regression tests to this driver script and finally test the work by running on some language pairs that don't have many rules and add those rules to the pair if they improve quality.
+I am interested in the task User-friendly lexical selection training. I plan to extend [https://github.com/tiefling-cat/apertium-flst/ Nikita Medyankin's work] on the driver script. The config file he uses could be made more user-friendly, his work does not have an installer script currently and some scripts that he uses could be factored out. Also, some hard-to-install dependencies can be replaced. Hence, what I plan to do is refactor his code and remove unnecessary scripts, adopt a more user friendly yaml config file, make the installation of third-party tools easier (maybe even removing some of them that are currently used), provide regression tests to this driver script and finally test the work by running on some language pairs that don't have many rules and add those rules to the pair if they improve quality.
 == Reasons why Google and Apertium should sponsor it ==

@@ Line 55: / Line 55: @@
 '''Week-7 & 8'''
 * Repeat the tasks done in the past two weeks on Non-Parallel Corpora.
-'''Week-8'''
 * Complete the module on testing the .lrx file on the held-out test corpus. Module must take care of editing the pipeline in the modes.xml file of the language pair and then using apertium-eval-translator after that to check the quality of translation.
 '''Week-9'''

@@ Line 48: / Line 48: @@
 * Complete the following steps in the script (Parallel corpora):
 ** Preprocess corpora.
-** Split the corpus provided by the user into training and test corpus. (Include a parameter for the ratio of training to test corpus which the user can modify. Set the ratio default to 80% training and 20% Test)  Run training on the training part of the corpus.
+** Split the corpus provided by the language pair developer into training and test corpus. (Include a parameter for the ratio of training to test corpus which the language pair developer can modify. Set the ratio default to 80% training and 20% Test)  Run training on the training part of the corpus.
 '''Week-6'''
 * Complete the following steps in the script (Parallel corpora):

@@ Line 43: / Line 43: @@
 '''Week-4'''
 * Make the process more user-friendly by checking and fixing common issues automatically. (Reference: http://wiki.apertium.org/wiki/Installation_troubleshooting)
-* Print helpful messages to user which aid the user in fixing the issue when the script fails.
+* Print helpful messages to the language pair developer which aids them in fixing the issue when the script fails.
 * Test the deliverable up till now on different operating systems, versions of GCC, version of Python etc. to check compatibility.
 '''Week-5'''
@@ Line 65: / Line 65: @@
 '''Week-12'''
 * Check whether the rules acquired in the above step improve quality and add them to the existing language pairs if they do. (Perform this step in collaboration with language pair maintainers.)
 == Skills & Qualifications ==
 I am a B.Tech Computer Science & Engineering student at IIIT Hyderabad. I am proficient in many languages including but not limited to:

@@ Line 22: / Line 22: @@
 == How and who will it benefit in society? ==
-* Pair maintainers as they will have an easier way of adding rules to the pair.
+* Pair maintainers as they will have an easier way of generating a corpus based lrx and adding rules from it.
-* Users who want to start a new language pair.
+* Language pair developers who want to start a new language pair.
 == Plan ==
@@ Line 29: / Line 29: @@
 * Understand the current workflow for extracting lexical selection rules.
 * Understand existing code of Nikita Medyankin.
+* Pick a language pair to use for testing the driver script.
 * Try making a new language pair by myself to understand how Apertium works more clearly.
 === Coding Phase ===
 '''Week-1'''
+* Begin work on the driver script.
 * Devise a config format that incorporates all the different options possible for lexical selection training.
 '''Week-2'''
+* Complete the module on validating config file.
-* Begin work on the driver script.
-** Complete the module on validating config file.
 * Write unit tests to ensure that the validation module is working properly.
 '''Week-3'''
@@ Line 55: / Line 56: @@
 * Repeat the tasks done in the past two weeks on Non-Parallel Corpora.
 '''Week-8'''
-* Test the .lrx file on the held-out test corpus by editing the pipeline in the modes.xml file of the language pair and using apertium-eval-translator after that to check the quality of translation.
+* Complete the module on testing the .lrx file on the held-out test corpus. Module must take care of editing the pipeline in the modes.xml file of the language pair and then using apertium-eval-translator after that to check the quality of translation.
 '''Week-9'''
 * Won't be available as I am going on a vacation.