User:Ksnmi/Application - Revision history

Ksnmi: /* WorkPlan */

2014-03-21T09:19:21Z

WorkPlan

Ksnmi at 00:00, 21 March 2014

2014-03-21T00:00:33Z

Ksnmi at 11:29, 20 March 2014

2014-03-20T11:29:59Z

Ksnmi at 11:08, 20 March 2014

2014-03-20T11:08:37Z

Ksnmi at 10:10, 20 March 2014

2014-03-20T10:10:11Z

Ksnmi: /* Literature Review */

2014-03-20T09:13:06Z

Literature Review

Ksnmi: /* Introduction */

2014-03-20T09:09:16Z

Introduction

Ksnmi at 08:40, 20 March 2014

2014-03-20T08:40:10Z

Ksnmi at 08:23, 20 March 2014

2014-03-20T08:23:10Z

Ksnmi: /* Introduction */

2014-03-20T08:02:35Z

Introduction

@@ Line 323: / Line 323: @@
 *Repeat Steps from Week1 for these languages
 *Improve and add non-standard->standard features to the pipeline as we go forward
+|-
+!colspan="2" style="text-align: right"|Deliverable #1<br />15 June
+|
+Compile the Weekly codes for 7 languages and Produce the deliverable.
 |-
 ! 5 !! 15 - 21 June
@@ Line 350: / Line 354: @@
 *adding main monolingual languages worked on ( English,Spanish,Irish )
 *The one's which were worked on earlier ( in the initial phase )Improvement on these to be made and verified.
+|-
+!colspan="2" style="text-align: right"|Deliverable #2<br />19 July
+|
+Compile the Weekly codes for about 7 languages and Produce the deliverable.
 |-
 ! 9 !! 20 - 26 July
@@ Line 364: / Line 372: @@
 Working on binaries to be implemented efficiently on Apertium
 |-
-!colspan="2" style="text-align: right"|pencils-down week<br />final evaluation<br />11 August - 18 August
+! 12 !! 11 August - 18 August
+|
+Integrating and testing support for the set of deliverables, along with information from last week.
+|-
+!colspan="2" style="text-align: right"|pencils-down week<br />final evaluation<br />18 August - 22 August
 |
 *Continuing work from last week
-*Making Documentation and other Deliverables
+*Making Documentation and other reqd. deliverables.
 |}
 ==References==

@@ Line 24: / Line 24: @@
 **I have been associated with various kinds of projects, Some important one's are listed below -<br/>
 **I'm an active member of the Translation Process Research Community from Europe and under the guidance of Michael Carl, Copenhagen Business school and Srinivas Bangalore from AT&T, USA. I have completed a project which models translators as expert or novice, based on their eye movements tracked while they are performing a translation Task, last summer at CBS. In Proceedings for ETRA 2014, Florida (Recognition of translator expertise using sequences of fixations and keystrokes, Pascual Martinez-Gomez, Akshay Minocha, Jin Huang, Michael Carl, Srinivas Bangalore, Akiko Aizawa )
-**I had been associated to SketchEngine as a Programmer for almost a year. - My work during the inital phase there resulted in a publication at ACL SIGWAC 2013, which was about building Quality Web Corpus from Feeds Information.
+**I had been associated to SketchEngine as a Programmer for almost a year. - My work during the initial phase there resulted in a publication at ACL SIGWAC 2013, which was about building Quality Web Corpus from Feeds Information.
-**Was an initial contributor to the Health triangulation System, under the guidance of Dr. Jennifer Mankoff
+**Was an initial contributor to the Health triangulation System, under the guidance of Dr. Jennifer Mankoff, HCI, Carnegie Mellon University.
 **Have worked vigorously on Corpus Linguistics and Data Mining tasks. to support other contributions.
 **A detailed CV can be found at [[http://web.iiit.ac.in/~akshay.minocha/Akshay_Minocha_CV.pdf This Link]]
@@ Line 31: / Line 31: @@
 *'''List any non-Summer-of-Code plans you have for the Summer, especially employment, if you are applying for internships, and class-taking. Be specific about schedules and time commitments. we would like to be sure you have at least 30 free hours a week to develop for our project.'''
 **I am dedicated towards this project and am very excited about working on it. A commitment of 30 hours a week would not be a problem, I might even work on some more so as to complete the weekly goals as described in the Workplan.
 == Primary Goal ==
@@ Line 39: / Line 39: @@
 *Integrate the whole support efficiently with Apertium
 *Test it with other MT systems
-*Publish a Research Paper from the results of this work
+*Publish Research Paper(s) from the results of this work

@@ Line 215: / Line 215: @@
 === Step.9  Solving issue for Abbreviation ( NON-CONVENTIONAL ) ===
- It was very usually noted that people have a habit of referring to long words in abbreviated form. These words are incomplete but are an originating subsequence for word which should have been present
+It was very usually noted that people have a habit of referring to long words in abbreviated form. These words are incomplete but are an originating subsequence for word which should have been present
   rehab -> rehabilitation

@@ Line 159: / Line 159: @@
 The functional Prototype involving the whole processing, and right now correctly working for English is available at the github link ->
+https://github.com/akshayminocha5/apertium-non-standard-input-task
-<LINK TO GITHUB REPO >
 Below is the description of the working of the Modules -
@@ Line 215: / Line 215: @@
 === Step.9  Solving issue for Abbreviation ( NON-CONVENTIONAL ) ===
  It was very usually noted that people have a habit of referring to long words in abbreviated form. These words are incomplete but are an originating subsequence for word which should have been present

@@ Line 14: / Line 14: @@
 *'''Why is it that they are interested in the Apertium project?'''
-** This current project on "non-standard text input" has everything I love to work on. From, Informal data from Twitter/IRC/etc., handling noise removal, building FST's, analysing data and at the end building Machine Translation systems. I believe that this approach can be standardized for many source languages with the approach I have in mind. Having learned from the process in English the language independent module will be a good contribution. Also, the translation quality we are working on should be intact when we are giving back to the community, well at least this is an important step. This is also one of the kind of projects whose implementation will help translation on all the language pairs on apertium at the end.
+** This current project on "non-standard text input" has everything I love to work on. From, Informal data from Twitter/IRC/etc., handling noise removal, building FST's, analysing data and at the end building Machine Translation systems. I believe that this approach can be standardized for many source languages with the approach I have in mind. Having learned from the process in English, the language independent module will be a good contribution. Also, the translation quality we are working on should be intact when we are giving back to the community, well at least this is an important step. This is also one of the kind of projects whose implementation will help translation on all the language pairs on apertium at the end.
 *'''Reasons why Google and Apertium should sponsor it''' - I'd love to work on the current project with the set of mentors. This project is important because the MT community on an open level should also welcome the change in the use of language, in the form of the popular non standard text, by the people. This will extend our reach to several people and practically increase the efficiency of the translation task no doubt.
@@ Line 31: / Line 31: @@
 *'''List any non-Summer-of-Code plans you have for the Summer, especially employment, if you are applying for internships, and class-taking. Be specific about schedules and time commitments. we would like to be sure you have at least 30 free hours a week to develop for our project.'''
-**I am dedicated towards this project and am very excited about working on it. A committment of 30 hours a week would not be a problem, I might even work on, some more so as to complete the weekly goals as describes in the Workplan.
+**I am dedicated towards this project and am very excited about working on it. A commitment of 30 hours a week would not be a problem, I might even work on some more so as to complete the weekly goals as described in the Workplan.
 == Primary Goal ==

@@ Line 251: / Line 251: @@
 **Other sources of non standard data don’t see to get a significant improvement
 **BLEU score improvement marginal
-<br/>
 *This is a standard research<ref>Sproat, Richard, et al. "Normalis(z)ation of non-standard words." Computer Speech & Language 15.3 (2001): 287-333.</ref> on the Non-Standard Words, (NSW) It suggests that Non-standard words are more ambiguous with respect to ordinary words in the ways of pronunciation and interpretation. In many applications, it is desirable to “normalize” text by replacing the NSWs with the contextually appropriate ordinary word or sequence of words. They have generally categorized numbers, abbreviations, other markup, url’s, handled capitalisation, etc. A very interesting method on tree based abbreviation model has been suggested in the research, which can give us ideas on improving our current abbreviation model or just have another addition to it in the model. This includes suggestion for vowel Dropping, shortened words and first syllable usage. <br/> The issue with most of the research is the limitation to a particular language in this case English. They have standardized the most common points of english leaving scope for a lot of improvement. In our kind of processing at the moment we are not considering any specific markup techniques within the pipeline but this paper shows some promising work on the same which can be useful for developers, and other users who want to analyse the data in more detail. Such a convention can be added easily after conducting experiments and seeing results.
-<br/>
 *This <ref> Pennell, Deana, and Yang Liu. "A Character-Level Machine Translation Approach for Normalis(z)ation of SMS Abbreviations." IJCNLP. 2011. </ref> is a completely different approach where the author tries to solve the problem by proposing a character level machine translation approach. The issue here is accuracy, they have used the Jazzy spell checker<ref> Mindaugas Idzelis. 2005. Jazzy: The java open source spell checker. </ref> as baseline and the compared it with previous such research. The issue here is the huge resource being used up in training and tuning the MT system and also, such a system would have complications being included on the run with Apertium.
-<br/>
 *This research <ref> Lopez, Adam, and Matt Post. "Beyond bitext: Five open problems in machine translation." </ref>  is more idea centric, where the author says the MT research is far from complete and we face many challenges. With our project we aim to target these problems specifically. Translation of Informal text and Translation of low resource language pairs are the ones which concern Apertium and us the most.
-<br/>
 *This research <ref>  Lo, Chi-kiu, and Dekai Wu. "Can informal genres be better translated by tuning on automatic semantic metrics." Proceedings of the 14th Machine Translation Summit (MTSummit-XIV) (2013). </ref> identifies the difficulties faced not only by the Translation community with the web forum data and other informal genres but also by people working on semantic role labelling, and probably many more who rely on data analytics, etc. <br/> They propose that evaluation of systems which are MEANT tuned performed significantly better than other systems tuned according to BLEU and TER. With our module the Error analysis suggested would improve on the system, because of a significant rise in the number of known words, grammar and word sense, the semantic parser being used here would perform better.
-<br/>
 *This research<ref>Pennell, Deana L., and Yang Liu. "Normalis(z)ation of informal text." Computer Speech & Language 28.1 (2014): 256-277.</ref> idea’s to the approach very similar to ours. But they have focussed mainly on the abbreviated word re-modelling and expansion, by implementing a character based translation model.
-<br/>
 *Inspired by this <ref>S. Bangalore, V. Murdock, G. Riccardi - Bootstrapping bilingual data using consensus translation for a multilingual instant messaging system, 19th International Conference on Computational Linguistics, Taipei, Taiwan (2002), pp. 1–7
 </ref> research, Srinivas Bangalore has suggested a method of bootstrapping from the data on the chat forums and other informal sources. So that we can build up abbreviation resources for a particular language. The way we can proceed with this task in Apertium, as I had suggested before was to first take in a list of few abbreviations and the use them to suggest what other more frequent words in the data might also count as abbreviations. This resource can be verified and then included for building up the system for the particular language.
 == Outline of Research Paper ==

@@ Line 16: / Line 16: @@
 ** This current project on "non-standard text input" has everything I love to work on. From, Informal data from Twitter/IRC/etc., handling noise removal, building FST's, analysing data and at the end building Machine Translation systems. I believe that this approach can be standardized for many source languages with the approach I have in mind. Having learned from the process in English the language independent module will be a good contribution. Also, the translation quality we are working on should be intact when we are giving back to the community, well at least this is an important step. This is also one of the kind of projects whose implementation will help translation on all the language pairs on apertium at the end.
+*'''Reasons why Google and Apertium should sponsor it''' - I'd love to work on the current project with the set of mentors. This project is important because the MT community on an open level should also welcome the change in the use of language, in the form of the popular non standard text, by the people. This will extend our reach to several people and practically increase the efficiency of the translation task no doubt.
-*'''Include a proposal, including'''
-**'''Reasons why Google and Apertium should sponsor it''' - I'd love to work on the current project with the set of mentors. This project is important because the MT community on an open level should also welcome the change in the use of language, in the form of the popular non standard text, by the people. This will extend our reach to several people and practically increase the efficiency of the translation task no doubt.
 *'''And a detailed work plan (including, if possible, a brief schedule with milestones and deliverables). Include time needed to think, to program, to document and to disseminate.'''