User:Mjaskowski

Name: Maciej Jaśkowski

E-mail address maciej.jaskowski on gmail account IRC: mjaskowski (not registered!) skype: maciej.jaskowski (usually I'm offline though)

I live in Poland => CE time

Why is it you are interested in machine translation?

I can speak 4 languages (Polish, French, English and German) and learning a new langauge is always for me something cool. I know as well how much time does one need to learn yet another language. I can only imagine problems arising if one wants to learn a language which is not as popular as languages I have mentioned above (specifically, try learning Polish!)

But why do we learn languages in the first place? Everyone has his own reasons but arguably the most important one is simply that we want to understand what other people have to say, what they have said or written already. In other words, for numerous reasons, there is a lot of information which doesn't get translated into your mother tongue but still it might be relevant for you!

The knowledge of languages helps me quite a lot in everyday web crawling. In fact I can't imagine ignoring the english or french part of Web. And it is the information which is arguably the most important thing in the 21st century! MT may give us access to all the information world wide which we don't right now! Translation is of specifically high importance for EU where each language is equally important according to the EU law.

That is why MT is sooo important these days and also why it interesting. It is always interesting to mess with stuff which is needful.

MT is like a Holy Grail for Computational Linguistics in which I find a particular interest. And why do I like it? Because it combines to a large extent all my scientific interests: machine learning, data mining and linguistics.

Why is it that they are interested in the Apertium project?

To be honest I knew nothing about Apertium before I started looking for a nice Open-Source community for GSoC. I believe that in such a community, through knowledge sharing, I can learn many things not necessarily directly connected with programming or scientific stuff but also how to maintain big projects and how to work in a sizable group.

From the beginning, however, I knew that I want to get involved into a project related to linguistics, machine learning or data mining. These are stuff I am interested in and I want to dive into them during my PhD studies. And that is how I found Apertium.

But the more I talk with you, the more I learn about Apertium, the more code I read, I get persuaded that Apertium is not only the best known (to Google) open-source MT system but also one with fine community around which is I believe of crucial importance for huge projects like that one.

And I like it even more :-)

Which of the published tasks are you interested in?

"Accent and diacritic restoration"

Understanding of the problem

We are to write an application (in C++) which takes as input text (a result of deformatter) and outputs a text in UTF-8 with diacritics restored, the superblanks leaving untouched.

Thanks to the work of Kevin Scannell we have already a Perl script which does the work for us. A drawback of his script is that it's... a script. It can serve us as a reference, though.

pipeline As such the application can be introduced into the Apertium pipeline between deformatter and the morphological analyser. Changes to apertium-header.sh script are therefore necessary; we need to introduce a new switch including the application into pipeline.

details In fact we are to write 3 applications performing the same task [1] (LLU, LL2, FS) but in different manner. The first two being word based and intensively using dictionaries. LLU is a unigram model and LL2 is a bigram model. The latter is letter/grapheme based. The dictionary based algorithms are generally better. Their disadvantage, however, is that they can't provide an answer for a word never seen in the dictionary and they work only if the dictionary is big enough.

Each app will share the same interface. Each one should allow at least for:

• diacritic restoration of a text (a result of deformatter) given it's language
• learning new language (i.e. takes as input a corpus in some language and transforms it into data necessary for diacritic restoration)

evaluation Once they are all implemented we should perform automatic performance tests in order to choose the best combination of the three and build on top of them a metapplication (CMB) combining them in the best possible way for a language given.

We are to write a simple script counting number of words which diacritics where restored correctly and incorrectly (or to count precision and recall).

• The tests should be composed from parts of corpus which was never seen by the app.
• Each test set will be composed of the original file (with right diacritics) an input file (asciified or partially asciified).
• In the case of languages with little training data we will perform 10-fold cross-validation

MT evaluation Finally, one should check if the app improves or deteriorates MT. It is very well probable that unicodifying a file where diacritics were about ok, may deteriorate the quality of translation but it is worth checking. See also "texts partially deprived of diacritics" idea. To this end we will use apertium-eval-translator tool to measure Word Error Rate (WER).

PoS evaluation We can also consider measuring the robustness of the PoS tagger with or without diacritic restoration. If we decide to implement some kind of diacritic restoration built in one of the existing apps in the pipeline, this measure will help us compare different approaches.

Some ideas and remarks

Texts partially deprived of diacritics

In real world applications it might very well be that the input file is only partially deprived of diacritics. We could ascify the file completely before processing but it seems to be important to take advantage of the diacritics given.

The assumption that the diacritics given are the right ones seem plausible; instead of ascifying the file, we can employ a lazy approach and (roughly speaking) ascify only if we can't find any other solution for a word (in a context) given.

How about text with the wrong diacritics? e.g. seeing ǎ where it should be ă ? - Francis Tyers 20:17, 2 April 2010 (UTC)

For example, Francis,

• if we see 'połka' we look for words which ascification yields 'polka' but the third letter is 'ł'.

And we get 'półka' or 'półką' (but not: 'polka' or 'polką')

• if, however, we see 'pólska' (i.e. the diacritics are wrong) then we can't find any relevant word. We ascify it then to 'polska' and find two possible answers: 'polską' and 'polska'
• finally, if we see a word which diacritics are improper but we do find a word as in the first sample above then... well... no one is perfect :-) it is just a heuristic which might help with some languages but deteriorate results in others.

If we were to consider such cases we would have to build a system correcting orthographical spelling errors.

Ideas to improve LL

WSD Although Kevin Scannell is not sure if my proposition will give us any improvement, I am keen to check the impact of applying Word Sense Disambiguation methods to LL algorithm. Of course the algorithm might work only if we have a dictionary big enough (which is also the case for ordinary LL and LL2)

dictionary asciification Our aim is to improve Apertium in such a way that it can translate a text even if it is partially deprived of diacritics. Therefore, it is not insane to check if simple asciification of training corpus and dictionary would not yield comparable or better results.

In some languages it might help a lot. Consider a noun phrase: without diacritics: 'z piekna polka'. It is obvious for anyone speaking polish that it should be converted to 'z piękną polką' because after 'z' we have instrumental. That information, however, knows the PoS tagger not our app.

Obviously using such asciified dictionaries might in turn deteriorate overall performance of Apertium. But that is also the case with apps in the pipeline. If we choose to try that approach we will need an evaluation method to compare it with the standard approach. To this end, we might use the post PoS evaluation proposed above.

Investigating occuring errors

It is tempting for me to look in detail on the output of each and every of the algorithms to figure out what kind of errors are made. E.g. for the LL and LL2 algorithms one can foresee such kind of errors: 0. a word is misspelled 1. the ascified word is spelled correctly but it has never occured in the dictionary 2. two or more unicodification of an ascified word occur in the dictionary in the same context

the last three propositions are rather "low priority". To be done if time

A detailed work plan

czy da się wczytać perlowe dane c++em? plugin do ooffice? --> ooocostam?

Time Line:
Community Bonding Period
Week 1: April 27 - May 2

Week 2: May 3 - May 9

Week 3: May 10 - May 16

Week 4: May 17 - May 23

Coding Period
Week 5: May 24 - May 30

Week 6: May 31 - June 6

Week 7: June 7 - June 13

Week 8: June 14 - June 20

Deliverable:

Week 9: June 21 - June 27

Week 10: June 28 - July 24

Week 11: July 5 - July 11

Week 12: July 12 - July 18 (Mid-term Evaluation)

Deliverable:

Week 13: July 19 - July 25

Week 14: July 26 - August 1

Week 15: August 2 - August 8

Week 16: August 9 - August 16

Final Evaluation: August 9 - August 16

non-summer-of-Code plans

Until may 30th -> classes (10h/week, half-time job 20h/week)
After may 30th -> free of commitments. I may need a 3-4 free days in june due to exams.

List your skills and give evidence of your qualifications

Education: I have just graduated in mathematics on the University of Warsaw [2] (got 5 on MSc diploma; 5 is the best mark in Poland). I spend 10 months on Erasmus in Ecole Polytechnique (Palaiseau) and had some brilliant results
Right now I am pursuing MSc in computer science on the same faculty (expected graduation date: june 2011).
My GPA in both subjects exceeds 4.6 (5 being the best mark, 2 the worst one).
I am also looking for a good oportunity to start PhD studies on problems related to machine learning and/or computational linguistics in general.

Linguistics: Last semester I had the opportunity to take part in a course "Linguistic Engeeniring -- Words" conducted by Adam Przepiórkowski[nlp.ipipan.waw.pl/~adamp/]. Right now I take part in the second part of that course. Mr Przepiórkowski has a very good opinion of me, my skills and of the project I have done during the course. See: [3]

C++: My biggest extra-academic experience with C++ I had last summer during internship at ICM [4]. We were to port an app from Fortran to C++ and then to CUDA. We managed to do the first part, the second appeared to be a bit more complicated -- we continue working on it again since last month.

Open-Source experience: I have, however, no experience in Open-Source projects. It might be seen as a shortcoming of my proposal. On the other hand, I am keen to join a fine community like Apertium seems to be and GSOC is a great opportunity to do so and to gain this experience :-)

Team work: A thing I am trully proud of, is my presidency of MIMUW[5] Students Government[6] between November 2009 and March 2010.
We managed to make a long jump from an almost non-existing SG with a single person actively acting, to a SG where 10 guys (or 1% of the students) is highly involved and other 10-15 is supporting it from time to time. To a SG which works like a small company :-)
Here[7] you can find some statistics.