Talk:Ideas for Google Summer of Code - Revision history

Popcorndude: /* 2020 ideas */ new section

2021-01-18T21:14:56Z

2020 ideas: new section

Show changes

Francis Tyers: /* Old ideas */

2017-10-20T09:37:23Z

Old ideas

Francis Tyers: /* 2014 ideas */

2015-02-09T09:45:19Z

2014 ideas

Francis Tyers: /* 2014 ideas */

2015-02-09T09:44:23Z

2014 ideas

Francis Tyers: /* 2013 ideas */

2015-02-09T09:43:24Z

2013 ideas

Show changes

Francis Tyers: /* Apertium on Android with Necessitas QT */

2014-02-12T11:55:07Z

Apertium on Android with Necessitas QT

Jimregan: /* Apertium on Android with Necessitas QT */ update - found the link

2012-03-02T20:08:02Z

Apertium on Android with Necessitas QT: update - found the link

Jimregan: /* Apertium on Android with Necessitas QT */

2012-03-02T19:39:30Z

Apertium on Android with Necessitas QT

Francis Tyers: /* Apertium on Android with Necessitas QT */

2012-02-29T21:17:40Z

Apertium on Android with Necessitas QT

Francis Tyers: /* Apertium on Android with Necessitas QT */

2012-02-29T21:08:25Z

Apertium on Android with Necessitas QT

@@ Line 36: / Line 36: @@
 * Simard, Michel (1998). "Automatic Insertion of Accents in French Texts". Proceedings of EMNLP-3. Granada, Spain.
 * Rada F. Mihalcea. (2002). "Diacritics Restoration: Learning from Letters versus Learning from Words". ''Lecture Notes in Computer Science'' 2276/2002 pp. 96--113
+==2017==
+=== <u>Back-off morphological generation with RNNs</u> ===
+* '''Difficulty''':<br><span style="background-color: #ffbdbd">1. Hard</span>
+* '''Required skills''':<br>c++
+* '''Description''':<br>Write a pared-down RNN library to do morphological generation, add it as a backoff to [[lt-proc]] in case of a generation fail.
+* '''Rationale''':<br>One of the most frustrating things when developing a new language pair is that you have to get the tags ''just right'' in order to be able to generate, if they come in the wrong order, or if you're missing a tag or if you have too many tags, then you don't get any surface form generated. For many languages it is possible to train RNNs to a high-level of accuracy.
+* '''Mentors''':<br>[[User:Francis Tyers]] [[User:Mlforcada]]
+* '''[[Morphological generation using RNNs|read more...]]'''
+=== <u>Shallow-function labeller</u> ===
+* '''Difficulty''':<br><span style="background-color: #cdefcd">2. Medium</span>
+* '''Required skills''':<br>Python, shell scripting
+* '''Description''':<br>Implement a prototype shallow syntactic function labeller for Apertium
+* '''Rationale''':<br>In many pairs it is useful to know in addition to the morphological tags of a word, syntactic function tags in order to make an adequate translation. For instance, you might want to know in an ergative language if an absolutive is subject or object while translating. A shallow function labeller takes an annotated corpus and produces a model which can annotate new text.
+* '''Mentors''':<br>[[User:Unhammer|Unhammer]], [[User:Francis Tyers|Francis Tyers]], [[User:Mlforcada|Mikel Forcada]]
+* '''[[/Shallow-function labeller|read more...]]'''
+=== <u>Discontiguous multiwords</u> ===
+* '''Difficulty''':<br><span style="background-color: #cdefcd">2. Medium</span>
+* '''Required skills''':<br>C++, Knowledge of FSTs
+* '''Description''':<br>The task will be to develop, or adapt a module to deal with these kind of contiguous multiword expressions, for example, taking 'liggja ekki fyrir' and reordering it as 'liggja# fyrir ekki'.
+* '''Rationale''':<br>In many languages, such as English, Norwegian and Icelandic, there are discontiguous multiwords, e.g. phrasal verbs, that we cannot easily support. For example 'liggja ekki fyrir' in Icelandic should be translated in English as 'to be not clear', but we cannot have 'liggja fyrir' as a traditional multiword because of the extra 'adverb', or it could even be a whole NP.
+* '''Mentors''':<br>[[User:Francis Tyers|Francis Tyers]]
+* '''[[/Discontiguous multiwords|read more...]]'''

@@ Line 283: / Line 283: @@
 |-
 !style="background-color: #cdcdef"| 3.&nbsp;Entry&nbsp;level ||colspan=2| ||  [[/Apertium assimilation evaluation toolkit|read&nbsp;more...]]
+|-
+!colspan=4 style="background-color: #cdcdcd"|'''Corpus-based lexicalised feature transfer'''
+|-
+|align=center| '''How ?'''<br/><small>(required skills)</small> ||align=center| '''What ?'''<br/><small>(description)</small> ||align=center| '''Why ?'''<br/><small>(rationale)</small> ||align=center| '''Who ?'''<br/><small>(mentors)</small>
+|-
+| C++, NLP || Make a module that sits somewhere in the Apertium pipeline (somewhere after the lexical selection and before morphological generation) that sets features (eg. tags) based on a model generated from a corpus. ||  Let's get down to brass tacks, sometimes we get really inadequate translations even though you'd never hear stuff like that. One of those things is when we output something as definite when it is never used as definite. One way of dealing with this is a lot of rules and lists in transfer, but those are hard to do. So, how about looking at a corpus for information about some features like definiteness, aspect, evidentiality, impersonal/reflexive pronoun use in Romance languages etc. || [[User:Francis Tyers|Francis&nbsp;Tyers]], [[User:Jimregan|Jimregan]]
+|-
+!style="background-color: #efcdcd"|1.&nbsp;Hard  ||colspan=2| || [[/Corpus-based lexicalised feature transfer|read&nbsp;more...]] ([[Google_Summer_of_Code/Report_2012|2012 project]])
 |-
 |}

@@ Line 275: / Line 275: @@
 |-
 !style="background-color: #cdcdef"|3.&nbsp;Entry&nbsp;level ||colspan=2| || [[/Improving support for non-standard text input|read&nbsp;more...]]
+|-
+!colspan=4 style="background-color: #cdcdcd"|'''Apertium assimilation evaluation toolkit''' ([[Google_Summer_of_Code/Report_2014|2014 project]])
+|-
+|align=center| '''How ?'''<br/><small>(required skills)</small> ||align=center| '''What ?'''<br/><small>(description)</small> ||align=center| '''Why ?'''<br/><small>(rationale)</small> ||align=center| '''Who ?'''<br/><small>(mentors)</small>
+|-
+| A scripting language || Starting from files containing sentences in the source language and reference translations, generate tests for human evaluation consisting of: (1) (optionally) the source sentence, (2) (optionally) the machine-translated version of the source sentences and (3) a reference translation of the sentence in which one or more content words have been deleted. The idea is to measure how the ability of human subjects to fill in the holes improves when the source or a machine translation of it are presented. The task involves also generating a program that computes the success as a function of the information presented to the user, and utilities to make the whole process automatic given an Apertium language pair. || Many Apertium language pairs are designed for assimilation (gisting) purposes. The evaluation described would measure how helpful they are in the task. ||  [[User:Francis Tyers|Francis&nbsp;Tyers]], [[User:mlforcada| Mikel Forcada]]
+|-
+!style="background-color: #cdcdef"| 3.&nbsp;Entry&nbsp;level ||colspan=2| ||  [[/Apertium assimilation evaluation toolkit|read&nbsp;more...]]
 |-
 |}

@@ Line 132: / Line 132: @@
 ::STL support: that's what I was saying. Rephrased: first, the primary issue was lack of C++ support; that resolved, the primary issue became lack of STL support; that resolved, the primary issue became lack of wstring support. I've done some checking, and can't seem to find a straight answer, so "probably not". There is an alternative version of stlport for android, that seems to have the necessary pieces, [http://www.crystax.net/android/ndk-r5.php here]. I still say Qt is a dead end. Even if it's possible to create an Intent with Qt -- and it's by no means certain that it is: http://comments.gmane.org/gmane.comp.lib.qt.android/2719 -- it seems a lot of trouble to go to for an outcome that will never be truly satisfactory. Either way, there's going to be a lot of dicking around with JNI, so it would probably just be best to wrap Apertium in a Java shim. On the plus side, there are tools like jnaerator to automate most of that. -- [[User:Jimregan|Jimregan]] 19:39, 2 March 2012 (UTC)
+==2013 ideas==
+{|class="wikitable"
+|-
+!colspan=4 style="background-color: #cdcdcd"|'''Template-based bilingual dictionary'''
+|-
+|align=center| '''How ?'''<br/><small>(required skills)</small> ||align=center| '''What ?'''<br/><small>(description)</small> ||align=center| '''Why ?'''<br/><small>(rationale)</small> ||align=center| '''Who ?'''<br/><small>(mentors)</small>
+|-
+| XML, C++, python  || Design a format similar to [[bidix]] (declarative XML establishing language 1 <> language 2 correspondences) that allows the use of templates, as well as the back-end to process it (i.e., it should compile into an FST).  It should deal with [[/Discontiguous multiwords|discontiguous multiwords]] and [[/Complex multiwords|complex multiwords]], allowing them to be easily translated, and should provide some mechanism (some sort of ranking) to deal with multiple matching sets of templates for a given translation (similar to [[CG]]).  It should essentially allow one to bypass [[transfer]] rules and [[constraint grammar|disambiguation]] and produce similar (if not better) accuracy in translation.  || A templatic bidix forces the designer of a language pair to be more explicit, and also consolidates pair development. Furthermore, there are several types of phenomenon such a system could deal with that are currently highly problematic.  || [[User:Firespeaker|Firespeaker]] [[User:Francis Tyers|Francis&nbsp;Tyers]]
+|-
+!style="background-color: #efcdcd"|1.&nbsp;Hard  ||colspan=2| || [[/Template-based bilingual dictionary|read&nbsp;more...]]
+|-
+|-
+!colspan=4 style="background-color: #cdcdcd"|'''Improved bilingual dictionary induction'''
+|-
+|align=center| '''How ?'''<br/><small>(required skills)</small> ||align=center| '''What ?'''<br/><small>(description)</small> ||align=center| '''Why ?'''<br/><small>(rationale)</small> ||align=center| '''Who ?'''<br/><small>(mentors)</small>
+|-
+| Python, XML || Write a set of scripts that can generate valid and consistent Apertium bilingual dictionary entries from a word-aligned parallel corpus. This will involve making a basic templating system. The scripts should ideally be able to incorporate quality measures to determine how reliable the translations extracted from the corpus are. || There are some tools to make bilingual dictionaries from parallel corpora (such as [[retratos]]) but they don't take into account that words in different languages can require different entries in the bilingual dictionary depending on their morphological characteristics. This means that although finding the translations is automatic, most generated entries have to be checked, which can greatly increase the amount of time it takes to make a new translation system.   ||  [[User:Francis Tyers|Francis&nbsp;Tyers]]
+|-
+!style="background-color: #cdcdef"|3.&nbsp;Entry&nbsp;level ||colspan=2| || [[/Improved bilingual dictionary induction|read&nbsp;more...]]
+|-
+|-
+!colspan=4 style="background-color: #cdcdcd"|'''Improvements in lexical-selection module'''
+|-
+|align=center| '''How ?'''<br/><small>(required skills)</small> ||align=center| '''What ?'''<br/><small>(description)</small> ||align=center| '''Why ?'''<br/><small>(rationale)</small> ||align=center| '''Who ?'''<br/><small>(mentors)</small>
+|-
+| C++, python || Implement a number of improvements to the [[lexical selection module]], particularly involving the rule-learning scripts. || The lexical selection module in Apertium is currently a prototype. There are many optimisations that could be made to make it faster and more efficient. There are a number of scripts which can be used for learning lexical-selection rules, but the scripts are not particularly well written. Part of the task will be to rewrite the scripts taking into account all possible corner cases.  ||  [[User:Francis Tyers|Francis&nbsp;Tyers]]
+|-
+!style="background-color: #cdefcd"|2.&nbsp;Medium  ||colspan=2| ||  [[/Improvements in lexical-selection module|read&nbsp;more...]]
+|-
+|-
+!colspan=4 style="background-color: #cdcdcd"|'''Visual interface to write structural transfer rules'''
+|-
+|align=center| '''How ?'''<br/><small>(required skills)</small> ||align=center| '''What ?'''<br/><small>(description)</small> ||align=center| '''Why ?'''<br/><small>(rationale)</small> ||align=center| '''Who ?'''<br/><small>(mentors)</small>
+|-
+| C++, scripting languages, GUI design || Write a graphical user interface to write structural transfer rules (one that reads in (a subset of) the current XML-based language, allows for a graphical, intuitive editing of the rules, and writes compilable .t1x, .t2x or .t3x files) || Apertium structural transfer rules are currently encoded in XML-based formats. These are very overt and clear, but clumsy and may be hard to write. The idea is to design a visual programming language of the style of like  [http://scratch.mit.edu/ Scratch], where jigsaw-puzzle-style pieces corresponding to statements and control structures fit only if the syntax is right.
+ || [[User:mlforcada| Mikel Forcada]], mentors wanted!
+|-
+!style="background-color: #efcdcd"|1.&nbsp;Hard ||colspan=2| || [[/Visual interface to write structural transfer rules|read&nbsp;more]]
+|-
+|}

← Older revision		Revision as of 20:08, 2 March 2012
Line 131:		Line 131:
	:According to Boyán: "Parece ser que sí que soporta STL. ... por lo visto la propuesta de Sergio de pasar Apertium a Android sigue siendo viable y tarde o temprano habrá que hacerlo. Tiene Jim la razón en que no se usaría QT para nada, sino un Intent de Android" - [[User:Francis Tyers\|Francis Tyers]] 21:08, 29 February 2012 (UTC)		:According to Boyán: "Parece ser que sí que soporta STL. ... por lo visto la propuesta de Sergio de pasar Apertium a Android sigue siendo viable y tarde o temprano habrá que hacerlo. Tiene Jim la razón en que no se usaría QT para nada, sino un Intent de Android" - [[User:Francis Tyers\|Francis Tyers]] 21:08, 29 February 2012 (UTC)

	::STL support: that's what I was saying. Rephrased: first, the primary issue was lack of C++ support; that resolved, the primary issue became lack of STL support; that resolved, the primary issue became lack of wstring support. I've done some checking, and can't seem to find a straight answer, so "probably not". There is an alternative version of stlport for android, that seems to have the necessary pieces, ~~but~~ ~~I've forgotten the name~~. I still say Qt is a dead end. Even if it's possible to create an Intent with Qt -- and it's by no means certain that it is: http://comments.gmane.org/gmane.comp.lib.qt.android/2719 -- it seems a lot of trouble to go to for an outcome that will never be truly satisfactory. Either way, there's going to be a lot of dicking around with JNI, so it would probably just be best to wrap Apertium in a Java shim. On the plus side, there are tools like jnaerator to automate most of that. -- [[User:Jimregan\|Jimregan]] 19:39, 2 March 2012 (UTC)		::STL support: that's what I was saying. Rephrased: first, the primary issue was lack of C++ support; that resolved, the primary issue became lack of STL support; that resolved, the primary issue became lack of wstring support. I've done some checking, and can't seem to find a straight answer, so "probably not". There is an alternative version of stlport for android, that seems to have the necessary pieces, [http://www.crystax.net/android/ndk-r5.php here]. I still say Qt is a dead end. Even if it's possible to create an Intent with Qt -- and it's by no means certain that it is: http://comments.gmane.org/gmane.comp.lib.qt.android/2719 -- it seems a lot of trouble to go to for an outcome that will never be truly satisfactory. Either way, there's going to be a lot of dicking around with JNI, so it would probably just be best to wrap Apertium in a Java shim. On the plus side, there are tools like jnaerator to automate most of that. -- [[User:Jimregan\|Jimregan]] 19:39, 2 March 2012 (UTC)

← Older revision		Revision as of 19:39, 2 March 2012
Line 130:		Line 130:

	:According to Boyán: "Parece ser que sí que soporta STL. ... por lo visto la propuesta de Sergio de pasar Apertium a Android sigue siendo viable y tarde o temprano habrá que hacerlo. Tiene Jim la razón en que no se usaría QT para nada, sino un Intent de Android" - [[User:Francis Tyers\|Francis Tyers]] 21:08, 29 February 2012 (UTC)		:According to Boyán: "Parece ser que sí que soporta STL. ... por lo visto la propuesta de Sergio de pasar Apertium a Android sigue siendo viable y tarde o temprano habrá que hacerlo. Tiene Jim la razón en que no se usaría QT para nada, sino un Intent de Android" - [[User:Francis Tyers\|Francis Tyers]] 21:08, 29 February 2012 (UTC)

			::STL support: that's what I was saying. Rephrased: first, the primary issue was lack of C++ support; that resolved, the primary issue became lack of STL support; that resolved, the primary issue became lack of wstring support. I've done some checking, and can't seem to find a straight answer, so "probably not". There is an alternative version of stlport for android, that seems to have the necessary pieces, but I've forgotten the name. I still say Qt is a dead end. Even if it's possible to create an Intent with Qt -- and it's by no means certain that it is: http://comments.gmane.org/gmane.comp.lib.qt.android/2719 -- it seems a lot of trouble to go to for an outcome that will never be truly satisfactory. Either way, there's going to be a lot of dicking around with JNI, so it would probably just be best to wrap Apertium in a Java shim. On the plus side, there are tools like jnaerator to automate most of that. -- [[User:Jimregan\|Jimregan]] 19:39, 2 March 2012 (UTC)

← Older revision		Revision as of 21:17, 29 February 2012
Line 129:		Line 129:
	If the wstring issue no longer exists, it should be possible to run apertium-service unmodified. A JNI wrapper around it is the best way forward for a C++ port. Necessitas QT is a dead end idea. -- [[User:Jimregan\|Jimregan]] 15:53, 29 February 2012 (UTC)		If the wstring issue no longer exists, it should be possible to run apertium-service unmodified. A JNI wrapper around it is the best way forward for a C++ port. Necessitas QT is a dead end idea. -- [[User:Jimregan\|Jimregan]] 15:53, 29 February 2012 (UTC)

	:According to Boyán: "Parece ser que sí que soporta STL. ... por lo visto la propuesta de Sergio de pasar Apertium a Android sigue siendo viable y tarde o temprano habrá que hacerlo." - [[User:Francis Tyers\|Francis Tyers]] 21:08, 29 February 2012 (UTC)		:According to Boyán: "Parece ser que sí que soporta STL. ... por lo visto la propuesta de Sergio de pasar Apertium a Android sigue siendo viable y tarde o temprano habrá que hacerlo. Tiene Jim la razón en que no se usaría QT para nada, sino un Intent de Android" - [[User:Francis Tyers\|Francis Tyers]] 21:08, 29 February 2012 (UTC)

← Older revision		Revision as of 21:08, 29 February 2012
Line 128:		Line 128:

	If the wstring issue no longer exists, it should be possible to run apertium-service unmodified. A JNI wrapper around it is the best way forward for a C++ port. Necessitas QT is a dead end idea. -- [[User:Jimregan\|Jimregan]] 15:53, 29 February 2012 (UTC)		If the wstring issue no longer exists, it should be possible to run apertium-service unmodified. A JNI wrapper around it is the best way forward for a C++ port. Necessitas QT is a dead end idea. -- [[User:Jimregan\|Jimregan]] 15:53, 29 February 2012 (UTC)

			:According to Boyán: "Parece ser que sí que soporta STL. ... por lo visto la propuesta de Sergio de pasar Apertium a Android sigue siendo viable y tarde o temprano habrá que hacerlo." - [[User:Francis Tyers\|Francis Tyers]] 21:08, 29 February 2012 (UTC)