Talk:Google Summer of Code/Application 2019 - Revision history

Xavivars: /* GSoC proposals */

2019-01-31T16:57:30Z

GSoC proposals

Xavivars: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T12:02:56Z

Discussion: Apertium strategy and the GSoC

Xavivars: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T11:39:36Z

Discussion: Apertium strategy and the GSoC

Mlforcada: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T10:40:39Z

Discussion: Apertium strategy and the GSoC

Mlforcada: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T10:40:05Z

Discussion: Apertium strategy and the GSoC

Mlforcada: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T10:39:42Z

Discussion: Apertium strategy and the GSoC

Mlforcada: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T10:36:56Z

Discussion: Apertium strategy and the GSoC

Mlforcada: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T10:36:34Z

Discussion: Apertium strategy and the GSoC

Mlforcada: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T10:36:12Z

Discussion: Apertium strategy and the GSoC

Mlforcada: /* Discussion: Apertium strategy and the GSoC */

2019-01-26T10:35:46Z

Discussion: Apertium strategy and the GSoC

@@ Line 84: / Line 84: @@
 # Update the functionality of the Java engine. Port a CG3 engine. Port (or adapt existing) HFST engine.
 ## [[User:Francis Tyers|ftyers]]:  Disagree with this one. Perhaps I would rephrase it "get pipeline for C++ to Android" or "get pipeline for generating from C++ to usage in OmegaT". I would like to add a section on integration/interoperability with other language data. Currently the UD project has freely-licensed annotated corpora for a good many languages in Apertium. Using them in Apertium is problematic because of different tokenisation standards and different tagsets. There should be some effort to make these more interoperable so that we can avoid duplicating effort (having people annotate English text).
-## [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) +1
+### [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) +1 to Fran's comment
 # [[User:Francis Tyers|ftyers]]: '''MEETING PROPOSAL''': If many people are going to be in Dublin this year for the [https://www.mtsummit2019.com/ MT Summit], how about having an Apertium meeting/workshop where we flesh this stuff out and try and gain some direction? It won't be in time for GSOC this year, but it's definitely nearly ten years since we had a real planning meeting.
 ## --[[User:Mlforcada|Mlforcada]] ([[User talk:Mlforcada|talk]]) 11:32, 26 January 2019 (CET) This is really a cool idea. Let me see what we can do.

@@ Line 5: / Line 5: @@
 Does Apertium as a project need more direction? Where would Apertium stand in the [https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar cathedral–bazaar] line? Most work in Apertium is bazaar-like (people work on the languages and platform features they want) but we have some cathedral-style action such as this elected PMC. We also vote on GSoC projects, etc. But ---and this is mainly my fault as long-standing president--- we have not devoted enough effort to planning. One could say that Apertium basically flies on autopilot, and that, every now and then, some Apertiumers correct the course. I think we need to decide and clarify how to implement our mission (see below). And we need to give ourselves a method to do so.
+* [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) IMHO, having some project roadmap, regardless of GSoC, would be an extremely powerful tool to understand how we want to achieve our mission. Even if the mission of Apertium "as an idea" is clear, there hasn't been too much clarity on "Apertium as a FOSS project".
+** Do we want to grow Apertium by getting more contributors?
+** If we had to choose, do we prefer a solid foundation (core modules easier to understand/contribute) or a wider coverage (more specific modules that do specific things)?
 Desired impact, intended mission. We need to collectively decide what is important and concentrate our efforts there. Our declared mission is laid out in the bylaws as follows:
@@ Line 30: / Line 35: @@
 # Languages that may be well supported by more than one MT vendor and have good or very good Apertium engines: Catalan–Spanish, Portuguese–Spanish, French–Spanish, etc. Their impact is large because they are widely used. Improving their quality may increase their impact and give Apertium some prestige.
 ## [[User:Francis Tyers|ftyers]]: Agree, some of these don't even use the latest code from -lex-tools and -separable, and a substantial amount of work could be done to improve them based on these. Out of the three: Portuguese-Spanish = coverage is the main problem as far as I can tell, we have 15k entries, we need 50k or so, e.g. >95% coverage instead of >90%; French--Spanish = needs more rules, and much better disambiguation.
+* [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) There has been a lot of effort recently in fra-cat (Hèctor) and spa-cat (Jaume). I'm pretty sure a lot of rules could be reused for fra-spa.
@@ Line 38: / Line 44: @@
 ## [[User:Francis Tyers|ftyers]]: Here I disagree. I think that having a Java port is a dead end. It is inevitable that it will get out of date very quickly. I think that a better strategy is to (sentence incomplete in Fran's message).
 ## [[User:Tino_Didriksen|Tino Didriksen]]: My very strong opinion about Java is to broadly speaking forget about Java. Maintaining Java ports is quite frankly a waste of time, and such ports would too quickly lag behind the C++ codebases. All the C++ code works cross-platform, also on Android - it just needs scripts to be built and bundled correctly, which is a mostly one-time setup task perfectly suitable for GSoC. What we may need is a JNI layer and better C++ APIs so that Java can call the native backends without spawning separate piped processes - this would also be a good GSoC project. An alternative if one really wants non-native libraries is something like Emscripten that can compile the C++ code to JavaScript, but that's just yet another build target similar to Android.
+##  [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) I agree with Fran and Tino: having two implementations of Apertium doesn't scale in general, and even less in a project like ours. There is simply not enough people writing code to keep both versions in sync. It's true that we've been pushing the JAVA port up to speed via GSoCs, but most of the recent modules (apertium-separable, apertium-lex-tools) or other tools broadly used in pairs (cg) are not available in the Java port. And if we had to focus on something, I'd rather spend the time fixing the known bugs of the C++ codebase than reimplementing what we already have in C++ into Java. That said, I personally like better the Java codebase, the C++ code is quite difficult to understand and contribute to, even for someone relatively familiar to the project, and who writes code for a living.
 # improve the CG3 processor (it is too slow now, I gather, and is heavily used in many languages).
 ## [[User:Francis Tyers|ftyers]]: About CG3, it is rarely the bottleneck until you get up to thousands of rules. The transfer component is usually a bigger bottleneck, although I would welcome hard facts about that. In any case, I think Apertium is reasonably fast, most of the time.---Improvements are always possible though.
+### [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) If I'm not wrong, Java transfer was faster than the C++ one. Could we maybe come up with a solution to leverage the ideas (not the implementation) of it? Moving from "interpreted" transfer rules to a "compiled" version of them (like we do for all dictionaries)?
 ## [[User:Tino_Didriksen|Tino Didriksen]]: it's been tried and mostly failed. To get truly explosive performance boosts, the corresponding limitations in rule number and rule complexity one has to live with are stifling. But I'm always open for new ideas, and I hope to be proven wrong - while I don't think CG-3 is too slow, faster is better.
+## [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) I agree CG-3 is quite slow, and in some languages where the disambiguation has improved a lot via rules (i.e. Catalan) it can become the bottleneck pretty easily. But, even if it's the bottleneck, and an improvement in there would be helpful, Apertium is still quite fast.
 ## [[User:Francis Tyers|ftyers]]:For me, there are a number of things that should be improved in the engine.
 ### The first is that each component should have the potential to be weighted, and these weights should be able to be learnt jointly. This is a big project   probably at least a PhD amount of work, but has the potential to offer very big improvements.
@@ Line 46: / Line 55: @@
 ### The format handling should be reworked, we really need it and it's so close. But unfortunately not something that could be done in a GSOC project by a  new Apertiumer.
 ### More expressive, and more linguist-friendly transfer formalism. The problem is that this is a non-trivial task, and one that is also hard to find money   for, when I mentioned it to someone recently, they said "that sounds like something the EU would have funded in the early-2000s".
-### An interesting research idea that people have discussed with me would   be to have Apertium be able to generate data for neural MT, but with a   more expressive formalism. We have done some work on this already, but not with a very generative system. I have some grant ideas about that,   e.g. from field linguistics -> NMT, if can expound on them if people are interested.
+### An interesting research idea that people have discussed with me would be to have Apertium be able to generate data for neural MT, but with a   more expressive formalism. We have done some work on this already, but not with a very generative system. I have some grant ideas about that,   e.g. from field linguistics -> NMT, if can expound on them if people are interested.
+### [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) As I mentioned before, the codebase as a whole is a mess. And one of the reasons is how it has been evolving. From a proof-of-concept used for some specific research, something has become a "production" module. And it's amazing to get new modules implementing new features like that. But we could do way more with a cleaner codebase, with more automated tests that prevented regressions, etc. Inverting on quality of the existing codebase I think would pay back in the future.
 === Evaluation ===
@@ Line 74: / Line 84: @@
 # Update the functionality of the Java engine. Port a CG3 engine. Port (or adapt existing) HFST engine.
 ## [[User:Francis Tyers|ftyers]]:  Disagree with this one. Perhaps I would rephrase it "get pipeline for C++ to Android" or "get pipeline for generating from C++ to usage in OmegaT". I would like to add a section on integration/interoperability with other language data. Currently the UD project has freely-licensed annotated corpora for a good many languages in Apertium. Using them in Apertium is problematic because of different tokenisation standards and different tagsets. There should be some effort to make these more interoperable so that we can avoid duplicating effort (having people annotate English text).
+## [[User:Xavivars|Xavivars]] ([[User talk:Xavivars|talk]]) +1
 # [[User:Francis Tyers|ftyers]]: '''MEETING PROPOSAL''': If many people are going to be in Dublin this year for the [https://www.mtsummit2019.com/ MT Summit], how about having an Apertium meeting/workshop where we flesh this stuff out and try and gain some direction? It won't be in time for GSOC this year, but it's definitely nearly ten years since we had a real planning meeting.
 ## --[[User:Mlforcada|Mlforcada]] ([[User talk:Mlforcada|talk]]) 11:32, 26 January 2019 (CET) This is really a cool idea. Let me see what we can do.

@@ Line 18: / Line 18: @@
 We have some interesting data collected in our wiki, but I think we need to evaluate impact better. We would need a more complete survey about where Apertium is used (in research, commercially, etc.). I know GSoC does not pay for this, but we do have some money in our kitty (€35,000 or more).
+=== Languages ===
-'''Languages'''. I believe we need to be more specific, particularly about languages. Our mission does not say much about where to concentrate our effort. Let me give some ideas about different kinds of languages (the list is not exhaustive).
+I believe we need to be more specific, particularly about languages. Our mission does not say much about where to concentrate our effort. Let me give some ideas about different kinds of languages (the list is not exhaustive).
 # Languages that do not have (other) machine translation, free/open-source or otherwise, and there is little chance that they will be tackled with the new neural approach, as the parallel corpora available are too small. Nice examples are Occitan, Sardinian or Breton. We do have some Apertium for these languages, but they are different.
@@ Line 31: / Line 32: @@
+=== Engine ===
-'''Engine''': I will not be too specific here as I am not that familiar and I may be wrong. For instance we could
+I will not be too specific here as I am not that familiar and I may be wrong. For instance we could
 # improve the Java engine and, consequently, the Android app (remember it can be used offline, as in the [https://translatorswithoutborders.org/translators-without-borders-develops-worlds-first-crisis-specific-machine-translation-system-kurdish-languages/ Kurdish language developed by Translators without Borders]) and the [http://wiki.apertium.org/wiki/Apertium-OmegaT OmegaT plugin]
 ## [[User:Francis Tyers|ftyers]]: Here I disagree. I think that having a Java port is a dead end. It is inevitable that it will get out of date very quickly. I think that a better strategy is to (sentence incomplete in Fran's message).
@@ Line 45: / Line 48: @@
 ### An interesting research idea that people have discussed with me would   be to have Apertium be able to generate data for neural MT, but with a   more expressive formalism. We have done some work on this already, but not with a very generative system. I have some grant ideas about that,   e.g. from field linguistics -> NMT, if can expound on them if people are interested.
+=== Evaluation ===
-'''Evaluation''': we have some [http://wiki.apertium.org/wiki/Evaluation automatic evaluation] (word-error rate, naïve coverage) in our wiki but we should select some language pairs for evaluation. Gap-filling (see [https://arxiv.org/abs/1809.00315 this recent paper] and references therein) could be used when we expect a langauge pair to be used for gisting. For language pairs which may be used for dissemination, we could pay professional post-editors. The Universitat d'Alacant has experience on this (we had four people post-editing for a week, and we spent about €3000). Tools like [https://github.com/ghpaetzold/PET PET] could be used. But there are also some unexplored usages of Apertium such as [https://github.com/transducens/Forecat-OmegaT  interactive machine translation] (see also [https://ufal.mff.cuni.cz/pbml/102/art-torregrosa-forcada-perez-ortiz.pdf this paper]).
+We have some [http://wiki.apertium.org/wiki/Evaluation automatic evaluation] (word-error rate, naïve coverage) in our wiki but we should select some language pairs for evaluation. Gap-filling (see [https://arxiv.org/abs/1809.00315 this recent paper] and references therein) could be used when we expect a langauge pair to be used for gisting. For language pairs which may be used for dissemination, we could pay professional post-editors. The Universitat d'Alacant has experience on this (we had four people post-editing for a week, and we spent about €3000). Tools like [https://github.com/ghpaetzold/PET PET] could be used. But there are also some unexplored usages of Apertium such as [https://github.com/transducens/Forecat-OmegaT  interactive machine translation] (see also [https://ufal.mff.cuni.cz/pbml/102/art-torregrosa-forcada-perez-ortiz.pdf this paper]).
 Things we can spend our money on:
@@ Line 58: / Line 63: @@
 ### Classification of translation errors by module
+=== GSoC proposals ===
-'''GSoC proposals'''. Very specific proposals that should be part of GSoC 2019 if we can find interested people ("Mikel's list"), to be fleshed out.
+Very specific proposals that should be part of GSoC 2019 if we can find interested people ("Mikel's list"), to be fleshed out.
 # Improve the quality of Occitan pairs. Contact Occitanists so that they get involved and spread the word.

← Older revision		Revision as of 10:40, 26 January 2019
Line 69:		Line 69:


	# [[User:Francis Tyers\|ftyers]]: MEETING PROPOSAL: If many people are going to be in Dublin this year for the [https://www.mtsummit2019.com/ MT Summit], how about having an Apertium meeting/workshop where we flesh this stuff out and try and gain some direction? It won't be in time for GSOC this year, but it's definitely nearly ten years since we had a real planning meeting.		# [[User:Francis Tyers\|ftyers]]: '''MEETING PROPOSAL''': If many people are going to be in Dublin this year for the [https://www.mtsummit2019.com/ MT Summit], how about having an Apertium meeting/workshop where we flesh this stuff out and try and gain some direction? It won't be in time for GSOC this year, but it's definitely nearly ten years since we had a real planning meeting.
	## --[[User:Mlforcada\|Mlforcada]] ([[User talk:Mlforcada\|talk]]) 11:32, 26 January 2019 (CET) This is really a cool idea. Let me see what we can do.		## --[[User:Mlforcada\|Mlforcada]] ([[User talk:Mlforcada\|talk]]) 11:32, 26 January 2019 (CET) This is really a cool idea. Let me see what we can do.

@@ Line 28: / Line 28: @@
 ## [[User:Francis Tyers|ftyers]]: Tentatively agree, but we should do a careful study of where the best impact can be found before doing e.g. Swahili--English.
 # Languages that may be well supported by more than one MT vendor and have good or very good Apertium engines: Catalan–Spanish, Portuguese–Spanish, French–Spanish, etc. Their impact is large because they are widely used. Improving their quality may increase their impact and give Apertium some prestige.
-## [[User:Francis Tyers|ftyers]]: Agree, some of these don't even use the latest code from -lex-tools and -separable, and a substantial amount of work could be done to
+## [[User:Francis Tyers|ftyers]]: Agree, some of these don't even use the latest code from -lex-tools and -separable, and a substantial amount of work could be done to improve them based on these. Out of the three: Portuguese-Spanish = coverage is the main problem as far as I can tell, we have 15k entries, we need 50k or so, e.g. >95% coverage instead of >90%; French--Spanish = needs more rules, and much better disambiguation.
-improve them based on these. Out of the three: Portuguese-Spanish = coverage is the main problem as far as I can tell, we have 15k entries, we need 50k or so, e.g. >95% coverage instead of >90%; French--Spanish = needs more rules, and much better disambiguation.

@@ Line 72: / Line 72: @@
-# [[User:Francis Tyers|ftyers]]: *MEETING PROPOSAL*: If many people are going to be in Dublin this year for MT
+# [[User:Francis Tyers|ftyers]]: *MEETING PROPOSAL*: If many people are going to be in Dublin this year for the [https://www.mtsummit2019.com/ MT Summit], how about having an Apertium meeting/workshop where we flesh this stuff out and try and gain some direction? It won't be in time for GSOC this year, but it's definitely nearly ten years since we had a real planning meeting.
+## --[[User:Mlforcada|Mlforcada]] ([[User talk:Mlforcada|talk]]) 11:32, 26 January 2019 (CET) This is really a cool idea. Let me see what we can do.
-Summit, how about having an Apertium meeting/workshop where we flesh this stuff out and try and gain some direction? It won't be in time for GSOC this year, but it's definitely nearly ten years since we had a real planning meeting.
-## --[[User:Mlforcada|Mlforcada]] ([[User talk:Mlforcada|talk]]) 11:32, 26 January 2019 (CET) This is really a cool idea.