Difference between revisions of "Task ideas for Google Code-in"

From Apertium
Jump to navigation Jump to search
Line 318: Line 318:
 
|description=Write a script that matches bilingual dictionary entries (in dix format) to monolingual dictionary entries in one of the languages (in [[Apertium-specific conventions for lexc|lexc]] format) and adds glosses from the other side of the bilingual dictionary if not already there. The script should combine glosses into one when there's more than one in the bilingual dictionary. Some level of user control might be justified, from simply defaulting to a dry run unless otherwise specified, to controls for adding to versus replacing versus leaving alone existing glosses, and the like. A [http://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/inject-words-from-bidix-to-lexc.py prototype of this script] is available in SVN, though it's buggy and doesn't fully work—so this task may just end up being to debug it and make it work as intended. A good test case might be the [http://svn.code.sf.net/p/apertium/svn/trunk/apertium-eng-kaz/apertium-eng-kaz.eng-kaz.dix English-Kazakh bilingual dictionary] and the [http://svn.code.sf.net/p/apertium/svn/languages/apertium-kaz/apertium-kaz.kaz.lexc Kazakh monolingual dictionary].
 
|description=Write a script that matches bilingual dictionary entries (in dix format) to monolingual dictionary entries in one of the languages (in [[Apertium-specific conventions for lexc|lexc]] format) and adds glosses from the other side of the bilingual dictionary if not already there. The script should combine glosses into one when there's more than one in the bilingual dictionary. Some level of user control might be justified, from simply defaulting to a dry run unless otherwise specified, to controls for adding to versus replacing versus leaving alone existing glosses, and the like. A [http://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/inject-words-from-bidix-to-lexc.py prototype of this script] is available in SVN, though it's buggy and doesn't fully work—so this task may just end up being to debug it and make it work as intended. A good test case might be the [http://svn.code.sf.net/p/apertium/svn/trunk/apertium-eng-kaz/apertium-eng-kaz.eng-kaz.dix English-Kazakh bilingual dictionary] and the [http://svn.code.sf.net/p/apertium/svn/languages/apertium-kaz/apertium-kaz.kaz.lexc Kazakh monolingual dictionary].
 
|tags=python, lexc, dix, xml
 
|tags=python, lexc, dix, xml
  +
|mentors=Jonathan
  +
}}{{Taskidea
  +
|type=code
  +
|title=Write a script to deduplicate and/or sort individual lexc lexica.
  +
|description=The lexc format is a way to specify a monolingual dictionary that gets compiled into a transducer: see [[Apertium-specific conventions for lexc]] and [[Lttoolbox and lexc#lexc]]. A single lexc file may contain quite a few individual lexicons of stems, e.g. for nouns, verbs, prepositions, etc. Write a script (in python or ruby) that reads a specified lexicon, and based on which option the user specifies, identifies and removes duplicates from the lexicon, and/or sorts the entries in the lexicon. Be sure to make a dry-run (i.e., do not actually make the changes) the default, and add different levels debugging (such as displaying a number of duplicates versus printing each duplicate). Also consider allowing for different criteria for matching duplicates: e.g., whether or not the comment matches too. There are two scripts that parse lexc files already that would be a good point to start from: [http://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/lexccounter.py lexccounter.py] and [http://svn.code.sf.net/p/apertium/svn/trunk/apertium-tools/inject-words-from-bidix-to-lexc.py inject-words-from-bidix-to-lexc.py] (not fully functional).
  +
|tags=python, ruby, lexc
 
|mentors=Jonathan
 
|mentors=Jonathan
 
}}
 
}}

Revision as of 18:21, 15 October 2017

Contents

This is the task ideas page for Google Code-in, here you can find ideas on interesting tasks that will improve your knowledge of Apertium and help you get into the world of open-source development.

The people column lists people who you should get in contact with to request further information. All tasks are 2 hours maximum estimated amount of time that would be spent on the task by an experienced developer, however:

  1. this does not include time taken to install / set up apertium (and relevant tools).
  2. this is the time expected to take by an experienced developer, you may find that you spend more time on the task because of the learning curve.

Categories:

  • code: Tasks related to writing or refactoring code
  • documentation: Tasks related to creating/editing documents and helping others learn more
  • research: Tasks related to community management, outreach/marketting, or studying problems and recommending solutions
  • quality: Tasks related to testing and ensuring code is of high quality.
  • interface: Tasks related to user experience research or user interface design and interaction

Clarification of "multiple task" types

  • multi = number of students who can do a given task
  • dup = number of times a student can do the same task

You can find descriptions of some of the mentors here.

Task ideas

typetitledescriptiontagsmentorsbgnr?multi?duplicates
interface Nicely laid out interface for ud-annotatrix Design an HTML layout for the annotatrix tool that makes best use of the space and functions nicely at different screen resolutions. annotation, annotatrix Fran, Masha, Jonathan
interface Come up with a CSS style for annotatrix annotation, annotatrix, css Fran, Masha, Jonathan
code SDparse to CoNLL-U converter in JavaScript SDparse is a format for describing dependency trees, they look like relation(head, dependency). CoNLL-U is another format for describing dependency trees. Make a converter between the two formats. You will probably need to learn more about the specifics of these formats. The GitHub issue is here. annotation, annotatrix, javascript Fran, Masha, Jonathan
quality Write a test for the format converters in annotatrix annotation, annotatrix Fran, Masha yes
code Write a function to detect invalid trees in the UD annotatrix software and advise the user about it It is possible to detect invalid trees (such as those that have cycles). We would like to write a function to detect those kinds of trees and advise the user. The GitHub issue is here. annotation, annotatrix, javascript Fran, Masha, Jonathan
documentation Write a tutorial on how to use annotatrix to annotate a dependency tree Give step by step instructions to annotating a dependency tree with Annotatrix. Make sure you include all possibilities in the app, for example tokenisation options. annotation, annotatrix Fran, Masha, Jonathan
documentation Make a video tutorial on annotating a dependency tree using the UD annotatrix software. Give step by step instructions to annotating a dependency tree with Annotatrix. Make sure you include all possibilities available in the app, for example tokenisation options. annotation, annotatrix, video Fran, Masha
quality Merge two versions of the Polish morphological dictionary At some point in the past, someone deleted a lot of entries from the Polish morphological dictionary, and unfortunately we didn't notice at the time and have since added stuff to it. The objective of this task is to take the last

version before the mass deletion and the current version and merge them. Getting list of the changes:

$ svn diff --old apertium-pol.pol.dix@73196 --new apertium-pol.pol.dix@73199 > changes.diff
xml, dictionaries, svn Masha
quality Add 200 new entries to a bidix to language pair %AAA%-%BBB% Our translation systems require large lexicons so as to provide production-quality coverage of any input data. This task requires the student to add 500 new words to a bidirectional dictionary. xml, dictionaries, svn fotonzade, Jonathan yes yes
quality Add 500 new entries to a bidix to language pair %AAA%-%BBB% Our translation systems require large lexicons so as to provide production-quality coverage of any input data. This task requires the student to add 500 new words to a bidirectional dictionary. xml, dictionaries, svn fotonzade, Jonathan yes
quality Disambiguate 500 tokens of text in %AAA% Run some text through a morphological analyser and disambiguate the output. Contact the mentor beforehand to approve the choice of language and text. disambiguation, svn fotonzade yes
code Use apertium-init to start a new morphological analyser for %AAA% Use apertium-init to start a new morphological analyser (for a language we don't already have, e.g. %AAA%) and add 100 words. morphology, languages, finite-state, fst Fran, Katya yes
documentation add comments to .dix file symbol definitions dix Jonathan
documentation find symbols that aren't on the list of symbols page Go through symbol definitions in Apertium dictionaries in svn (.lexc and .dix format), and document any symbols you don't find on the List of symbols page. This task is fulfilled by adding at least one class of related symbols (e.g., xyz_*) or one major symbol (e.g., abc), along with notes about what it means. wiki,lexc,dix Jonathan
code conllu parser and searching Write a script (preferably in python3) that will parse files in conllu format, and perform basic searches, such as "find a node that has an nsubj relation to another node that has a noun POS" or "find all nodes with a cop label and a past feature" python,dependencies Jonathan, Fran, Wei En
code group and count possible lemmas output by guesser Currently a "guesser" version of Apertium transducers can output a list of possible analyses for unknown forms. Develop a new pipleine, preferably with shell scripts or python, that uses a guesser on all unknown forms in a corpus, and takes the list of all possible analyses, and output a hit count of the most common combinations of lemma and POS tag. guesser, transducers, shellscripts Jonathan, Fran, Wei En
code vim mode/tools for annotating dependency corpora in CG3 format includes formatting, syntax highlighting, navigation, adding/removing nodes, updating node numbers, etc. vim, dependencies, CG3 Jonathan, Fran
code vim mode/tools for annotating dependency corpora in CoNLL-U format includes formatting, syntax highlighting, navigation, adding/removing nodes, updating node numbers, etc. vim, dependencies, conllu Jonathan, Fran
quality figure out one-to-many bug in the lsx module There is a bug in the lsx module referred to as the one-to-many bug because lsx-proc will not convert one form to many given an appropriately compiled transducer. Your job is to figure out why this happens and fix it. C++, transducers, lsx Jonathan, Fran, Wei En, Irene
code add an option for reverse compiling to the lsx module this should be simple as it can just leverage the existing lttoolbox options for left-right / right-left compiling C++, transducers, lsx Jonathan, Fran, Wei En, Irene
quality remove extraneous functions from lsx-comp and clean up the code C++, transducers, lsx Jonathan, Fran, Wei En, Irene
quality remove extraneous functions from lsx-proc and clean up the code C++, transducers, lsx Jonathan, Fran, Wei En, Irene
code script to test coverage over wikipedia corpus Write a script (in python or ruby) that in one mode checks out a specified language module to a given directory, compiles it (or updates it if already existant), and then gets the most recently nightly wikipedia archive for that language and runs coverage over it (as much in RAM if possible). In another mode, it compiles the language pair in a docker instance that it then disposes of after successfully running coverage. Scripts exist in Apertium already for finding where a wikipedia is, extracting a wikipedia archive into a text file, and running coverage. python, ruby, wikipedia Jonathan, Wei En
quality,code fix any open ticket Fix any open ticket in any of our issues trackers: main, html-tools, begiak. When you claim this task, let your mentor know which issue you plan to work on. Jonathan, Wei En, Sushain 25 10
quality,code make html-tools do better on Chrome's audit Currently, apertium.org and generally any html-tools installation fails lots of Chrome audit tests. As many as possible should be fixed. Ones that require substantial work should be filed as tickets and measures should be taken to prevent problems from reappearing (e.g. a test or linter rule). More information is available in the issue tracker (#201) and asynchronous discussion should occur there. javascript, html, css, web Jonathan, Sushain
code,interface upgrade html-tools to Bootstrap 4 Currently, html-tools uses Bootstrap 3.x. Bootstrap 4 beta is out and we can upgrade (hopefully)! If an upgrade is not possible, you should document why it's not and ensure that it's easy to upgrade when the blockers are removed. More information may be available in the issue tracker (#200) and asynchronous discussion should occur there. javascript, html, css, web Sushain yes
code,interface display API endpoint on sandbox Currently, html-tools has an "APy" mode where users can easily test out the API. However, it doesn't display the actual URL of the API endpoint and it would be nice to show that to the user. More information is available in the issue tracker (#147) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan yes
code,quality,research set up a testing framework for html-tools Currently, html-tools has no tests (sad!). This task requires researching what solutions there are for testing jQuery based web applications and putting one into place with a couple tests as a proof of concept. More information is available in the issue tracker (#116) and asynchronous discussion should occur there. javascript, html, css, web Sushain
code,research make html-tools automatically download translated files in Safari, IE, etc. Currently, html-tools is capable of translating files. However, this translation does not always result in the file immediately being download to the user on all browsers. It would be awesome if it did! This task requires researching what solutions there are, evaluating them against each other and it may result in a conclusion that it just isn't possible (yet). More information is available in the issue tracker (#97) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Unhammer
code,interface make html-tools fail more gracefully when API is down Currently, html-tools relies on an API endpoint to translate documents, files, etc. However, when this API is down the interface also breaks! This task requires fixing this breakage. More information is available in the issue tracker (#207) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan yes
code,interface make html-tools properly align text in mixed RTL/LTR contexts Currently, html-tools is capable of displaying results/allowing input for RTL languages in a LTR context (e.g. we're translating Arabic in an English website). However, this doesn't always look exactly how it should look, i.e. things are not aligned correctly. More information is available in the issue tracker (#49) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan yes
code,interface de-conflict the 'make a suggestion' interface in html-tools There has been much demand for html-tools to support an interface for users making suggestions regarding e.g. incorrect translations (c.f. Google translate). An interface was designed for this purpose. However, since it has been a while since anyone touched it, the code now conflicts with the current master branch. This task requires de-conflicting this branch with master and providing screenshot/video(s) of the interface to show that it functions. More information is available in the issue tracker (#74) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan
code,quality make html-tools capable of translating itself Currently, html-tools supports website translation. However, if asked to translate itself, weird things happen and the interface does not properly load. This task requires figuring out the root problem and correcting the fault. More information is available in the issue tracker (#203) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan yes
interface create mock-ups for variant support in html-tools Currently, html-tools supports translation using language variants. However, we do not have first-class style/interface support for it. This task requires speaking with mentors/reading existing discussion to understand the problem and then produce design mockups for a solution. More information is available in the issue tracker (#82) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Fran
code,interface refine the html-tools dictionary interface Significant progress has been made towards providing a dictionary-style interface within html-tools. This task requires refining the existing PR by de-conflicting it with master and resolving the interface concerns discussed here. More information is available in the issue tracker (#105) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan
code,quality,interface eliminate inline styles from html-tools Currently, html-tools has inline styles. These are not very maintainable and widely considered as bad style. This task requires surveying the uses, removing all of them in a clean manner, i.e. semantically, and re-enabling the linter rule that will prevent them going forward. More information is available in the issue tracker (#114) and asynchronous discussion should occur there. html, css, web Sushain yes
code,interface refine the html-tools spell checking interface Spell checking is a feature that would greatly benefit html-tools. Significant effort has been put towards implementing an effective interface to provide spelling suggestions to users (this PR contains the current progress). This task requires solving the problems highlighted in the code review on the PR and fixing any other bugs uncovered in conversations with the mentors. More information is available in the issue tracker (#12) and asynchronous discussion should occur there. html, css, web Sushain, Jonathan
quality find an apertium module not developed in svn and import it Find an Apertium module developed elsewhere (e.g., github) released under a compatible open license, and import it into Apertium's svn, being sure to attribute any authors (in an AUTHORS file) and keeping the original license. Once place to look for such modules might be among the final projects in a recent Computational Linguistics course. Jonathan, Wei En 10 2
code add an incubator mode to the wikipedia scraper Add a mode to scrape a Wikipedia in incubator (e.g,. the Ingush incubator) to the WikiExtractor script wikipedia, python Jonathan, Wei En
code,interface add a translation mode interface to the geriaoueg plugin for firefox Fork the geriaoueg firefox plugin and add an interface for translation mode. It doesn't have to translate at this point, but it should communicate with the server (as it currently does) to load available languages. javascript Jonathan
code, interface add a translation mode interface to the geriaoueg plugin for chrome Fork the geriaoueg chrome plugin and add an interface for translation mode. It doesn't have to translate at this point, but it should communicate with the server (as it currently does) to load available languages. javascript Jonathan
quality update bidix included in apertium-init There are some issues with the bidix currently included in apertium-init: the alphabet should be empty (or non-existant?) and the "sg" tags shouldn't be in the example entries. It would also be good to have entries in two different languages, especially ones with incompatible POS sub-categories (e.g. casa<n><f>). There is a github issue for this task. python, xml, dix Jonathan, Sushain yes
code apertium-init support for more features in hfst modules Add optional support to hfst modules for enabling spelling modules, an extra twoc module for morphotactic constraints, and spellrelax. You'll want to figure out how to integrate this into the Makefile template. There is a github issue for this task. python, xml, Makefile Jonathan
code, quality make apertium-init README files show only relevant dictionary file Currently in apertium-init, the README files for HFST modules show the "dix" file in the list of files, and it's likely that lttoolbox modules show "hfst" files in their README too. Check this and make it so that READMEs for these two types of monolingual modules display only the right dictionary files. There is a github issue for this task. python, xml, Makefile Jonathan, Sushain
code, quality Write a script to add glosses to a monolingual dictionary from a bilingual dictionary Write a script that matches bilingual dictionary entries (in dix format) to monolingual dictionary entries in one of the languages (in lexc format) and adds glosses from the other side of the bilingual dictionary if not already there. The script should combine glosses into one when there's more than one in the bilingual dictionary. Some level of user control might be justified, from simply defaulting to a dry run unless otherwise specified, to controls for adding to versus replacing versus leaving alone existing glosses, and the like. A prototype of this script is available in SVN, though it's buggy and doesn't fully work—so this task may just end up being to debug it and make it work as intended. A good test case might be the English-Kazakh bilingual dictionary and the Kazakh monolingual dictionary. python, lexc, dix, xml Jonathan
code Write a script to deduplicate and/or sort individual lexc lexica. The lexc format is a way to specify a monolingual dictionary that gets compiled into a transducer: see Apertium-specific conventions for lexc and Lttoolbox and lexc#lexc. A single lexc file may contain quite a few individual lexicons of stems, e.g. for nouns, verbs, prepositions, etc. Write a script (in python or ruby) that reads a specified lexicon, and based on which option the user specifies, identifies and removes duplicates from the lexicon, and/or sorts the entries in the lexicon. Be sure to make a dry-run (i.e., do not actually make the changes) the default, and add different levels debugging (such as displaying a number of duplicates versus printing each duplicate). Also consider allowing for different criteria for matching duplicates: e.g., whether or not the comment matches too. There are two scripts that parse lexc files already that would be a good point to start from: lexccounter.py and inject-words-from-bidix-to-lexc.py (not fully functional). python, ruby, lexc Jonathan