Apertium has moved from SourceForge to GitHub.
If you have any questions, please come and talk to us on #apertium on irc.freenode.net or contact the GitHub migration team.

Task ideas for Google Code-in

From Apertium
Revision as of 03:17, 16 October 2017 by Shardulc (Talk | contribs)

Jump to: navigation, search

Contents

This is the task ideas page for Google Code-in, here you can find ideas on interesting tasks that will improve your knowledge of Apertium and help you get into the world of open-source development.

The people column lists people who you should get in contact with to request further information. All tasks are 2 hours maximum estimated amount of time that would be spent on the task by an experienced developer, however:

  1. this does not include time taken to install / set up apertium (and relevant tools).
  2. this is the time expected to take by an experienced developer, you may find that you spend more time on the task because of the learning curve.

Categories:

  • code: Tasks related to writing or refactoring code
  • documentation: Tasks related to creating/editing documents and helping others learn more
  • research: Tasks related to community management, outreach/marketting, or studying problems and recommending solutions
  • quality: Tasks related to testing and ensuring code is of high quality.
  • interface: Tasks related to user experience research or user interface design and interaction

Clarification of "multiple task" types

  • multi = number of students who can do a given task
  • dup = number of times a student can do the same task

You can find descriptions of some of the mentors here.

Task ideas

typetitledescriptiontagsmentorsbgnr?multi?duplicates
research Document resources for a language Document resources for a language without resources already documented on the Apertium wiki. read more... Jonathan, Vin yes 40
research Write a contrastive grammar Document 6 differences between two (preferably related) languages and where they would need to be addressed in the Apertium pipeline (morph analysis, transfer, etc). Use a grammar book/resource for inspiration. Each difference should have no fewer than 3 examples. Put your work on the Apertium wiki under Language1_and_Language2/Contrastive_grammar. See Farsi_and_English/Pending_tests for an example of a contrastive grammar that a previous GCI student made. Vin, Jonathan, Fran yes 40
interface Nicely laid out interface for ud-annotatrix Design an HTML layout for the annotatrix tool that makes best use of the space and functions nicely at different screen resolutions. annotation, annotatrix Fran, Masha, Jonathan
interface Come up with a CSS style for annotatrix annotation, annotatrix, css Fran, Masha, Jonathan, Vin
code SDparse to CoNLL-U converter in JavaScript SDparse is a format for describing dependency trees, they look like relation(head, dependency). CoNLL-U is another format for describing dependency trees. Make a converter between the two formats. You will probably need to learn more about the specifics of these formats. The GitHub issue is here. annotation, annotatrix, javascript Fran, Masha, Jonathan, Vin
quality Write a test for the format converters in annotatrix annotation, annotatrix Fran, Masha yes
code Write a function to detect invalid trees in the UD annotatrix software and advise the user about it It is possible to detect invalid trees (such as those that have cycles). We would like to write a function to detect those kinds of trees and advise the user. The GitHub issue is here. annotation, annotatrix, javascript Fran, Masha, Jonathan
documentation Write a tutorial on how to use annotatrix to annotate a dependency tree Give step by step instructions to annotating a dependency tree with Annotatrix. Make sure you include all possibilities in the app, for example tokenisation options. annotation, annotatrix Fran, Masha, Jonathan
documentation Make a video tutorial on annotating a dependency tree using the UD annotatrix software. Give step by step instructions to annotating a dependency tree with Annotatrix. Make sure you include all possibilities available in the app, for example tokenisation options. annotation, annotatrix, video Fran, Masha
quality Merge two versions of the Polish morphological dictionary At some point in the past, someone deleted a lot of entries from the Polish morphological dictionary, and unfortunately we didn't notice at the time and have since added stuff to it. The objective of this task is to take the last

version before the mass deletion and the current version and merge them. Getting list of the changes:

$ svn diff --old apertium-pol.pol.dix@73196 --new apertium-pol.pol.dix@73199 > changes.diff
xml, dictionaries, svn Masha
quality Add 200 new entries to a bidix to language pair %AAA%-%BBB% Our translation systems require large lexicons so as to provide production-quality coverage of any input data. This task requires the student to add 500 new words to a bidirectional dictionary. xml, dictionaries, svn fotonzade, Jonathan yes yes
quality Add 500 new entries to a bidix to language pair %AAA%-%BBB% Our translation systems require large lexicons so as to provide production-quality coverage of any input data. This task requires the student to add 500 new words to a bidirectional dictionary. xml, dictionaries, svn fotonzade, Jonathan yes
quality Disambiguate 500 tokens of text in %AAA% Run some text through a morphological analyser and disambiguate the output. Contact the mentor beforehand to approve the choice of language and text. disambiguation, svn fotonzade yes
code Use apertium-init to start a new morphological analyser for %AAA% Use apertium-init to start a new morphological analyser (for a language we don't already have, e.g. %AAA%) and add 100 words. morphology, languages, finite-state, fst Fran, Katya yes
documentation add comments to .dix file symbol definitions dix Jonathan
documentation find symbols that aren't on the list of symbols page Go through symbol definitions in Apertium dictionaries in svn (.lexc and .dix format), and document any symbols you don't find on the List of symbols page. This task is fulfilled by adding at least one class of related symbols (e.g., xyz_*) or one major symbol (e.g., abc), along with notes about what it means. wiki,lexc,dix Jonathan
code conllu parser and searching Write a script (preferably in python3) that will parse files in conllu format, and perform basic searches, such as "find a node that has an nsubj relation to another node that has a noun POS" or "find all nodes with a cop label and a past feature" python,dependencies Jonathan, Fran, Wei En
code group and count possible lemmas output by guesser Currently a "guesser" version of Apertium transducers can output a list of possible analyses for unknown forms. Develop a new pipleine, preferably with shell scripts or python, that uses a guesser on all unknown forms in a corpus, and takes the list of all possible analyses, and output a hit count of the most common combinations of lemma and POS tag. guesser, transducers, shellscripts Jonathan, Fran, Wei En
code vim mode/tools for annotating dependency corpora in CG3 format includes formatting, syntax highlighting, navigation, adding/removing nodes, updating node numbers, etc. vim, dependencies, CG3 Jonathan, Fran
code vim mode/tools for annotating dependency corpora in CoNLL-U format includes formatting, syntax highlighting, navigation, adding/removing nodes, updating node numbers, etc. vim, dependencies, conllu Jonathan, Fran
quality figure out one-to-many bug in the lsx module There is a bug in the lsx module referred to as the one-to-many bug because lsx-proc will not convert one form to many given an appropriately compiled transducer. Your job is to figure out why this happens and fix it. C++, transducers, lsx Jonathan, Fran, Wei En, Irene
code add an option for reverse compiling to the lsx module this should be simple as it can just leverage the existing lttoolbox options for left-right / right-left compiling C++, transducers, lsx Jonathan, Fran, Wei En, Irene
quality remove extraneous functions from lsx-comp and clean up the code C++, transducers, lsx Jonathan, Fran, Wei En, Irene
quality remove extraneous functions from lsx-proc and clean up the code C++, transducers, lsx Jonathan, Fran, Wei En, Irene
code script to test coverage over wikipedia corpus Write a script (in python or ruby) that in one mode checks out a specified language module to a given directory, compiles it (or updates it if already existant), and then gets the most recently nightly wikipedia archive for that language and runs coverage over it (as much in RAM if possible). In another mode, it compiles the language pair in a docker instance that it then disposes of after successfully running coverage. Scripts exist in Apertium already for finding where a wikipedia is, extracting a wikipedia archive into a text file, and running coverage. python, ruby, wikipedia Jonathan, Wei En, Shardul
quality,code fix any open ticket Fix any open ticket in any of our issues trackers: main, html-tools, begiak. When you claim this task, let your mentor know which issue you plan to work on. Jonathan, Wei En, Sushain, Shardul 25 10
quality,code make html-tools do better on Chrome's audit Currently, apertium.org and generally any html-tools installation fails lots of Chrome audit tests. As many as possible should be fixed. Ones that require substantial work should be filed as tickets and measures should be taken to prevent problems from reappearing (e.g. a test or linter rule). More information is available in the issue tracker (#201) and asynchronous discussion should occur there. javascript, html, css, web Jonathan, Sushain, Shardul
code,interface upgrade html-tools to Bootstrap 4 Currently, html-tools uses Bootstrap 3.x. Bootstrap 4 beta is out and we can upgrade (hopefully)! If an upgrade is not possible, you should document why it's not and ensure that it's easy to upgrade when the blockers are removed. More information may be available in the issue tracker (#200) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Shardul yes
code,interface display API endpoint on sandbox Currently, html-tools has an "APy" mode where users can easily test out the API. However, it doesn't display the actual URL of the API endpoint and it would be nice to show that to the user. More information is available in the issue tracker (#147) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Shardul yes
code,quality,research set up a testing framework for html-tools Currently, html-tools has no tests (sad!). This task requires researching what solutions there are for testing jQuery based web applications and putting one into place with a couple tests as a proof of concept. More information is available in the issue tracker (#116) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Shardul
code,research make html-tools automatically download translated files in Safari, IE, etc. Currently, html-tools is capable of translating files. However, this translation does not always result in the file immediately being download to the user on all browsers. It would be awesome if it did! This task requires researching what solutions there are, evaluating them against each other and it may result in a conclusion that it just isn't possible (yet). More information is available in the issue tracker (#97) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Unhammer, Shardul
code,interface make html-tools fail more gracefully when API is down Currently, html-tools relies on an API endpoint to translate documents, files, etc. However, when this API is down the interface also breaks! This task requires fixing this breakage. More information is available in the issue tracker (#207) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Shardul yes
code,interface make html-tools properly align text in mixed RTL/LTR contexts Currently, html-tools is capable of displaying results/allowing input for RTL languages in a LTR context (e.g. we're translating Arabic in an English website). However, this doesn't always look exactly how it should look, i.e. things are not aligned correctly. More information is available in the issue tracker (#49) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Shardul yes
code,interface de-conflict the 'make a suggestion' interface in html-tools There has been much demand for html-tools to support an interface for users making suggestions regarding e.g. incorrect translations (c.f. Google translate). An interface was designed for this purpose. However, since it has been a while since anyone touched it, the code now conflicts with the current master branch. This task requires de-conflicting this branch with master and providing screenshot/video(s) of the interface to show that it functions. More information is available in the issue tracker (#74) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Shardul
code,quality make html-tools capable of translating itself Currently, html-tools supports website translation. However, if asked to translate itself, weird things happen and the interface does not properly load. This task requires figuring out the root problem and correcting the fault. More information is available in the issue tracker (#203) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Shardul yes
interface create mock-ups for variant support in html-tools Currently, html-tools supports translation using language variants. However, we do not have first-class style/interface support for it. This task requires speaking with mentors/reading existing discussion to understand the problem and then produce design mockups for a solution. More information is available in the issue tracker (#82) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan, Fran, Shardul
code,interface refine the html-tools dictionary interface Significant progress has been made towards providing a dictionary-style interface within html-tools. This task requires refining the existing PR by de-conflicting it with master and resolving the interface concerns discussed here. More information is available in the issue tracker (#105) and asynchronous discussion should occur there. javascript, html, css, web Sushain, Jonathan
code,quality,interface eliminate inline styles from html-tools Currently, html-tools has inline styles. These are not very maintainable and widely considered as bad style. This task requires surveying the uses, removing all of them in a clean manner, i.e. semantically, and re-enabling the linter rule that will prevent them going forward. More information is available in the issue tracker (#114) and asynchronous discussion should occur there. html, css, web Sushain, Shardul yes
code,interface refine the html-tools spell checking interface Spell checking is a feature that would greatly benefit html-tools. Significant effort has been put towards implementing an effective interface to provide spelling suggestions to users (this PR contains the current progress). This task requires solving the problems highlighted in the code review on the PR and fixing any other bugs uncovered in conversations with the mentors. More information is available in the issue tracker (#12) and asynchronous discussion should occur there. html, css, web Sushain, Jonathan
quality find an apertium module not developed in svn and import it Find an Apertium module developed elsewhere (e.g., github) released under a compatible open license, and import it into Apertium's svn, being sure to attribute any authors (in an AUTHORS file) and keeping the original license. Once place to look for such modules might be among the final projects in a recent Computational Linguistics course. Jonathan, Wei En 10 2
code add an incubator mode to the wikipedia scraper Add a mode to scrape a Wikipedia in incubator (e.g,. the Ingush incubator) to the WikiExtractor script wikipedia, python Jonathan, Wei En
code,interface add a translation mode interface to the geriaoueg plugin for firefox Fork the geriaoueg firefox plugin and add an interface for translation mode. It doesn't have to translate at this point, but it should communicate with the server (as it currently does) to load available languages. javascript Jonathan
code, interface add a translation mode interface to the geriaoueg plugin for chrome Fork the geriaoueg chrome plugin and add an interface for translation mode. It doesn't have to translate at this point, but it should communicate with the server (as it currently does) to load available languages. javascript Jonathan
quality update bidix included in apertium-init There are some issues with the bidix currently included in apertium-init: the alphabet should be empty (or non-existant?) and the "sg" tags shouldn't be in the example entries. It would also be good to have entries in two different languages, especially ones with incompatible POS sub-categories (e.g. casa<n><f>). There is a github issue for this task. python, xml, dix Jonathan, Sushain yes
code apertium-init support for more features in hfst modules Add optional support to hfst modules for enabling spelling modules, an extra twoc module for morphotactic constraints, and spellrelax. You'll want to figure out how to integrate this into the Makefile template. There is a github issue for this task. python, xml, Makefile Jonathan
code, quality make apertium-init README files show only relevant dictionary file Currently in apertium-init, the README files for HFST modules show the "dix" file in the list of files, and it's likely that lttoolbox modules show "hfst" files in their README too. Check this and make it so that READMEs for these two types of monolingual modules display only the right dictionary files. There is a github issue for this task. python, xml, Makefile Jonathan, Sushain
code, quality Write a script to add glosses to a monolingual dictionary from a bilingual dictionary Write a script that matches bilingual dictionary entries (in dix format) to monolingual dictionary entries in one of the languages (in lexc format) and adds glosses from the other side of the bilingual dictionary if not already there. The script should combine glosses into one when there's more than one in the bilingual dictionary. Some level of user control might be justified, from simply defaulting to a dry run unless otherwise specified, to controls for adding to versus replacing versus leaving alone existing glosses, and the like. A prototype of this script is available in SVN, though it's buggy and doesn't fully work—so this task may just end up being to debug it and make it work as intended. A good test case might be the English-Kazakh bilingual dictionary and the Kazakh monolingual dictionary. python, lexc, dix, xml Jonathan
code Write a script to deduplicate and/or sort individual lexc lexica. The lexc format is a way to specify a monolingual dictionary that gets compiled into a transducer: see Apertium-specific conventions for lexc and Lttoolbox and lexc#lexc. A single lexc file may contain quite a few individual lexicons of stems, e.g. for nouns, verbs, prepositions, etc. Write a script (in python or ruby) that reads a specified lexicon, and based on which option the user specifies, identifies and removes duplicates from the lexicon, and/or sorts the entries in the lexicon. Be sure to make a dry-run (i.e., do not actually make the changes) the default, and add different levels debugging (such as displaying a number of duplicates versus printing each duplicate). Also consider allowing for different criteria for matching duplicates: e.g., whether or not the comment matches too. There are two scripts that parse lexc files already that would be a good point to start from: lexccounter.py and inject-words-from-bidix-to-lexc.py (not fully functional). python, ruby, lexc Jonathan
quality, interface Interface improvement for Apertium Globe Viewer The Apertium Globe Viewer is a tool to visualise the translation pairs that Apertium currently offers, similar to the apertium pair viewer. Choose any interface or usability issue listed in the tool's documentation in consultation with your mentor, file an issue, and fix it. javascript, maps Jonathan 3 5
quality, code Separate geographic and module data for Apertium Globe Viewer The Apertium Globe Viewer is a tool to visualise the translation pairs that Apertium currently offers, similar to the apertium pair viewer. Currently, geographic data for languages and pairs (latitude, longitude) is stored with the size of the dictionary, etc. Find a way to separate this data into distinct files (named sensibly), and at the same time make it possible to specify only the points for each language and not the endpoints for the arcs for language pairs (those should be trivial to generate dynamically). javascript, json Jonathan
quality, code Scraper of information needed for Apertium visualisers There are currently three prototype visualisers for the translation pairs Apertium offers: Apertium Globe Viewer and apertium pair viewer and language family visualisation tool. They all rely on data about Apertium linguistic modules, and that data has to be scraped. There are some tools which do various parts of this already, but they are not unified: There are scripts that do different pieces of all of this already: queries svn, queries svn revisions, counting bidix stems. Evaluate how well the script works, and attempt to make it output data that will be compatible with all viewers (and/or modify the viewers to make sure it is compatible with the general output format). python, json, scrapers Jonathan
quality fix pairviewer's 2- and 3-letter code conflation problems pairviewer doesn't always conflate languages that have two codes. E.g. sv/swe, nb/nob, de/deu, da/dan, uk/ukr, et/est, nl/nld, he/heb, ar/ara, eus/eu are each two separate nodes, but should instead each be collapsed into one node. Figure out why this isn't happening and fix it. Also, implement an algorithm to generate 2-to-3-letter mappings for available languages based on having the identical language name in languages.json instead of loading the huge list from codes.json; try to make this as processor- and memory-efficient as possible. javascript Jonathan
quality, code split nor into nob and nno in pairviewer Currently in pairviewer, nor is displayed as a language separately from nob and nno. However, the nor pair actually consists of both an nob and an nno component. Figure out a way for pairviewer (or pairsOut.py / get_all_lang_pairs.py) to detect this split. So instead of having swe-nor, there would be swe-nob and swe-nno displayed (connected seemlessly with other nob-* and nno-* pairs), though the paths between the nodes would each still give information about the swe-nor pair. Implement a solution, trying to make sure it's future-proof (i.e., will work with similar sorts of things in the future). javascript Jonathan, Fran, Unhammer
quality, code add support to pairviewer for regional and alternate orthograpic modes Currently in pairviewer, there is no way to detect or display modes like zh_TW. Add suppor to pairsOut.py / get_all_lang_pairs.py to detect pairs containing abbreviations like this, as well as alternate orthographic modes in pairs (e.g. uzb_Latn and uzb_Cyrl). Also, figure out a way to display these nicely in the pairviewer's front-end. Get creative. I can imagine something like zh_CN and zh_TW nodes that are in some fixed relation to zho (think Mickey Mouse configuration?). Run some ideas by your mentor and implement what's decided on. javascript Jonathan, Fran
code Extend visualisation of pairs involving a language in language family visualisation tool The language family visualisation tool currently has a visualisation of all pairs involving the language. Extend this to include pairs that involve those languages, and so on, until there are no more pairs. This should result in a graph of quite a few languages, with the current language in the middle. Note that if language x is the center, and there are x-y and x-z pairs, but also a y-z pair, this should display the y-z pair with a link, not with an extra z and y node each, connected to the original y and z nodes, respectively. The best way to do this may involve some sort of filtering of the data. javascript Jonathan
code Scrape Crimean Tatar Quran translation from a website Bible and Quran translations often serve as a parallel corpus useful for solving NLP tasks because both texts are available in many languages. Your goal in this task is to write a program in the language of your choice which scrapes the Quran translation in the Crimean Tatar language available on the following website: http://crimean.org/islam/koran/dizen-qurtnezir/. You can adapt the scraper described on the Writing a scraper page or write your own from scratch. The output should be plain text in Tanzil format ('text with aya numbers'). You can see examples of that format on http://tanzil.net/trans/ page. When scraping, please be polite and request data at a reasonable rate. scraper Ilnar, Jonathan, fotonzade
code Scrape Quran translations from a website Bible and Quran translations often serve as a parallel corpus useful for solving NLP tasks because both texts are available in many languages. Your goal in this task is to write a program in the language of your choice which scrapes the Quran translations available on the following website: http://www.quran-ebook.com/. You can adapt the scraper described on the Writing a scraper page or write your own from scratch. The output should be plain text in Tanzil format ('text with aya numbers'). You can see examples of that format on http://tanzil.net/trans/ page. Before starting, check whether the translation is not already available on the Tanzil project's page. Although the format of the translations seems to be the same and thus your program is expected to work for all of them, translations we are interested the most are the following: Azerbaijani version 2, Bashkir, Chechen, Karachay and Kyrgyz. When scraping, please be polite and request data at a reasonable rate. scraper Ilnar, Jonathan, fotonzade
documentation Unified documentation on Apertium visualisers There are currently three prototype visualisers for the translation pairs Apertium offers: Apertium Globe Viewer and apertium pair viewer and language family visualisation tool. Make a page on the Apertium wiki that showcases these three visualisers and links to further documentation on each. If documentation for any of them is available somewhere other than the Apertium wiki, then (assuming compatible licenses) integrate it into the Apertium wiki, with a link back to the original. wiki, visualisers Jonathan
research Investigate FST backends for Swype-type input Investigate what options exist for implementing an FST (of the sort used in Apertium spell checking) for auto-correction into an existing open source Swype-type input method on Android. You don't need to do any coding, but you should determine what would need to be done to add an FST backend into the software. Write up your findings on the Apertium wiki. spelling,android Jonathan
research tesseract interface for apertium languages Find out what it would take to integrate apertium or voikkospell into tesseract. Document thoroughly available options on the wiki. spelling,ocr Jonathan
documentation Integrate documentation of the Apertium deformatter/reformatter into system architecture page Integrate documentation of the Apertium deformatter and reformatter into the wiki page on the Apertium system architecture. wiki, architecture Jonathan, Shardul
documentation Document a full example through the Apertium pipeline Come up with an example sentence that could hypothetically rely on each stage of the Apertium pipeline, and show the input and output of each stage under the Example translation at each stage section on the Apertium wiki. wiki, architecture Jonathan, Shardul
documentation Create a visual overview of structural transfer rules Based on an existing overview of Apertium structural transfer rules, come up with a visual presentation of transfer rules that shows what parts of a set of rules correspond to which changes in input and output, and also which definitions are used where in the rules. Get creative—you can do this all in any format easily viewed across platforms, especially as a webpage using modern effects like those offered by d3 or similar. wiki, architecture, visualisations, transfer Jonathan, Shardul
documentation Complete the Linguistic Data chart on Apertium system architecture wiki page With the assistance of the Apertium community (our IRC channel) and the resources available on the Apertium wiki, fill in the remaining cells of the table in the "Linguistic data" section of the Apertium system architecture wiki page. wiki, architecture Jonathan yes
Personal tools