Apertium has moved from SourceForge to GitHub.
If you have any questions, please come and talk to us on #apertium on irc.freenode.net or contact the GitHub migration team.

Ideas for Google Summer of Code/Improving support for non-standard text input

From Apertium
Jump to: navigation, search

Contents

Create a module that will standardise non-standard input. For example, slang, abbreviations.

[edit] Some examples from English

  • Extra space: "he he" (hehe)
  • Spacing and hyphen variation: no-one, noone, no one
  • Optional hyphen: re-integrate, reintegrate
  • Missing apostrophe: shes thinking about it
  • Non-standard capitalisation: im thinking about it
  • Abbreviated words: fav,
  • Emoticons: :)

[edit] Coding challenge

  • Make a test corpus of non-standard texts for a particular domain (could be IRC, tweets, forums etc.)
  • Translate them with Apertium
  • Come up with examples of non-standard features that effect translation quality
  • Propose ways in which they might be solved.

[edit] Tasks

  • Do a literature review of papers on normalisation of input.

[edit] Frequently asked questions

[edit] See also

Personal tools