Ideas for Google Summer of Code/Automatic blank handling
A superblank is something that we don't want to translate, but keep in the output, often things like formatting tags. Currently there is a major problem with how formatting / superblanks interacts with word/chunk reordering in Apertium. Transfer rules can reorder words, but never look at the tags themselves, and have no general way of allowing the tags themselves to be reordered along with the words, without potentially messing up the resulting code (e.g. putting an end tag before a start tag).
There is a solution: let the format handler treat some tags as "inline" (e.g. <em/>, <b/>) and others as "block-level" (e.g. <div>, <p<). Inline tags always stick with their words. If there are several words covered by one inline tag, they're duplicated to each word, and since they're "glued to the word", a transfer rule can move that word around without worrying about blanks. Block-level tags are always moved "outside" the input that a transfer rule gets; so a <p> will never be moved around by a rule.
This GsoC project involves changing the whole Apertium pipeline to do this new automatic blank handling, in particular deformatters and the the transfer module(s).
Read Format handling and Apertium stream format for background, then Reordering superblanks for some more explanations of the problem and solution.
Tasks
- Make format handlers include a list of inline tags, and disperse these to the words covered by them.
- Make pretransfer disperse tags when splitting lexical units
- Make transfer output the non-inline blanks on before the rule output
- Make transfer handle inline-blanks, and ignore <b pos="N">
- Ensure all other modules are fine with the new format for inline blanks
Coding challenge
- Make the HTML format handler turn "<i>foo bar</i>" into "<nowiki>[{<i>}]foo [{<b>}]bar<nowiki>"
Frequently asked questions
- none yet, ask us something! :)
