User:Khannatanmai/GSoC2019Report

Anaphora Resolution in Apertium

For this project , I coded a new module for the Apertium Machine Translation Pipeline. The purpose of this module is to do Anaphora Resolution in the source text so that we can get a better translation output.

Documentation

I have prepared a verbose documentation for this module which explains what Anaphora Resolution means, it's role in the Apertium pipeline, the algorithm used, how to use the module and how to add the module to a new language pair.

You can find this here: Anaphora Resolution Module

Final Module

Here is the final released version of the Anaphora Module. Follow the instructions in the README to install this module on your system and run it.

Repository of Module: https://github.com/apertium/apertium-anaphora

Work Done during GSoC 2019

The above module is the final version of the Anaphora Resolution module, complete with a build system.

You can refer to the issues to see implementation of features and comments as they were being implemented during GSoC: https://github.com/apertium/apertium-anaphora/issues

During GSoC, this repository was used: https://github.com/khannatanmai/apertium-anaphora (This repository is just to highlight the peripheral work done during GSoC)

Links to Work Done

Code for the module

Main Repository: https://github.com/apertium/apertium-anaphora

Changes made to Apertium Code

To accomodate for the new module in the pipeline.

Pull Request: https://github.com/apertium/apertium/pull/55

Changes made to Apertium Spanish-English Language Pair

To test the new module on the Spanish-English language pair.

Pull Request: https://github.com/apertium/apertium-eng-spa/pull/13

Evaluation

The Anaphora Resolution module was tested on multiple languages with some basic indicators. I'll be presenting the results of the evaluation, which was done manually.

Spanish - English

Spanish has a possessive determiner su, which can translate to his/her/its in English, so we need to resolve it as an anaphor.

The Anaphora Resolution Module was run on a corpus of a 1000 sentences from Europarl, using this arx file: [Add link]

Results

• No Change, Correct: Anaphora Resolution didn't change the anaphor and it is correct.
• No Change, Incorrect: Anaphora Resolution didn't change the anaphor, and it is incorrect, i.e. it should have changed.
• Change, Correct: Anaphora Resolution changed the anaphor and it is now correct (was incorrect earlier).
• Change, Incorrect: Anaphora Resolution changed the anaphor and it is now incorrect. (was correct earlier)

No Change		Change
Correct	Incorrect	Correct	Incorrect
33	53	32	2

Number of anaphors translated correctly without the Anaphora Resolution module and with:

Total 3rd Person Anaphors	Without Anaphora Resolution	With Anaphora Resolution
	Correct	Correct
120	35	65

Observations

• A lot of the errors are made because the tagger gives the singular tag to group nouns such as Parliament, Commission, Group. If this is fixed, the results should improve significantly.
• Since the module only outputs his/her/their right now, all the examples with its haven't been resolved. Adding this would improve the results as well.

• The indicators one uses are corpus dependent. This corpus has a dialogue and hence we added an impeding indicator to patterns such as: <NP> <comma>, as that NP is usually the addressee.

For detailed observations, refer to the Complete Evaluation

Future Ideas

There are several future ideas to make this module better that I'll be trying my hand on after GSoC.

These are all mentioned in the Issues in the Module Repository: https://github.com/apertium/apertium-anaphora/issues .