Difference between revisions of "Apertium-kir"

From Apertium
Jump to navigation Jump to search
Line 10: Line 10:


== Adding analyses to conllu ==
== Adding analyses to conllu ==

This guide assumes that you
* have <tt>apertium-kir</tt> compiled and running,
* have a working recent version of python3, and
* are saving your files in the <tt>apertium-kir/</tt> directory.


# Get CG3 format of conllu data
# Get CG3 format of conllu data
#* From Annotatrix: click CG3 tab, click menu, download, rename file to <tt>.dep</tt>
#* From Annotatrix: click CG3 tab, click menu, download, rename file to <tt>.dep</tt>
# Use <tt>add_morph.py</tt> script:
# Populate morphology in CG3 file from Apertium using <tt>add_morph.py</tt> script:
#* <code>$ cat corpus.dep | ./corpora/add_morph.py > corpus_with_annotation.dep</code>
#* <code>$ cat corpus.dep | ./corpora/add_morph.py > corpus_with_annotation.dep</code>
#* Open <tt>corpus_with_annotation.dep</tt> and fix any errors
#* Open <tt>corpus_with_annotation.dep</tt> and fix any errors
# Use <tt>dep_to_seg.py</tt> script:
# Create segmentation file with <tt>dep_to_seg.py</tt> script:
#* <code>$ cat corpus_with_annotation.dep | ./corpora/dep_to_seg.py > corpus_with_annotation.seg</code>
#* <code>$ cat corpus_with_annotation.dep | ./corpora/dep_to_seg.py > corpus_with_annotation.seg</code>
#* Open <tt>corpus_with_annotation.seg</tt> and fix any errors
#* Open <tt>corpus_with_annotation.seg</tt> and fix any errors
# Use <tt>conllise.py</tt> script:
# Convert back to CoNLL-U format with <tt>conllise.py</tt> script:
#* <code>$ python3 ./scripts/conllise.py -e apertium-kir.kir.udx corpus_with_annotation.dep corpus_with_annotation.seg</code>
#* <code>$ python3 ./scripts/conllise.py -e apertium-kir.kir.udx corpus_with_annotation.dep corpus_with_annotation.seg</code>



Revision as of 14:47, 24 April 2024

Kymorph is a morphological analyser/generator for the Kyrgyz language, currently working. It is intended to be compatible with transducers for other Turkic languages so that they can be translated between.

Installation

$ git clone https://github.com/apertium/apertium-kir.git

Adding analyses to conllu

This guide assumes that you

  • have apertium-kir compiled and running,
  • have a working recent version of python3, and
  • are saving your files in the apertium-kir/ directory.
  1. Get CG3 format of conllu data
    • From Annotatrix: click CG3 tab, click menu, download, rename file to .dep
  2. Populate morphology in CG3 file from Apertium using add_morph.py script:
    • $ cat corpus.dep | ./corpora/add_morph.py > corpus_with_annotation.dep
    • Open corpus_with_annotation.dep and fix any errors
  3. Create segmentation file with dep_to_seg.py script:
    • $ cat corpus_with_annotation.dep | ./corpora/dep_to_seg.py > corpus_with_annotation.seg
    • Open corpus_with_annotation.seg and fix any errors
  4. Convert back to CoNLL-U format with conllise.py script:
    • $ python3 ./scripts/conllise.py -e apertium-kir.kir.udx corpus_with_annotation.dep corpus_with_annotation.seg

Current State

{{#set_param_default | corpus1 | None }} {{#set_param_default | corpus2 | None }} {{#set_param_default | corpus3 | None }} {{#set_param_default | corpus4 | None }} {{#set_param_default | corpus5 | None }} {{#set_param_default | corpus6 | None }} {{#set_param_default | corpus7 | None }} {{#set_param_default | corpus8 | None }} {{#set_param_default | corpus9 | None }} {{#set_param_default | corpus10 | None }}

  • Number of stems: 14,424 {{#ifneq | | | () }}
  • Disambiguation rules: 16
  • Coverage: ~90.4%

{{#ifneq | azattyk2010 | None |

{{#ifneq | RFERL_corpora | | | }}

}}

{{#ifneq | azattyk2009 | None |

{{#ifneq | RFERL_corpora | | | }}

}}

{{#ifneq | bible | None |

{{#ifneq | | | | }}

}}

{{#ifneq | wp2011/04 | None |

{{#ifneq | | | | }}

}}

{{#ifneq | {{{corpus5}}} | None |

{{#ifneq | | | | }}

}}

{{#ifneq | {{{corpus6}}} | None |

{{#ifneq | | | | }}

}}

{{#ifneq | {{{corpus7}}} | None |

{{#ifneq | | | | }}

}}

{{#ifneq | {{{corpus8}}} | None |

{{#ifneq | | | | }}

}}

{{#ifneq | {{{corpus9}}} | None |

{{#ifneq | | | | }}

}}

{{#ifneq | {{{corpus10}}} | None |

{{#ifneq | | | | }}

}}

corpuswordscoverage
<nowinter>azattyk2010</nowinter>azattyk20103.4M ~92.11%
<nowinter>azattyk2009</nowinter>azattyk20094.1M ~92.04%
<nowinter>[[|bible]]</nowinter>bible174K ~92.25%
<nowinter>[[|wp2011/04]]</nowinter>wp2011/04545K ~85.37%
<nowinter>[[|{{{corpus5}}}]]</nowinter>{{{corpus5}}} ~%
<nowinter>[[|{{{corpus6}}}]]</nowinter>{{{corpus6}}} ~%
<nowinter>[[|{{{corpus7}}}]]</nowinter>{{{corpus7}}} ~%
<nowinter>[[|{{{corpus8}}}]]</nowinter>{{{corpus8}}} ~%
<nowinter>[[|{{{corpus9}}}]]</nowinter>{{{corpus9}}} ~%
<nowinter>[[|{{{corpus10}}}]]</nowinter>{{{corpus10}}} ~%

To-do