Apertium-uzb - Revision history

Ilnar.salimzyan: s/sourceforge link/github link/

2018-03-08T17:49:58Z

s/sourceforge link/github link/

Firespeaker: /* Dealing with Cyrillic and Latin */

2015-07-20T06:52:19Z

Dealing with Cyrillic and Latin

Firespeaker at 20:09, 17 July 2015

2015-07-17T20:09:05Z

Firespeaker at 20:08, 17 July 2015

2015-07-17T20:08:27Z

Tarjimon: /* Dealing with Cyrillic and Latin */

2015-07-17T17:48:56Z

Dealing with Cyrillic and Latin

Firespeaker: /* Installation */

2015-07-17T17:40:09Z

Installation

Firespeaker: /* oʻ and gʻ */

2015-07-17T17:36:02Z

oʻ and gʻ

Firespeaker at 17:35, 17 July 2015

2015-07-17T17:35:37Z

Firespeaker at 17:30, 17 July 2015

2015-07-17T17:30:36Z

Firespeaker: /* Dealing with Cyrillic and Latin */

2013-05-18T21:28:38Z

Dealing with Cyrillic and Latin

@@ Line 18: / Line 18: @@
 == Installation ==
-'''Apertium-uzb''' is currently located in [https://svn.code.sf.net/p/apertium/svn/languages/apertium-uzb/ languages/apertium-uzb].
+'''Apertium-uzb''' is currently located in [https://github.com/apertium/apertium-uzb languages/apertium-uzb].
 == Developers ==

@@ Line 42: / Line 42: @@
 Another script will then generate a Cyrillic version of dix and lexc from the Latin-script versions, using the above mentioned database.
+==== Final Plan ====
+We'll go with [[#Plan B|Plan B]], with file names roughly like this:
+ apertium-uzb.uzb.lexc
+ apertium-uzb.uzb.twol
+ apertium-uzb.uzb_Cyrl.lexc
+ apertium-uzb.uzb_Cyrl.twol
+ apertium-uzb.uzb_Latn-Cyrl.tsv
+To do:
+* Hand-convert the morphotactics (in lexc) and morphophonology (twol) to Cyrillic..
+* Future development of either morphotactics (in lexc) or morphophonology (twol) ''must'' be done in parallel (i.e., both Latin and Cyrillic versions will need to be updated with any changes)!
+* Write a short script that dumps all [new] stems from the lexc file and uses the transliterator script to update any entries not marked as GOOD.
+* Write a short script (that uses the lexc parsing script Sushain developed as a library?) that replaces stems in certain categories (e.g., not numbers and other truly closed categories) in the uzb_Cyrl.lexc file based on the uzb.lexc and Latn-Cyrl.tsv files.
+* Update modes file to support Cyrl stuff.
+* Write disambiguation rules to be as script-agnostic as possible.  For word-specific rules, make a version for each script?
+* Develop a plan for pairs, e.g.:
+** In uzb-kir, add _Cyrl.dix, _Cyrl.lrx (could also be automatically generated), and even transfer files (???)...
+** Set up conversion of stems using same tsv file and similar scripts..
+** Update modes file for _Cyrl

@@ Line 11: / Line 11: @@
 * '''[[Kyrgyz and Uzbek]]'''
-A recent version of the transducer can be tested in its analysis and generation capabilities at [http://turkic.apertium.org/index.html?choice=uzb#analyzation turkic.apertium.org].
+A recent version of the transducer is running live at [http://turkic.apertium.org/index.html?choice=uzb#analyzation turkic.apertium.org] and can be tested in its analysis and generation capabilities there.
 == Current State ==

@@ Line 10: / Line 10: @@
 * '''[[Kyrgyz and Uzbek]]'''
+A recent version of the transducer can be tested in its analysis and generation capabilities at [http://turkic.apertium.org/index.html?choice=uzb#analyzation turkic.apertium.org].
 == Current State ==

@@ Line 37: / Line 37: @@
 The Cyrillic lexc and dix will be generated from the Latin-script ones.
-A script will take all the stems from dix and automatically convert them to Cyrillic, updating a three-column text-file database (Latin Cyrillic Checked).  The Checked column will have two states: TOCHECK, GOOD.  This will allow a checker to fix the output of the conversion script for corner cases (mostly Russian words).
+A script will take all the stems from dix and automatically convert them to Cyrillic, updating a three-column text-file database (Latin Cyrillic Checked).  The Checked column will have two states: TOCHECK, GOOD.  This will allow a checker to fix the output of the conversion script for corner cases (mostly Russian words). [https://github.com/kodchi/uzbek-transliterator This script] (although slow) covers the edge cases and produces good results.
 Another script will then generate a Cyrillic version of dix and lexc from the Latin-script versions, using the above mentioned database.

@@ Line 19: / Line 19: @@
 === oʻ and gʻ ===
-The Uzbek letters ‹oʻ› and ‹gʻ› are properly written using ‹ʻ›, unicode character 02BB, "modifier letter turned comma".
+The Uzbek letters ‹oʻ› and ‹gʻ› are properly written using ‹ʻ›, unicode character 02BB, "modifier letter turned comma".  Use of other apostrophe letters is incorrect.
 === Dealing with Cyrillic and Latin ===

@@ Line 1: / Line 1: @@
+{{TOCD}}
+{{Infobox language
+|name=Uzbek
+|states=Uzbekistan (, Tajikistan, Kazakhstan, Kyrgyzstan)
+|iso=uzb
+|iso2=uzb
+}}
+'''Apertium-uzb''' is a morphological analyser/generator and CG tagger for [[Uzbek]], currently under development.  It is intended to be compatible with transducers for other [[Turkic languages]] so that they can be translated between.  It's used in the following language pairs:
+* '''[[Kyrgyz and Uzbek]]'''
+== Installation ==
+'''Apertium-uzb''' is currently located in [https://svn.code.sf.net/p/apertium/svn/languages/apertium-uzb/ languages/apertium-uzb].
 == Developers ==

@@ Line 2: / Line 2: @@
 === Dealing with Cyrillic and Latin ===
+==== Plan A ====
-'''The plan''': there will be two separate lexcs and twols (.lat and .cyr) with the continuation lexica and rules and all, though you may be able to get by with one twol considering how simple things are.  There will also be a master .dix, in Latin, with comments in a standarised format in Cyrillic (also possible the other way around).
+There will be two separate lexcs and twols (.lat and .cyr) with the continuation lexica and rules and all, though you may be able to get by with one twol considering how simple things are.  There will also be a master .dix, in Latin, with comments in a standarised format in Cyrillic (also possible the other way around).
 There will also be a simple script to check for dix entries without Cyrillic comments in the standard format in the master .dix, and automatically generate them, updating the Cyrillic dix, outputting "TOCHECK" or something in a comment with the converted words.  Someone then goes through and checks anything with "TOCHECK", and fixes / gets rid of "TOCHECK".
 This is how we can trivially "convert" the dix to Cyrillic, and even convert the stems in lexc when we copy/update it from -uzb.
+==== Plan B ====
+The Cyrillic lexc and dix will be generated from the Latin-script ones.
+A script will take all the stems from dix and automatically convert them to Cyrillic, updating a three-column text-file database (Latin Cyrillic Checked).  The Checked column will have two states: TOCHECK, GOOD.  This will allow a checker to fix the output of the conversion script for corner cases (mostly Russian words).
+Another script will then generate a Cyrillic version of dix and lexc from the Latin-script versions, using the above mentioned database.