Difference between revisions of "Crimean Tatar and Turkish/Work plan"

From Apertium
Jump to navigation Jump to search
 
(2 intermediate revisions by one other user not shown)
Line 3: Line 3:


|-
|-
| 3 ||22nd May — 28th May || 40% || 43.9% ||
| 3 ||22nd May — 28th May || 40% || 43.9% || '''✔'''
|-
|-


Line 15: Line 15:


|-
|-
| 4 ||29th May — 4th June || 40% || ||
| 4 ||29th May — 4th June || 40% || || '''✔'''
|-
|-


Line 21: Line 21:


|-
|-
| 5 ||5th June — 11th June || 65% || ||
| 5 ||5th June — 11th June || 65% || || '''✔'''
|-
|-


Line 27: Line 27:


|-
|-
| 6 ||12th June — 18th June || 70% || ||
| 6 ||12th June — 18th June || 75% || || '''✔'''
|-
|-


Line 51: Line 51:


|-
|-
| 9 ||3rd July — 9th July || 82% || ||
| 9 ||3rd July — 9th July || 84% || ||
|-
|-


Latest revision as of 18:27, 19 June 2017

Week Dates Coverage Achieved Evaluation
3 22nd May — 28th May 40% 43.9% ✔
* Add all non-inflecting words
* Finish challenge text (no *,#)
* Do baseline evaluation (WER)
Official start
4 29th May — 4th June 40% ✔
* Break
5 5th June — 11th June 65% ✔
* ?
6 12th June — 18th June 75% ✔
* ?
* ?
7 19th June — 25th June 80%
Phase 1 evaluation
Deliverable: All closed classes + numerals testvoc clean
8 26th June — 2nd July 84%
* ?
* ?
9 3rd July — 9th July 84%
* ?
10 10th July — 16th July 84%
* ?
* ?
11 17th July — 23rd July 86%
Phase 2 evaluation
Deliverable: Nouns, adjectives testvoc clean
* ?
12 24th July — 30th July 88%
* ?
13 1st August — 6th August 89%
* ?
14 7th August — 13th August 90%
* ?
15 14th August — 20th August 91%
* ?
16 21th August — 27th August 92%
Final evaluation
Final deliverable: Full MT system, testvoc clean.
* Evaluation
* Write paper
17 28th August — 3rd September
* Write paper
18 4th September — 6th September
* Write paper

Coverage[edit]

To measure the bidix-trimmed coverage, use apertium-crh-tur/testvoc/corpus/trimmed-coverage.sh: 

apertium-crh-tur$ bzcat ~/src/turkiccorpora/crh.wpdump.20151123.txt.bz2 | \
                  bash testvoc/corpus/trimmed-coverage.sh | less

Number of tokenised words in the corpus:         148013
Number of tokenised words unknown to analyser:    63730  —  43.1 % of tokens had *
                          unknown to bidix:         112  —   0.1 % of tokens had @
     w/transfer errors or unknown to generator:    2473  —   1.7 % of tokens had #

Error-free coverage of analyser only:             84283  —  56.9 % of tokens had no *
Error-free coverage of analyser and bidix:        84171  —  56.9 % of tokens had no */@
Error-free coverage of the full translator:       81698  —  55.2 % of tokens had no */@/#

Top unknown words in the corpus:
    972 ^*Ukrainanıñ$
    939 ^*vilâyetinde$
    631 ^*şeklinde$
    607 ^*qasaba$
    508 ^*merkezi$
    434 ^*rayonınıñ$
    329 ^*da$
    283 ^*de$
    235 ^*adı$
    221 ^*vilâyeti$

Tokens needed to get 65.0 % bidix-trimmed coverage (no */@/#): 12037
Storing corresponding wordlist in /tmp/corpus-stat-all-needed.txt
        
        
^Baş<n><nom>$   Baş
^*Saife$        *Saife

...

Testvoc[edit]

Requirements for testvoc in week 1:

  1. all pronouns from Crimean Tatar corpora are translated without debug symbols
  2. all pronouns the transducer generates must pass without debug symbols (this is less important, and only to focus on if done with 1)

To achieve 1:

  • analyse corpora with crh-morph mode
  • grep pronouns
  • make sure they pass through the rest of the pipeline without getting @ or #

To achieve 2:

  • in 'Root' lexicon of the .lexc file, comment out everything except Pronouns
  • generate pronouns with hfst-fst2string crh.automorf.hfst
  • make sure they pass through the rest of the pipeline without getting @ or #

We don't want to spend too much time on forms which might be over-generated by the transducer. This is the reason why we focus on 1 first.

Retrieved from "https://wiki.apertium.org/w/index.php?title=Crimean_Tatar_and_Turkish/Work_plan&oldid=63336"