Search results

The ultimate goal is to have multi-purpose transducers and annotated corpora (i.e. treebanks) for a variety of Turkic languages. These can then be pair Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

...(grammatical descriptions, wordlists, dictionaries, spellcheckers, papers, corpora, etc.) for Aromanian, along with the licences they are under. || || [[User: ...=center| {{sc|research}} || 3. Easy || Create manually tagged corpora: Occitan || Fix tagging errors in a piece of analysed text, for use in tag

...however, a language package should have over 60% coverage on a variety of corpora and should probably have at least 2500 stems to be considered minimally use * The coverage of the transducer on a variety of corpora

== Estimating rules using parallel corpora == ...see [[Running the monolingual rule learning]] if you only have monolingual corpora).

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc === Corpora and corpora projects ===

* directory es-tagger-data : Contains data needed for the Spanish tagger (corpora, etc.) * directory ca-tagger-data : Contains data needed for the Catalan tagger (corpora, etc.)

| name = Dictionary induction from parallel corpora / Revive ReTraTos | description = Extract dictionaries from parallel corpora

...quirements for corpora, and a number of different formats for storing such corpora have sprung up. Some examples include: ...ng on). The following is an idea Jonathan has for implementing a standard corpora format for use by apertium.

...millions of words as in the statistical methods: it takes only two smaller corpora and a dictionary containing rules to conjugate verbs and to match nouns and ...anguage pairs of the same linguistic family without the need of linguistic corpora. The experience of Apertium with several minoritised languages such as Occi

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

=== Annotated corpora ===

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

* Collect parallel kaz-eng corpora! By new plan, we focused on adding vocabulary from 4 corpora.

* [http://corpus.leeds.ac.uk/query-zh.html A Collection of Chinese Corpora and Frequency Lists.] ===Corpora===

....za/Faculties/ART/Xhosa/Pages/Research-.aspx "Cross linguistics upon Xhosa Corpora Research"] == Monolingual/Parallel Corpora ==

{{deprecated2|Learning rules from parallel and non-parallel corpora}} * a parallel corpus (see [[Corpora]])

==Getting corpora== WORDLIST=/home/spectre/corpora/afrikaans-meester-utf8.txt

...ert the data between the formats. It also allows to either upload or paste corpora in plain text and then convert them into CoNLL-U. ...des support for saving user corpora on server and then accessing the saved corpora via unique URL.

...is an open source tool included on the Apertium project that let you train corpora and manage related files with a friendly user interface and letting you foc ...rom this view you are able to see corpora and training details, insert new corpora and train them easily

* [[Learning rules from parallel and non-parallel corpora]] – this is the current documentation on training/inferring rules ** preprocess corpora

===Corpora=== * [http://corpora.uni-leipzig.de/en?corpusId=hin_news_2011 Hindi News Corpus] Creative Common

Apertium-regtest is a program for managing regression tests and [[Corpus test|corpora]]. # in the browser, select one or all of the corpora to rerun tests for

===Corpora=== * [http://childes.talkbank.org/access/French/ CHIDES Corpora]. [http://talkbank.org/share/rules.html ''Requires reference'']

..., Tommi Pirinen, Jonathan Washington. "Finite-state morphologies and text corpora as resources for improving morphological descriptions". [https://sites.goog ...f Inferring shallow-transfer machine translation rules from small parallel corpora]". In Journal of Artificial Intelligence Research. volume 34, p. 605-635.

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

====Training on Corpora with Ambiguous Lexical Units==== ...m-tagger</code> '', the tagger prints warnings about ambiguous analyses in corpora to stderr.''

* Tufiş, D., A. M. Barbu, V. Pătraşcu, G. Rotariu, and C. Popescu. "Corpora and Corpus-Based Morpho-Lexical Processing."&nbsp;''Recent Advances in Roma ===Corpora===

...why the errors are so high. Re-evaluation will be done as soon as the two corpora are manually realigned. ...nslated corpora, and the label will be extracted from the manually written corpora. This method might provide better results since the model will be trained o

==Corpora== The tagged corpora used in the experiments are found in the monolingual packages in [[language

...n be found [https://github.com/taruen/apertiumpp/tree/master/data4apertium/corpora here]. ...m/corpora/jam/uzb.txt | apertium -d . uzb-kaa) -ref ../../../data4apertium/corpora/jam/kaa.txt

===Corpora=== * [http://opus.nlpl.eu/ Hindi-English Parallel Corpora]

===Corpora === * [http://www.elra.info/en/catalogues/free-resources/nepali-corpora/ ''"Nepali"'']

=== Obtaining corpora (and getAlignmentWithText.pl) === The corpora need to be untarred, and inserted into a new, common directory.

Automatic shallow-transfer rules generation from parallel corpora ...in statistical machine translation, that have been extracted from parallel corpora and extended with a set of restrictions controlling their application.

~/source/corpora/lm/en.blm europarl.en-es.es.multi-trimmed -f > europarl.en-es.es.annotated MODEL=/home/philip/Apertium/corpora/language-models/en/setimes.en.5.blm

...e project was a success. All the goals have been achieved: the creation of corpora in LSC; italian monodix: apertium-srd-srd.dix: 51,743 words; apertium-ita-i ...rned to use markup languages (XML and HTML) for the creation of linguistic corpora. At present, I attend a Master’s Degree in Translation of specialized tex

== Corpora and Coverage == ...he help of mentors on Kipchak languages. Most frequent unknown tokens from corpora of each language (mostly consisting of Wikipedia entries, Bible and Quran)

===Corpora=== * [https://korpora.zim.uni-duisburg-essen.de/Limas/ Corpora from Limas z.Hd. Prof. Dr. Bernhard Schröder Universität Duisburg-Essen,

== Corpora and Coverage == Our main corpora consisted of [https://www.rfa.org/uyghur/ RFA], [http://uy.ts.cn/ Tanritor]

== Corpora == ...8ba2a9c0e50bc885bfad3bfbff3b4afbd.pdf Building Open Javanese and Sundanese Corpora for Multilingual Text-to-Speech]

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

== Corpora, sets and alignment == The parallel corpora for the Macedonian - English pair, a total of 207.778 parallel sentences, c

== Corpora == * wikipage: <section begin=azadliq2012-wikipage />RFERL corpora<section end=azadliq2012-wikipage />

* Evaluate system on corpora === Various Potential Corpora ===

=== Corpora === * [https://corpora.uni-leipzig.de/en?corpusId=ind_mixed_2013 Leipzig Corpora Collection - Indonesian]

===Corpora===

...ine. Rules can be manually written, or learnt from monolingual or parallel corpora. {{main|Learning rules from parallel and non-parallel corpora}}

* d'un corpus parallèle (voir [[Corpora]]) Nous alors faire l'exemple avec [[Corpora|EuroParl]] et la paire anglais vers espagnol d'Apertium.

...language model for the target language in order to create pseudo-parallel corpora, and use them in the same way as parallel ones. IRSTLM is a tool for building n-gram language models from corpora. It supports different smoothing and interpolation methods, including Writt

$ xzcat ~/corpora/nob/*ntb*.xz | head -100000 | apertium -d . nob-nno_e > 2019-09-30.before $ xzcat ~/corpora/nob/*ntb*.xz | head -100000 | apertium -d . nob-nno_e > 2019-09-30.after

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

...like the page on [[Aromanian]]—i.e., all available dictionaries, grammars, corpora, machine translators, etc., print or digital, where available, whether Free

...essicale e dell’analisi contrastiva è stata provvidenziale la creazione di corpora costituiti da testi redatti nella variante LSC, estrapolati da riviste on-l ...vocabolario Logudorese-italiano di Mario Casu e l’analisi approfondita dei corpora paralleli che ci hanno permesso di capire quale fosse, caso per caso, il ma

* Corpus testvoc = apertium-tat-rus/testvoc/corpus/trimmed-coverage.sh. Corpora can be found in the turkiccorpora repository. |colspan="2" rowspan="4"| Corpus testvoc clean on all of the available corpora ||rowspan="4"| ||rowspan="4" colspan="2" style="text-align: center"| ✗||r

or adapting the software to fit your needs. Existing free (GPL) data and corpora easily reusable to feed Apertium's dictionaries are also welcome. ...esidades particulares. También se agradece la disponibilización de datos y corpora libres (GPL) que sean reutilizables para mejorar los diccionarios de Aperti

=== Annotated corpora ===

===Application for "Interface for creating tagged corpora" GSOC 2013===

...pertium-kaz/stats|~{{:apertium-kaz/stats/average}}%]] coverage over random corpora ...pertium-tat/stats|~{{:apertium-tat/stats/average}}%]] coverage over random corpora

===Corpora=== * [http://corpora.uni-leipzig.de/en?corpusId=hin_news_2011 Hindi News Corpus]

...aled and combined. The formulas for combining them can be learnt from gold corpora unsupervisedly. * gold-standard tagged corpora and

=== Corpora ===

== Corpora ==

*Trainer on_fly working and training corpora '''Done''' ***Now the trainer is able to train corpora using just the keyboard, with a friendly user interface, cleaned corpus to

== Corpora ==

== Corpora ==

...do some of the tests like generation testing or coverage testing, we need corpora, right? Have no fear, for `aq-wikicrp` is here! Let us get a Maltese wikipe ...t you'd expect, tests the dictionary for coverage. Using our newly created corpora, we can test the coverage! Feel free to use either one, but be consistent;

...tion quality. This will involve improving coverage to 95-98% on a range of corpora and decreasing word error rate by 30-50%. For example if the current word e ...cial languages of EU? : Then this task will be hard. Pairs which have huge corpora of parallel texts, like the 24 official EU languages or the 3 EU working la

=== Corpora ===

=== Corpora ===

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

===Corpora===

=== Corpora ===

=== Corpora ===

== Corpora ==

...r the lexical analysis and selection contrastive was providential creating corpora consist of texts written in the LSC variant, taken from magazines on -line ...io Casu's Logudorese-Italian vocabulary and in-depth analysis of parallel corpora that have allowed us to understand what, case by case, the greatest number

== Corpora ==

* http://permalink.gmane.org/gmane.science.linguistics.corpora/22281 Arabic names from dbpedia ===Corpora===

=== Corpora ===

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

...Pirinen, Jonathan Washington (2015). "Finite-state morphologies and text corpora as resources for improving morphological descriptions". [https://sites.goog

== Corpora ==

* Tagger training preparation (tagged corpora unification) * Tagger training preparation (tagged corpora unification)

== Corpora ==

== Corpora ==

===Corpora===

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

...is a program for aligning words and sequences of words in sentence aligned corpora. If you have parallel corpus you can use GIZA++ to make bilingual dictionar *[[Corpora]]

== Corpora ==

== Corpora ==

== Corpora ==

==Parallel corpora==

== Electronic Corpora ==

== Corpora ==

== Corpora ==

'''CORPORA''' ...sketchengine.eu/user-guide/user-manual/corpora/by-language/hausa-boko-text-corpora/

== Corpora ==

== Corpora ==

== Corpora ==

== Corpora ==

==Corpora==

== Corpora ==

== Corpora ==

===Corpora===

== Other Corpora ==

== Corpora ==

== Corpora ==

# the best taggers use hand-tagged corpora to train with (we use untagged corpora -- for English)

* [[Corpora]]

...toscore.txt | ~/source/apertium/trunk/apertium-lex-learner/irstlm-ranker ~/corpora/català/en.blm > unk.trans.scored.txt

to be related. Both corpora must be pre-processed before the training. This pre-processing, consisting in analysing the corpora and

The final coverage of the system was around 90%, e.g. over a set of corpora 10 unknown words out of 100 on average. The word-error rate was around 17%, When linguistic resources, for example corpora, dictionaries, grammars, morphological analysers, lists of lemmata etc. are

The final coverage of the system was around 90%, e.g. over a set of corpora 10 unknown words out of 100 on average. The word-error rate was around 17%, When linguistic resources, for example corpora, dictionaries, grammars, morphological analysers, lists of lemmata etc. are

The final coverage of the system was around 90%, e.g. over a set of corpora 10 unknown words out of 100 on average. The word-error rate was around 17%, When linguistic resources, for example corpora, dictionaries, grammars, morphological analysers, lists of lemmata etc. are

* '''Coverage''' is the naïve coverage over one or more free corpora.

...steps to create each one. But you'll want to build it up, testing against corpora, etc. You want to be able to have as many correct analyses as possible, an ...if your pair will support translation in both directions). The corpus or corpora ideally should represent a range of content—i.e., it shouldn't be just sp

...dinian to Italian. We started a manual morphological disambiguation of the corpora that will help the translator to recognize the correct morphology of each w We have treated two corpora: one journalistic and more dialectal, and other taken directly from literar

Word Sense Disambiguation for WordNet corpora ...oblem is that SMT requires a lot of data in the form of parallel languages corpora, since they very addicted to data, and many languages cannot afford it. Whi

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

...t consisting of a set of linguistic resources (dictionaries, cross models, corpora, links to other LRDs, etc.). ...uistics resources: morphological and bilingual dictionaries, cross models, corpora, etc.</description>

* [[Learning rules from parallel and non-parallel corpora]] ...sing statistics and requires 1) slightly pre-processed dictionaries and 2) corpora to train the module. '''The module is turned off in most cases as it does n

-lm 0:5:/home/fran/corpora/europarl/europarl.lm:0 >log 2>&1 & ...e> script to generate relative frequency lists of in-domain and out-domain corpora.

xzcat corpora/nno.xz | tr -d '#@/' | apertium -d . nno-nob-dgen | grep '.\{0,6\}[#@/].\{0

** Overall accuracy (over parallel corpora): WER/PER/BLEU ...generalised script that supports hfst and lttoolbox binaries and arbitrary corpora would be good. It should also (optionally) output hitparades (e.g., freque

...an give unsatisfactory results (WER ≈ 30%, coverage below 85% in Wikipedia corpora). Both were published in 2009 and, apparently, no one has worked on them si

Corpora in { gap } are large collections of texts enhanced with special markup. The Corpora in { gap } are large collections of { gap } with { gap }. They allow lingui

#* <code>$ cat corpus.dep | ./corpora/add_morph.py > corpus_with_annotation.dep</code> #* <code>$ cat corpus_with_annotation.dep | ./corpora/dep_to_seg.py > corpus_with_annotation.seg</code>

corpora. ...the precess, creating an easy-to-use framework for using constraints with corpora in order to obtain information about words of interest; and later to provid

* large monolingual corpora of the language * parallel corpora of the language and some other language

...ment specifying a set of linguistic resources (dictionaries, cross models, corpora, other LRD files, etc).

...ger and an initial set of translation rules from monolingual and bilingual corpora.

...ing with dictionaries, lexical selection rules, transfer rules, scripting, corpora. The objective is to facilitate the generation of varieties for languages w ...cial languages of EU? : Then this task will be hard. Pairs which have huge corpora of parallel texts, like the 24 official EU languages or the 3 EU working la

...are creating unvaluable linguistic resources such as disambiguated tagged corpora. [[Category:Ideas for Google Summer of Code|Interface for creating tagged corpora]]

Had an idea of fixing of out sync corpora automatically and started an "MVP" here: https://github.com/frankier/aperti ...opt (rather than the quality of its output). It could be used to help keep corpora, tagger models and morphologies in sync (though poking and possible automat

# all pronouns from Crimean Tatar corpora are translated without debug symbols * analyse corpora with crh-morph mode

...an vocabulary were used to good effect to reach a high coverage on all the corpora.

...edia dumps are useful for quickly getting a corpus. They are also the best corpora for making your language pair are useful for Wikipedia's [[Content Translat $ zcat ~/Nedlastingar/cx-corpora.nb2nn.text.tmx.gz \

...(grammatical descriptions, wordlists, dictionaries, spellcheckers, papers, corpora, etc.), along with the licences they are under. See for example the page [[

...ich can be inserted into them. This data might consist of wordlists, word-corpora derived from web-crawlers such as [http://borel.slu.edu/crubadan/ Crubadán

...ents immensely good at helping us out with these: for instance, annotating corpora that are needed to train Apertium modules, or finding bugs in the handling

...maximum usage of available resources for marginalised languages. Parallel corpora, user-feedback, other translation systems.

...ger and an initial set of translation rules from monolingual and bilingual corpora.

...uistics resources: morphological and bilingual dictionaries, cross models, corpora, etc.</description>

$ bzcat ~/corpora/nno.txt.bz2 |./make-freqlist.sh > nno.freqlist

Pour le corpus parallèle, on va utiliser Europarl, la page [[corpora]] (seulement en anglais) en liste d'autres :

TRAIN=/home/philip/Apertium/corpora/raw/setimes-hr-mk-nikola

...ents immensely good at helping us out with these: for instance, annotating corpora that are needed to train Apertium modules, or finding bugs in the handling

For the parallel corpus we're going to use Europarl, the page [[corpora]] lists some others:

Some corpora are formatted in XML and put e.g. the real text contents inside a particula

MODEL=/home/philip/Apertium/corpora/language-models/mk/setimes.mk.5.blm

...biguadore morfològicu chi at a èssere ùtile pro sa disambiguatzione de sos corpora e non si nd'at a pòdere fàghere a mancu pro isvilupare àteras crobas lin

...nually filtered and corrected source - target pairs from OpenSubtitles2018 corpora preprocessed with bicleaner + both ways Apertium translations (ukr -> rus,

...UD Annotatrix] - in-browser software for annotating Universal Dependencies corpora.

...lexicon database with part of speech. We achieved coverage of % on SETimes corpora. And i am really happy with kymorph. Special thanks to firspeaker.

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "produc

* corpora.

* consider including the web concordancer on the site (and consider what corpora to provide search access to...)

Some pre-processed corpora can be found [http://corpora.informatik.uni-leipzig.de/download.html here] and [http://wt.jrc.it/lt/Acqu

...t that they could increase the coverage significantly, because the testing corpora are either news or WP).

...rds is a semi-standard convention (it's occurring at least some in all the corpora). We should figure out where this is happening and see if it's something w

{{see-also|Corpora}}

.... (2008) "Automatic induction of bilingual resources from aligned parallel corpora: application to shallow-transfer machine translation". ''Machine Translatio

WORDLIST=/home/spectre/corpora/afrikaans-meester-utf8.txt

...ns: apertium-kaz-tat has at least 15000 top stems, 95% coverage on all the corpora we have, and no more than 15% Word-Error-Rate on any randomly selected text

* Optimised for small corpora (under 100k parallel sentences)

...li08j.pdf Automatic induction of bilingual resources from aligned parallel corpora: application to shallow-transfer machine translation]". ''Machine Translati

-x, --xml Output corpora in XML format

The corpora used for this task can be found here: http://www.statmt.org/europarl/v7/sl-

Before you start you first need a [[Corpora|corpus]]. Look in apertium-eo-en/corpa/enwiki.crp.txt.bz2 (run bunzip2 -c e

...learning to construct such n-level transducers, working with some learning corpora, and mostly using the OSTIA state-merging algorithm.

Avant que vous vous commenciez vous avez d'abord besoin d'un [[Corpora|corpus]]. Regardez dans apertium-eo-en/corpa/enwiki.crp.txt.bz2 ? Lancez

* Efficiency: Make it scale up to corpora of millions of words. This might involve doing (a) pre-analysis of the corp

...to make a translation guesser using the existing bidix and two monolingual corpora in a similar way.

...ingual dictionaries: At the beginning we started using Chinese and Spanish corpora in order to obtain lots of Chinese-Spanish word pairs. Using the Stanford S

...story] (or [https://github.com/taruen/apertiumpp/tree/master/data4apertium/corpora/jam from here] ) as possible &mdash; Minimum one sentence. ...stvoc]] clean, and has a coverage of around 80% or more on a range of free corpora.

* [[Corpora]]

...l trained on prepared datasets which were made from parsed syntax-labelled corpora (mostly UD-treebanks). The classifier analyzes the given sequence of morpho

|0|| || collecting Tatar and Bashkir corpora, scraping a parallel corpus, making a frequency dictionary

* [[Corpora]]

{{LangStats2 | lang = kaz | corpora = Әуезов,bible,azattyq2010,wp2011,quran | corpus1 = Әуезов | co

Corpora

...the weighted rules on overall quality and speed of translation using large corpora for training and evaluation.

* Discovering these kind of things should be possible using only monolingual corpora, or resources such as Wikipedia.

Write a script that reads two parallel corpora, applies the appropriate monolingual taggers and some word-aligner ([https:

wget ftp://ftp.ims.uni-stuttgart.de/pub/corpora/SFST/SFST-1.4.6g.tar.gz

*[[Corpora]]

'''Corpora:

* [[/Corpora]]

...helper when training (see [[Learning rules from parallel and non-parallel corpora]]).