Talk:Morphological dictionary - Revision history

Youssefsan: moved Talk:Morphological dictionaries to Talk:Morphological dictionary: Change title to singular.

2012-12-11T19:24:49Z

moved Talk:Morphological dictionaries to Talk:Morphological dictionary: Change title to singular.

Francis Tyers: /* Construction of minimised letter transducers */

2008-09-25T11:13:06Z

Construction of minimised letter transducers

Francis Tyers: /* Construction of minimised letter transducers */

2008-09-25T10:50:44Z

Construction of minimised letter transducers

Francis Tyers: /* Construction of minimised letter transducers */

2008-09-25T07:48:49Z

Construction of minimised letter transducers

Francis Tyers: Talk:Dictionaries moved to Talk:Morphological dictionaries

2008-08-14T09:28:12Z

Talk:Dictionaries moved to Talk:Morphological dictionaries

Francis Tyers: /* Results */

2007-09-29T22:53:59Z

Results

Francis Tyers: /* Introduction */

2007-09-29T22:12:20Z

Introduction

Francis Tyers: /* Construction of minimised letter transducers */

2007-09-23T12:55:05Z

Construction of minimised letter transducers

Francis Tyers: /* XML Format for the dictionaries */

2007-09-23T12:51:39Z

XML Format for the dictionaries

Francis Tyers: /* Dictionaries */

2007-09-23T12:47:23Z

Dictionaries

@@ Line 86: / Line 86: @@
 The method of construction uses two procedures, the ''construction'' procedure which is seen in equation 4, and
-that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheur, 1993) which consists of reverse, determinise, then reverse again and determinise again. It takes as the alphabet of the automata that is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
+that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheut, 1993) which consists of reverse, determinise, then reverse again and determinise again. It takes as the alphabet of the automata that is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
 In figure 2, a simple example of construction is provided. Transform by transform is introduced joined in equation 4 and the transducer in the form of ''prefix acceptor'', or ''trie'', that is to say the way in that there is only one node for every common prefix of the set of transforms which form the dictionary. With the suffixes of the

@@ Line 86: / Line 86: @@
 The method of construction uses two procedures, the ''construction'' procedure which is seen in equation 4, and
-that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheur, 1993) which consists of invert, determinise, then invert again and determinise again. It takes as the alphabet of the automata that is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
+that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheur, 1993) which consists of reverse, determinise, then reverse again and determinise again. It takes as the alphabet of the automata that is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
 In figure 2, a simple example of construction is provided. Transform by transform is introduced joined in equation 4 and the transducer in the form of ''prefix acceptor'', or ''trie'', that is to say the way in that there is only one node for every common prefix of the set of transforms which form the dictionary. With the suffixes of the

@@ Line 86: / Line 86: @@
 The method of construction uses two procedures, the ''construction'' procedure which is seen in equation 4, and
-that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheur, 1993) which consists of '''invertir, determinizar, volver a invertir y volver a determinizar''', taking as the alphabet of the automata that is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
+that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheur, 1993) which consists of invert, determinise, then invert again and determinise again. It takes as the alphabet of the automata that is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
 In figure 2, a simple example of construction is provided. Transform by transform is introduced joined in equation 4 and the transducer in the form of ''prefix acceptor'', or ''trie'', that is to say the way in that there is only one node for every common prefix of the set of transforms which form the dictionary. With the suffixes of the

@@ Line 121: / Line 121: @@
 By means of the procedure described here, data and language processors for the project "Open-source machine translation for the languages of the Spanish state" have been developed and analysers and generators have been produced which can process from 40,000 words per second on a PC. The time of construction of the letter transducers is around fifteen seconds for dictionaries of around 35,000 entries that with the paradigms developed have corresponded with four million different words.
-In terms of the size of memory consumed, the dictionaries currently occupy around four megabytes of XML documents. If we expand the transforms of the dictionaries &mdash;the non-cyclical part &mdash;, these occupy around 400 megabytes in any case. After being compiled, the resulting transducer occupies around 600 kilobytes on disc and in execution around 10 megabytes.
+In terms of the size of memory consumed, the dictionaries currently occupy around four megabytes of XML documents. If we expand the transforms of the dictionaries &mdash;the non-cyclical part &mdash;, these occupy around 400 megabytes each. After being compiled, the resulting transducer occupies around 600 kilobytes on disc and in execution around 10 megabytes.
 ==Conclusion==

@@ Line 9: / Line 9: @@
 Morphological analysis of a word involves retrieving from a dictionary the lexical form (the lemma and morphological attributes) of the word from the surface (superficial) form.
-The reverse process if morphological generation. This involves generating the surface form of a word from a lexical form. Word-for-word translation of lexical forms involves drawing correspondences between a lexical form in one language, and the translation in another. This final operation is crucial in constructing machine translation systems.
+The reverse process is morphological generation. This involves generating the surface form of a word from a lexical form. Word-for-word translation of lexical forms involves drawing correspondences between a lexical form in one language, and the translation in another. This final operation is crucial in constructing machine translation systems.
 Words may have one form (as in the case of invariant words), or more forms. The variation of words receives various names depending on their nature. For example, they may be ''derivational'', which is when a word combines with another word, or with morphemes which then modify its sense (e.g. president and  vicepresident etc.); ''inflectional'', which are grammatical modifications that occur in nouns, adjectives and verbs in languages like those in the Indo-European family (e.g. ''go'', ''goes'', etc.); ''agglutinative'', which is when affixes are added together and affect the whole grammatical phrase. Agglutination is found in languages like Turkish and Basque (e.g. urdin, urdina, urdinarena, that in Spanish correspond to ''azul'', ''el azul'', ''el del azul'' respectively); or every other type of orthographic variation that can occur in a language.

← Older revision	Revision as of 19:24, 11 December 2012
(No difference)

← Older revision	Revision as of 09:28, 14 August 2008
(No difference)

@@ Line 85: / Line 85: @@
 to <math>(\varepsilon : \varepsilon)</math>, this is crucial for the consistency of the method of construction  that can subsequently be seen.
-The method of construction uses two procedures, the procedure of ''assembly'' which is seen in equation 4, and
+The method of construction uses two procedures, the ''construction'' procedure which is seen in equation 4, and
-that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheur, 1993) which consists of '''invertir, determinizar, volver a invertir y volver a determinizar''', taking as the alphabet of the automata that there is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
+that of minimisation, which is produced by a conventional finite state automata minimisation algorithm (van de Snepscheur, 1993) which consists of '''invertir, determinizar, volver a invertir y volver a determinizar''', taking as the alphabet of the automata that is minimised the Cartesian product <math>L\,</math> and for null transition <math>(\theta : \theta)\,</math>.
-In figure 2, a simple example of assembly is provided. Transform by transform is introduced joined in equation 4
+In figure 2, a simple example of construction is provided. Transform by transform is introduced joined in equation 4 and the transducer in the form of ''prefix acceptor'', or ''trie'', that is to say the way in that there is only one node for every common prefix of the set of transforms which form the dictionary. With the suffixes of the
-and the transducer in the form of ''prefix acceptor'', or ''trie'', that is to say the way in that there is only one node for every common prefix of the set of transforms which form the dictionary. With the suffixes of the
 transforms (which are not separated), new states are created.

@@ Line 30: / Line 30: @@
 Figure 1. shows an example of the definition of a paradigm and its use in the dictionary. Each paradigm has ''entries'' (<code><e></code> elements), and in this example, every all of the entries consist of a pair (<code><p&gt;</code>) which has a left part (<code><l></code>) and a right part (<code><r></code>). Between these elements, text or morphological symbols (<code><s&gt;</code>) can be included. The entries of the dictionary are defined in the same way, the tag for identity <code> <i&gt;</code> is an abbreviated form of a pair where the left side and the right side are identical. The paradigm of a the word is expressed in this example by the paradigm reference <code><par></code>.
-Is it possible to define cyclic paradigms by setting an attribute in the paradigm. It is also possible to see that all paradigms can be defined as cyclic, with the exception of those which do not accept an empty string, '''ya que se puede dar el caso de que''' the output will be infinite for a given entry (loop without consuming the entry). Detecting whether a paradigm has been incorrectly defined cyclically is a job for the compiler which constructs the letter transducer.
+It is also possible to see that not all paradigms can be defined as cyclic, but only those that do not accept an empty string, given that there could be a case where the output for a given entry will be infinite (loop without consuming all the input). Detecting whether a paradigm has been incorrectly defined cyclically is a job for the compiler which constructs the letter transducer.
 ==Obtaining paradigms==

@@ Line 67: / Line 67: @@
 ===Dictionaries===
-A dictionary of transforms is defined (for example to incorporate linguistic knowledge) using paradigms which describe inflection. A dictionary of transforms is presented as a set <math>D = (E,P,P^C)</math> in which:
+A dictionary of transforms is defined (for example to incorporate linguistic knowledge) using paradigms which describe inflection. A dictionary of transforms is presented as a set <math>D = (E,P,P^C)\,</math> in which:
 * <math>E</math> is the set of the inputs of the dictionary of transforms. The set of inputs of the dictionary can be seen as a big paradigm, which the restriction that it cannot contain any insertions. In the dictionary the paradigms serve to represent the existing linguistic knowledge in the truths observed by the lexical processor.
@@ Line 76: / Line 76: @@
 A dictionary <math>D</math> can be represented by means of a letter transducer that has as its input alphabet <math>\Sigma \cup {\theta}</math> and as its output alphabet <math>\Gamma \cup {\theta}</math>; <math>L = (\Sigma \cup {\theta}) \times (\Gamma \cup {\theta})</math> is defined. The set of states of the transducer is <math>Q</math>, the initial state is <math>qI \in Q</math>, the set of final states <math>F \subset Q</math> and the transfer function <math>\delta : Q \times L \rightarrow 2^Q</math>, and therefore it is indeterministic with respect of the input
-as respect of the new alphabet <math>L</math>.
+as respect of the new alphabet <math>L\,</math>.
 ===Construction of minimised letter transducers===