Unification of metadix and parametrized dictionaries

En français

Different language-pair packages use different strategies to generate .dix dictionaries (monodix) and (bidix) from XML files using features not supported by the .dix format. The objectives of these new dix-like formats are:

• being able to use parametrized paradigms (so that a general paradigm may be defined and used with small parametrized variations), as discussed in the metadix page;
• being able to generate different versions of a translator (for instance, for two different varieties of a language, such as Brazilian and European Portuguese) whose names could be ideally tied to mode names

There is currently a debate on a unification of these formats into a single metadix format which in turn could also be used to support other desirable features such as

• having metadata (headers) in dictionaries which defines whether the dictionary is a bilingual or monolingual dictionary and the language pairs and modes it supports (perhaps this could be added to the basic .dix format)

Here is a proposal (open to discussion) on the first two issues.

Header

It would be useful to have the language name(s) and probably some other information (maybe on varieties?) specified in some kind of a header file.

<dictionary>
  <header>
    <type>monolingual</type>
    <language code="ca" full="Català"/>
    <alternatives>
      <alternative code="ca@valencian" full="Valencià"/>
      <alternative code="ca" full="Català"/>
    </alternatives>
  </header>
 
  ...

Alternatives

• Endowing the e element with a alt (variant) attribute (currently v in some dictionaries and aversion [sic] in others), so that the corresponding metadix entry will go to the generated .dix only if that alternative or variant is selected (entries without an alt will go to the .dix unconditionally); this will replace the use of the v or aversion attributes as found in the es-ca dictionaries to treat the Valencian dialect or in the es-pt dictionaries to distinguish European and Brazilian Portuguese, or in the oc-ca dictionaries to distinguish the Aranese variety. Here is an es-ca monodix example:

       <e alt="ca">
         <p>
           <l>haguéssim</l>
           <r>haver<s n="vbhaver"/><s n="pis"/><s n="p1"/><s n="pl"/><j/></r>
         </p>
       </e>

       <e alt="ca@valencian">
         <p>
           <l>haguérem</l>
           <r>haver<s n="vbhaver"/><s n="pis"/><s n="p1"/><s n="pl"/><j/></r>
         </p>
       </e>  

And an es-pt monodix example

    <e lm="correto" alt="pt_BR">
        <i>corret</i>
        <par n="abert/o__adj"/>
      </e>
      
      <e lm="correcto" alt="pt_PT">
        <i>correct</i>
        <par n="abert/o__adj"/>
      </e>

In structural transfer rule files, rule's may also have variants as observed in the es-pt package. The proposal is to endow element rule with the alt attribute (currently v or aversion):

• having a way to mark a group of entries as belonging to a certain variant. To that end, a wrapping element e-group (name in discussion; currently named aversion [sic]) would contain a set of entries and this will be equivalent to having all these entries marked with a certain value of the alt attribute so that writing

<e-group alt="xx">
<e> ... </e>
<e> ... </e>
</e-group>

would be equivalent to having

<e alt="xx"> ... </e>
<e alt="xx"> ... </e>

This may be seen as a way to factor out a common value of alt. Perhaps one could extend the use of e-group to factor out other attributes such as r.

This could be extended to other linguistic data files such as structural transfer files (.t1x, .t2x, etc.). The corresponding element would be rule-group.

Probably it is a good idea for all possible values of alt to be previously declared in the header (see above).

A generalisation of this would be a wrap element having three attributes: element (the name of the element affected), attribute (the name of the attribute) and value (the value of the attribute), so that <wrap element="e" attribute="alt" value="xx"> would be equivalent to <e-group alt="xx"> and <wrap element="rule" attribute="alt" value="xx"> would be equivalent to <rule-group alt="xx">, but perhaps this is too general.

Parametrized paradigms

There are two kinds of parametrized paradigms, which may be called word form paradigms and symbol paradigms (word form paradigms are observed in the oc-ca pair and grammatical symbol paradigms are observed in the en-ca dictionaries).

Current usage of word form paradigms

Here is an example of word form paradigms as currently used in the oc-ca dictionaries:

<pardef n="ab/i[T]ar__vblex">
        <e aversion="oc@aran-ca">
                <p>
                        <l>í</l>
                        <r>i</r>
                </p>
                <i><prm/></i>
                <par n="cànt/iga__vblex"/>
        </e>
        <e aversion="oc@aran-ca">
                <i>i<prm/></i>
                <par n="cant/ar__vblex"/>
        </e>
        <e aversion="oc-ca">
                <i>i<prm/></i>
                <par n="cant/as__vblex"/>
        </e>
     </pardef>

In the example, <prm/> will be substituted by the value of the single parameter, as in:

   <e lm="abitar">
                <i>ab</i>
                <par n="ab/i[T]ar__vblex" prm="t"/>
        </e>
        <e lm="abocinar">
                <i>aboc</i>
                <par n="ab/i[T]ar__vblex" prm="n"/>
        </e>
        <e lm="originar">
                <i>orig</i>
                <par n="ab/i[T]ar__vblex" prm="n"/>
        </e>
   
        <e lm="brilhar">
                <i>br</i>
                <par n="ab/i[T]ar__vblex" prm="lh"/>
        </e>

There's only a single unnamed parameter <prm/>, which may be a limitation in some applications. Don't be misled by the naming of paradigms: it isn't parsed, and it is just for the dictionary writer to remember what is substituted and kept.

Current usage of symbol paradigms

Here's an example of symbol paradigms as used in the en-ca dictionary.

 <pardef n="house__n">
   <e c="CP: nouns which add -s">
      <p>
         <l/>
         <r><s n="n"/><sa/><s n="sg"/></r>
      </p>
   </e>
   <e>
      <p>
         <l>s</l>
         <r><s n="n"/><sa/><s n="pl"/></r>
      </p>
   </e>
 </pardef>

Then, the placeholder <sa/> will be substituted by a symbol <s n="..."> named as the value of attribute sa:

<e lm="time"><i>time</i><par n="house__n" sa="unc"/></e>

This has the same limitations as in the case of word form parameters (single parameter, etc.). Furthermore, the use of the first and second method is heterogeneous and could be unified somehow.

A unifying proposal

Word-form parameters

Here are the two examples above in the new proposed notation, which will be explained below:

<pardef n="ab/i[T]ar__vblex" prm-list="cons">
        <e vnt="oc@aran">
                <p>
                        <l>í</l>
                        <r>i</r>
                </p>
                <i><txt-prm n="cons"/></i>
                <par n="cànt/iga__vblex"/>
        </e>
        <e vnt="oc@aran">
                <i>i<txt-prm n="cons"/></i>
                <par n="cant/ar__vblex"/>
        </e>
        <e vnt="oc">
                <i>i<txt-prm n="cons"/></i>
                <par n="cant/as__vblex"/>
        </e>
     </pardef>

And the call:

   <e lm="abitar">
                <i>ab</i>
                <par n="ab/i[T]ar__vblex" prms="cons='t'"/>
        </e>
        <e lm="abocinar">
                <i>aboc</i>
                <par n="ab/i[T]ar__vblex" prms="cons='n'"/>
        </e>
        <e lm="originar">
                <i>orig</i>
                <par n="ab/i[T]ar__vblex" prms="cons='n'"/>
        </e>
   
        <e lm="brilhar">
                <i>br</i>
                <par n="ab/i[T]ar__vblex" prms="cons='lh'"/>
        </e>

The use of named parameters allows for a simple mechanism for multiple parameters. For instance the parameterized paradigm could have prm-list="cons vowel" and calls could have, for instance, prms="cons='lh' vowel='a'".

Symbol parameters

Here are the two symbol-parameter examples above in the new notation.

The definition of the paradigm:

 <pardef n="house__n" prm-list="countable">
   <e c="CP: nouns which add -s">
      <p>
         <l/>
         <r><s n="n"/><symbol-prm n="countable"/><s n="sg"/></r>
      </p>
   </e>
   <e>
      <p>
         <l>s</l>
         <r><s n="n"/><symbol-prm n="countable"/><s n="pl"/></r>
      </p>
   </e>
 </pardef>

And the call:

<e lm="time"><i>time</i><par n="house__n" prms="countable='unc'"/></e>

in a completely analogous way to word-form parameters. The only difference is that <symbol-prm n="x"> generates <s n="y"> if the value of x is y, whereas <txt-prm n="x"> generates just y.

Fran's summary

[with attribute names slightly changed to conform with the above]

This is a brief summary detailing the various ways that the new format could be used. These are all in different languages, but it could be that any pair of languages might use all these features.

<pardef n="ab/i[T]ar__vblex" prm-list="cons cons2">
  <e alt="oc@aran">
    <p>
      <l>í</l>
      <r>i</r>
    </p>
    <i><txt-prm n="cons"/><txt-prm n="cons2"/></i>
    <par n="cànt/iga__vblex"/>
  </e>
  <e>
    <i>i<txt-prm n="cons"/></i>
    <par n="cant/as__vblex"/>
  </e>
</pardef>

<pardef n="house__n" prm-list="count">
  <e>
    <p>
      <l/>
      <r><s n="n"/><symbol-prm n="count"/><s n="sg"/></r>
    </p>
  </e>
  <e>
    <p>
      <l>s</l>
      <r><s n="n"/><symbol-prm n="count"/><s n="pl"/></r>
    </p>
  </e>
</pardef>

Calling them:

  <e lm="house"><i>house</i><par n="house__n"/></e>
  <e lm="time"><i>time</i><par n="house__n" prms="count='unc'"/></e>
  <e lm="brilhar"><i>br</i><par n="ab/i[T]ar__vblex" prms="cons='lh' cons2='tx'"/></e>
  <e lm="aerodrom"><i>aerodrom</i><par n="aerodrom__n"/></e>
  <e lm="avion"><i>avion<b/>luka</i><par n="aerodrom__n"/></e>
  <e-group alt="sh_HR">
    <e lm="zrakoplov"><i>zrakoplov</i><par n="aerodrom__n"/></e>
    <e lm="zračna luka"><i>zračna<b/>luka</i><par n="luka__n"/></e>
  </e-group>
  <e lm="fajl" alt="sh_BS"><i>fajl</i><par n="aerodrom__n"/></e>
  <e lm="datoteka"><i>datoteka</i><par n="luka__n"/></e>

See also

• Proposal for multiple prm's at http://wiki.apertium.org/wiki/Talk:Metadix