User:Mlforcada/Sandbox/basque - Revision history

Mlforcada at 10:21, 19 November 2008

2008-11-19T10:21:58Z

Mlforcada at 10:03, 19 November 2008

2008-11-19T10:03:14Z

Mlforcada at 09:58, 19 November 2008

2008-11-19T09:58:48Z

Mlforcada at 09:57, 19 November 2008

2008-11-19T09:57:32Z

Mlforcada at 09:57, 19 November 2008

2008-11-19T09:57:03Z

Mlforcada at 09:54, 19 November 2008

2008-11-19T09:54:08Z

Mlforcada at 09:52, 19 November 2008

2008-11-19T09:52:28Z

Mlforcada at 09:46, 19 November 2008

2008-11-19T09:46:34Z

Mlforcada at 09:44, 19 November 2008

2008-11-19T09:44:32Z

Mlforcada at 09:44, 19 November 2008

2008-11-19T09:44:14Z

@@ Line 68: / Line 68: @@
 ==== ''Simple'' NP chunks should be as complete as possible to minimize "word salad" ====
+Here is a list of "simple" NP chunks of systematically increased complexity. Those leading to wrong translations are marked (*):
-Simple
+* Softwarea.
+* Software traketsa.
+* Software traketsagoa.
+* Azken softwarea.
+* Azken software traketsa.
+* Azken software traketsagoa (**)
+* Hizkuntz softwarea.
+* Azken hizkuntz softwarea.
+* Hizkuntz software traketsa. (*)
+* Hizkuntz software traketsagoa. (*)
+* Azken hizkuntz software traketsa. (*)
+* Azken hizkuntz software traketsagoa. (*)
+Their corresponding genitives are tested in the context ("& etorkizuna"), where the "&" is ignored by the system. Note that some NPs are treated correctly above but not here when they are genitives:
+* Softwarearen & etorkizuna.
+* Software traketsaren & etorkizuna.
+* Software traketsagoaren & etorkizuna. (*)
+* Azken softwarearen & etorkizuna.
+* Azken software traketsaren & etorkizuna.
+* Azken software traketsagoaren & etorkizuna. (*)
+* Hizkuntz softwarearen & etorkizuna.
+* Azken hizkuntz softwarearen & etorkizuna.
+* Hizkuntz software traketsaren & etorkizuna (*)
+* Hizkuntz software traketsagoaren & etorkizuna. (*)
+* Azken hizkuntz software traketsaren & etorkizuna. (*)
+* Azken hizkuntz software traketsagoaren & etorkizuna. (*)
 ==== Other things missing from chunks ====

@@ Line 54: / Line 54: @@
 The idea of having Apertium 2.0 was to curtail the proliferation of long, flat patterns, by defining chunks as building bricks or factors for a later interchunk operation. This does not increase the computational power of our structural transfer, but allows factoring long Apertium 1 patterns into shorter chunks and interchunk operations.
-Having '''long chunks''' extends the range of interchunk operations but at the cost of writing many long chunks. Focussing on those long chunks that appear frequently in corpora could be a possible compromise.
+Having '''long chunks''' extends the range of interchunk operations but at the cost of writing many long chunks. Focussing on those long chunks that appear frequently in corpora could be a possible compromise. I would not remove any existing chunk now.
 Having '''short chunks''' and rich interchunk operations makes the description of chunks simpler but reduces the range (length) of structural transfer operations.
@@ Line 65: / Line 65: @@
 * long chunks should be ready for use with interchunk rules (I think they are now)
-==== Missing things in chunks ====
+==== ''Simple'' NP chunks should be as complete as possible to minimize "word salad" ====
+Simple
+==== Other things missing from chunks ====
 Some errors can easily be corrected by extending category definitions:

@@ Line 9: / Line 9: @@
 === Regular vocabulary ===
-* Collect large corpora of basque news text and search for unknown words (as has been done for version 0.3)
+* Collect large corpora of basque news text and search for unknown words (as has been done for version 0.3). Newcomers can use the spreadsheets designed by Mireia to do so.
 * Using possible new vocabulary from the new version of Matxin (extracting it and converting it to our format).

@@ Line 60: / Line 60: @@
 We have to find a way to reconcile both, writing our interchunk operations in the most general way, so that they can operate on short NP / PP chunks and frequent but long NP/PP chunks. This could help reducing the size of structural transfer files (this causes problems with the current interpreter).
-For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have very long chunks such as <code>D_n_a_pr_d_n<SN></code>, <code>^D_n_pr_d_n_pr_d_n<SN></code>, </code>^Pr_d_n_a_pr_d_n<SPGEN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicitly formulated and reviewed:
+For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have very long chunks such as <code>D_n_a_pr_d_n<SN></code>, <code>^D_n_pr_d_n_pr_d_n<SN></code>, <code>^Pr_d_n_a_pr_d_n<SPGEN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicitly formulated and reviewed:
 * if a long chunk is included, it should be for frequency reasons, and be added in a special part of the rule file

@@ Line 38: / Line 38: @@
 We need to have paradigms for the potential ("ezan") and other verb structures. Perhaps we can use information in Matxin for this and other analytical verb forms.
-Having "verb chunks" (when they are continuous, which is sometimes not the case for negation) could allow us to generate a correct Spanish word order for some short sentences using interchunk operations (for instance NP-erg, NP-abs VP-nor-nork --> NP-erg VP-nor-nork NP-abs)
+Having "verb chunks" (when they are continuous, which is sometimes not the case for negation) and explicitly marked ergative NPs could allow us to generate a correct Spanish word order for some short sentences using interchunk operations (for instance NP-erg, NP-abs VP-nor-nork --> NP-erg VP-nor-nork NP-abs: currently there is no such rule "Gizonek ogia erosi dute" <math>\to</math> "Los hombres el pan han comprado")
 === Noun phrases and prepositional phrases ===

@@ Line 31: / Line 31: @@
 </pre>
 could detect a place name such as Tuscaloosa if the text contains Tuscaloosatik, with a regular expression entry (well, we would also have other endings like ''etik'' and ''dik''; thanks Fran).
 == Structural transfer ==
@@ Line 61: / Line 60: @@
 We have to find a way to reconcile both, writing our interchunk operations in the most general way, so that they can operate on short NP / PP chunks and frequent but long NP/PP chunks. This could help reducing the size of structural transfer files (this causes problems with the current interpreter).
-For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have very long chunks such as <code>D_n_a_pr_d_n<SN></code>, <code>^D_n_pr_d_n_pr_d_n<SN></code>, </code>^Pr_d_n_a_pr_d_n<SPGEN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicitly formulated and reviewed.
+For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have very long chunks such as <code>D_n_a_pr_d_n<SN></code>, <code>^D_n_pr_d_n_pr_d_n<SN></code>, </code>^Pr_d_n_a_pr_d_n<SPGEN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicitly formulated and reviewed:
+* if a long chunk is included, it should be for frequency reasons, and be added in a special part of the rule file
+* long chunks should be ready for use with interchunk rules (I think they are now)
 ==== Missing things in chunks ====
@@ Line 68: / Line 70: @@
 * ''Gizon zahar baten etxea'' <math>\to</math> ''La casa de un hombre viejo''.
-* ''Gizon zaharrago baten etxea.'' <math>\to</math> ''De un hombre más viejo la casa.''
+* ''Gizon zaharrago baten etxea.'' <math>\to</math> *''De un hombre más viejo la casa.''

← Older revision		Revision as of 09:46, 19 November 2008
Line 61:		Line 61:
	We have to find a way to reconcile both, writing our interchunk operations in the most general way, so that they can operate on short NP / PP chunks and frequent but long NP/PP chunks. This could help reducing the size of structural transfer files (this causes problems with the current interpreter).		We have to find a way to reconcile both, writing our interchunk operations in the most general way, so that they can operate on short NP / PP chunks and frequent but long NP/PP chunks. This could help reducing the size of structural transfer files (this causes problems with the current interpreter).

	For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have a long ~~chunk~~ <code>D_n_a_pr_d_n<SN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicitly formulated.		For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have very long chunks such as <code>D_n_a_pr_d_n<SN></code> or even <code>^D_n_pr_d_n_pr_d_n<SN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicitly formulated.

← Older revision		Revision as of 09:44, 19 November 2008
Line 61:		Line 61:
	We have to find a way to reconcile both, writing our interchunk operations in the most general way, so that they can operate on short NP / PP chunks and frequent but long NP/PP chunks. This could help reducing the size of structural transfer files (this causes problems with the current interpreter).		We have to find a way to reconcile both, writing our interchunk operations in the most general way, so that they can operate on short NP / PP chunks and frequent but long NP/PP chunks. This could help reducing the size of structural transfer files (this causes problems with the current interpreter).

	For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have a long chunk <code>D_n_a_pr_d_n<SN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be ~~explicit~~.		For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have a long chunk <code>D_n_a_pr_d_n<SN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicitly formulated.

@@ Line 30: / Line 30: @@
        </e>
 </pre>
-could detect a place name such as Tuscaloosa if the text contains Tuscaloosatik, with a regular expression entry (well, we would also have other endings like ''etik'' and ''dik'')
+could detect a place name such as Tuscaloosa if the text contains Tuscaloosatik, with a regular expression entry (well, we would also have other endings like ''etik'' and ''dik''; thanks Fran).
@@ Line 61: / Line 61: @@
 We have to find a way to reconcile both, writing our interchunk operations in the most general way, so that they can operate on short NP / PP chunks and frequent but long NP/PP chunks. This could help reducing the size of structural transfer files (this causes problems with the current interpreter).
-For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as:
+For instance, currently we have "complex" chunks such as <code>D_n_pr_d_n<SN></code> "gizonaren etxea" (genitive structure treated at the chunk stage without interchunk operation) but "simple" chunks followed by an interchunk operation in cases such as what happens with "gizon zaharraren etxea". Two chunks are detected (<code>Pr_det_nom_adj<SPGEN></code> and <code>det_nom<SN></code>) and then reordered by an interchunk rule. Similarly, we have a long chunk <code>D_n_a_pr_d_n<SN></code>. The criteria for doing things in the chunk phase or as interchunk operations should be explicit.