User:Popcorndude - Revision history

Popcorndude at 18:06, 23 December 2022

2022-12-23T18:06:57Z

Popcorndude: more ideas

2022-12-15T07:14:50Z

more ideas

Popcorndude at 18:38, 7 October 2021

2021-10-07T18:38:02Z

Popcorndude at 17:03, 22 February 2021

2021-02-22T17:03:41Z

Popcorndude: /* Things I don't know how to do */

2021-01-29T00:08:56Z

Things I don't know how to do

Popcorndude: /* Things I (think I) know how to do */

2021-01-29T00:08:03Z

Things I (think I) know how to do

Popcorndude at 22:11, 30 July 2020

2020-07-30T22:11:12Z

Popcorndude: /* Automate transition to -separable */ html escapes

2020-04-27T15:28:07Z

Automate transition to -separable: html escapes

Popcorndude: /* Things I know how to do */

2020-04-09T15:01:10Z

Things I know how to do

Popcorndude: /* Things I know how to do */ -separable

2020-03-28T16:42:23Z

Things I know how to do: -separable

@@ Line 59: / Line 59: @@
 Something that operates like <code>lt-proc -g</code> but tries all tags in the input at all possible points so that it doesn't matter where they're supposed to be (start, end, middle, reverse order, ...) as long as the right set is present. Probably helpful for the prefix-tags debate.
-=== Using -separable for Postgen ===
-Have <code>lsx-proc -p</code> read things like <code>^lemma&lt;tags&gt;/surface$</code> and output the same, with LUs in the rules matching either <code>surface</code> or <code>lemma&lt;tags&gt;/surface</code> (with literal <code>/</code> - shouldn't require an updates to the compiler), and output the same. Might have to restrict it to not adding or deleting anything, but not sure.
 == Things I don't know how to do ==
@@ Line 108: / Line 104: @@
 Pretty much all the main modules should handle non-ASCII characters correctly now.
+=== Using -separable for Postgen ===
+Have <code>lsx-proc -p</code> read things like <code>^lemma&lt;tags&gt;/surface$</code> and output the same, with LUs in the rules matching either <code>surface</code> or <code>lemma&lt;tags&gt;/surface</code> (with literal <code>/</code> - shouldn't require an updates to the compiler), and output the same. Might have to restrict it to not adding or deleting anything, but not sure.
+See [[Postgenerator]]
+=== Capitalization Restoration ===
+Move capitalization handling out of transfer and into a separate post-processor. See [[Capitalization restoration]].
 == Relevant Pages ==

@@ Line 51: / Line 51: @@
 Some sort of language model that takes a list of LUs and dependency relations and determines the appropriate order (and agreement info?) for them, possibly taking source order into account. In conjunction with a UD parser, this should produce at least somewhat functional transfer between any pair of languages with a bidix.
+=== Rule-Explainer ===
+Using https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_web it should be possible to make a web page that parses a rule file and explains what each piece is doing (bonus points for live updating and hovering over code or explanation to highlight the relevant section of the other one). Would make a nice teaching and debugging tool.
+=== Unordered Generation ===
+Something that operates like <code>lt-proc -g</code> but tries all tags in the input at all possible points so that it doesn't matter where they're supposed to be (start, end, middle, reverse order, ...) as long as the right set is present. Probably helpful for the prefix-tags debate.
+=== Using -separable for Postgen ===
+Have <code>lsx-proc -p</code> read things like <code>^lemma&lt;tags&gt;/surface$</code> and output the same, with LUs in the rules matching either <code>surface</code> or <code>lemma&lt;tags&gt;/surface</code> (with literal <code>/</code> - shouldn't require an updates to the compiler), and output the same. Might have to restrict it to not adding or deleting anything, but not sure.
 == Things I don't know how to do ==

@@ Line 29: / Line 29: @@
 This should be reasonably self-explanatory.
-=== UD parsing in -recursive ===
+=== UD parsing ===
+See below for notes on doing this with rtx, but really it would probably be easier to do it separately.
+==== UD parsing in -recursive ====
  DET @det:3 ADJ @amod:3 NOUN
@@ Line 40: / Line 44: @@
 (the double atted tags are to keep track of (e.g.) if you need a rule that applies to a noun with case marking)
-=== Unicode everywhere ===
+=== Transfer system based on UD ===
+Basically a system for writing tree/graph rewrite operations for the handful of pair-specific things that remain after implementing a UD parser and un-parser.
+=== UD un-parsing ===
+Some sort of language model that takes a list of LUs and dependency relations and determines the appropriate order (and agreement info?) for them, possibly taking source order into account. In conjunction with a UD parser, this should produce at least somewhat functional transfer between any pair of languages with a bidix.
+== Things I don't know how to do ==
+=== Better tokenization ===
+Multiwords and space-less orthographies.
 * https://github.com/apertium/organisation/issues/24
@@ Line 46: / Line 62: @@
 * [[User:Francis_Tyers/Apertium_4]]
 * [[Rethinking_tokenisation_in_the_pipeline]]
-== Things I don't know how to do ==
 === Learning transfer rules from small corpora ===
@@ Line 78: / Line 92: @@
 I really don't like XML or finite-state chunking, hence the new transfer module.
+=== Unicode everywhere ===
+Pretty much all the main modules should handle non-ASCII characters correctly now.
 == Relevant Pages ==

@@ Line 2: / Line 2: @@
 The rest of this page is my project list. Feel free to steal ideas from it, especially if you want to collaborate.
+Also, because prioritizing is hard, [[User:Popcorndude/What_should_Daniel_work_on]] exists so that other people can vote on what order I do this stuff in.
 == Things I (think I) know how to do ==

@@ Line 65: / Line 65: @@
 === Rule-based semantics ===
 If we used -recursive's tree output as the input to some sort of semantics system, could we do anything interesting with information extraction?
+=== See rule effect ===
+A tool that takes a rule in any module and finds every line in a corpus where it would apply.
 == Things I've already done ==

@@ Line 3: / Line 3: @@
 The rest of this page is my project list. Feel free to steal ideas from it, especially if you want to collaborate.
-== Things I know how to do ==
+== Things I (think I) know how to do ==
-=== Learning transfer rules from small corpora ===
-Given a syntactic parser for one language and a fairly small parallel corpus it seems like it should be possible to learn decent transfer rules. I'll be checking that hypothesis this summer.
 === Modernizing old pairs ===
@@ Line 32: / Line 28: @@
 == Things I don't know how to do ==
+=== Learning transfer rules from small corpora ===
+Given a syntactic parser for one language and a fairly small parallel corpus it seems like it should be possible to learn decent transfer rules. (This turns out to be a lot harder than I thought.)
 === Import data from FieldWorks ===
@@ Line 54: / Line 54: @@
 I really don't like XML or finite-state chunking, hence the new transfer module.
+== Relevant Pages ==
+[[Automated_extraction_of_lexical_resources]]

@@ Line 22: / Line 22: @@
 <code>
-<e lm="you can lead a horse to water but you can't make it drink">&lt;i>you<b/>can<b/>lead<b/>a<b/>horse<b/>to<b/>water<b/>but<b/>you<b/>can't<b/>make<b/>it<b/>drink</i><par n="hello__ij"/></e>
+&lt;e lm="you can lead a horse to water but you can't make it drink">&lt;i>you&lt;b/>can&lt;b/>lead&lt;b/>a&lt;b/>horse&lt;b/>to&lt;b/>water&lt;b/>but&lt;b/>you&lt;b/>can't&lt;b/>make&lt;b/>it&lt;b/>drink&lt;/i>&lt;par n="hello__ij"/>&lt;/e>
 </code>

@@ Line 26: / Line 26: @@
 It shouldn't be too hard to extract the multiwords from a monodix and convert them to -separable entries. The fact that they're in the monodix means they're contiguous so there's no information lost.
+=== Updating documentation ===
+This should be reasonably self-explanatory.
 == Things I don't know how to do ==

@@ Line 16: / Line 16: @@
 See [[Ideas_for_Google_Summer_of_Code/Bidix_lookup_and_maintenance]]. I'm much less certain about similar contributions to monodix. For transfer though you could probably make something that shows the tree that gets built and then have a drag-and-drop interface to fix errors.
+=== Automate transition to -separable ===
+Some monodixes have slightly horrifying multiwords in them, such as this one:
+<code>
+<e lm="you can lead a horse to water but you can't make it drink">&lt;i>you<b/>can<b/>lead<b/>a<b/>horse<b/>to<b/>water<b/>but<b/>you<b/>can't<b/>make<b/>it<b/>drink</i><par n="hello__ij"/></e>
+</code>
+It shouldn't be too hard to extract the multiwords from a monodix and convert them to -separable entries. The fact that they're in the monodix means they're contiguous so there's no information lost.
 == Things I don't know how to do ==