Named entity recognition - Revision history

Shardulc at 16:52, 10 March 2018

2018-03-10T16:52:42Z

Francis Tyers at 22:07, 7 March 2018

2018-03-07T22:07:05Z

Unhammer: /* Disambiguating unknown proper nouns from known words */

2014-07-20T07:36:15Z

Disambiguating unknown proper nouns from known words

Unhammer: tocd→TOCD

2014-07-20T07:35:22Z

tocd→TOCD

Unhammer: apertium-pn-recogniser doesn't deal with the problem most explained here …

2014-07-20T07:35:10Z

apertium-pn-recogniser doesn't deal with the problem most explained here …

Bech: Category:Documentation in English

2012-03-24T11:40:31Z

Category:Documentation in English

Unhammer: from the README

2009-09-08T14:38:43Z

from the README

Francis Tyers: /* Examples */

2007-12-24T23:01:17Z

Examples

Francis Tyers at 23:00, 24 December 2007

2007-12-24T23:00:59Z

Francis Tyers at 23:00, 24 December 2007

2007-12-24T23:00:49Z

← Older revision		Revision as of 16:52, 10 March 2018
Line 1:		Line 1:
	{{Github-~~migration~~-~~check~~}}		{{Github-unmigrated-tool}}
	Named entity recognition is about recognising named entities, for example proper nouns, etc. in text.		Named entity recognition is about recognising named entities, for example proper nouns, etc. in text.

← Older revision		Revision as of 22:07, 7 March 2018
Line 1:		Line 1:
			{{Github-migration-check}}
	Named entity recognition is about recognising named entities, for example proper nouns, etc. in text.		Named entity recognition is about recognising named entities, for example proper nouns, etc. in text.

@@ Line 42: / Line 42: @@
 ==Disambiguating unknown proper nouns from known words==
-The module [[apertium-pn-recogniser]] detects proper nouns in the input and marks them as unknown words so that the rest of the modules in the pipeline do not process them. This avoids the common case of wrong translations of source-language proper nouns which are also common nouns according to the dictionaries. The proper noun recogniser is mainly based on the one already included in the [[Freeling]] project.
+The module [[apertium-pn-recogniser]] detects proper nouns in the input and marks them as unknown words so that the rest of the modules in the pipeline do not process them. This avoids the common case of wrong translations of source-language proper nouns which are also common nouns (or other parts-of-speech) according to the dictionaries. The proper noun recogniser is mainly based on the one already included in the [[Freeling]] project.
 The proper noun recogniser must be invoked between the tagger and the transfer modules. Option -p is needed in the tagger, so a version from Apertium greater or equal to 3.1.0 is needed.
@@ Line 48: / Line 48: @@
 Check out from SVN with:
  svn co https://svn.code.sf.net/p/apertium/svn/trunk/apertium-pn-recogniser
 ==Further reading==

@@ Line 3: / Line 3: @@
 There are several translation issues that can show up when there are ''unknown'' proper nouns in the input. One is that transfer rules that work on &lt;np&gt;-tagged words do not apply when the word is unknown. Another is that proper nouns can be ambiguous with other, known words, and thus be translated when they should stay untranslated.
-{{tocd}}
+{{TOCD}}
 ==Unknown proper nouns in transfer==

@@ Line 1: / Line 1: @@
+Named entity recognition is about recognising named entities, for example proper nouns, etc. in text.
-{{TOCD}}
-Named entity recognition is about recognising named entities, for example proper nouns, etc. in text. When working with long rules, one of the problems in having them applied can be proper nouns. For example, names, companies, places etc. that aren't in the dictionaries and thus are not analysed. So for example in a sentence like:
+There are several translation issues that can show up when there are ''unknown'' proper nouns in the input. One is that transfer rules that work on &lt;np&gt;-tagged words do not apply when the word is unknown. Another is that proper nouns can be ambiguous with other, known words, and thus be translated when they should stay untranslated.
+{{tocd}}
+==Unknown proper nouns in transfer==
+When working with long rules, one of the problems in having them applied can be proper nouns. For example, names, companies, places etc. that aren't in the dictionaries and thus are not analysed. So for example in a sentence like:
 * Die man het John gesien.
@@ Line 22: / Line 28: @@
 Which is less than ideal. What we need is something that can tag "John" as a proper noun (<code><np></code>), so that the rules may be applied in the appropriate fashion.
-==Examples==
+===Case handling===
 The problem becomes more acute in other language groups where proper nouns have cases. For example in Serbo-Croatian or Polish:
@@ Line 32: / Line 38: @@
 :Marijom → Marija{{fade|<np><ant><f><sg>&lt;ins&gt;}}
-==Pipeline==
+===Pipeline===
 It should probably go in between tagging and transfer, and work only on unknown words.
-==The Apertium proper noun recogniser==
+==Disambiguating unknown proper nouns from known words==
-This module detects proper nouns in the input and marks them as unknown words so that the rest of the modules in the pipeline do not process them. This avoids the common case of wrong translations of source-language proper nouns which are also common nouns according to the dictionaries. The proper noun recogniser is mainly based on the one already included in the [[Freeling]] project.
+The module [[apertium-pn-recogniser]] detects proper nouns in the input and marks them as unknown words so that the rest of the modules in the pipeline do not process them. This avoids the common case of wrong translations of source-language proper nouns which are also common nouns according to the dictionaries. The proper noun recogniser is mainly based on the one already included in the [[Freeling]] project.
 The proper noun recogniser must be invoked between the tagger and the transfer modules. Option -p is needed in the tagger, so a version from Apertium greater or equal to 3.1.0 is needed.
 Check out from SVN with:
- svn co https://apertium.svn.sourceforge.net/svnroot/apertium/trunk/apertium-pn-recogniser
+ svn co https://svn.code.sf.net/p/apertium/svn/trunk/apertium-pn-recogniser

← Older revision		Revision as of 11:40, 24 March 2012
Line 53:		Line 53:

	[[Category:Development]]		[[Category:Development]]
			[[Category:Documentation in English]]

← Older revision		Revision as of 23:00, 24 December 2007
Line 1:		Line 1:
			{{TOCD}}
	Named entity recognition is about recognising named entities, for example proper nouns, etc. in text. When working with long rules, one of the problems in having them applied can be proper nouns. For example, names, companies, places etc. that aren't in the dictionaries and thus are not analysed. So for example in a sentence like:		Named entity recognition is about recognising named entities, for example proper nouns, etc. in text. When working with long rules, one of the problems in having them applied can be proper nouns. For example, names, companies, places etc. that aren't in the dictionaries and thus are not analysed. So for example in a sentence like: