Xml grep

From Apertium
Revision as of 07:36, 16 May 2013 by Unhammer (talk | contribs)
Jump to navigation Jump to search

When working with xml, you'll often want to grep out an element that spans several lines. This can be hacked with awk or perl, but a more elegant solution is to use the parser in libxml2 (which is a requirement when installing apertium, so should be installed on your system already). This lets you use a simple version of XPath expressions to grep out full XML elements.

Specifying the full path and the full pardef name:

$ xmllint --xpath '/dictionary/pardefs/pardef[@n="gen__apos"]' apertium-eo-en.en.dix
<pardef n="gen__apos">
  <e>       <p><l/>          <r/></p></e>
  <e>       <p><l>'</l>         <r><j/>'<s n="gen"/></r></p></e>
</pardef>

But for dix files, it should be the same if you specify a relative path:

$ xmllint --xpath '//pardef[@n="gen__apos"]' apertium-eo-en.en.dix
<pardef n="gen__apos">
  <e>       <p><l/>          <r/></p></e>
  <e>       <p><l>'</l>         <r><j/>'<s n="gen"/></r></p></e>
</pardef>

You can also search for substrings by using the 'contains' function:

$ xmllint --xpath '//pardef[contains(@n,"_adj")]' apertium-eo-en.en.dix
<pardef n="expensive__adj">
  <e>       <p><l/>          <r><s n="adj"/></r></p></e>
</pardef>
<pardef n="ca__adj">…
# etc; gives all the adj pardefs


To get all c attributes:

$ xmllint --xpath '//@c' apertium-eo-en.en.dix

To get c attributes only from <e> elements:

$ xmllint --xpath '//e/@c' apertium-eo-en.en.dix

To get all attributes of the e element that has the lm "cake":

$ xmllint --xpath '//e[lm="cake"]/@*' apertium-eo-en.en.dix


To get the second dictionary section:

$ xmllint --xpath '/dictionary/section[2]/' apertium-eo-en.en.dix

(or section[position()=2])


Some corpora are formatted in XML and put e.g. the real text contents inside a particular element. Say the corpus puts all text inside <sentence> elements, you can grep them out with:

$ xmllint --xpath '*/sentence/text()' corpus.xml