Apertium on Windows

From Apertium
Jump to navigation Jump to search

En français

⚠ WARNING ⚠

This page is out of date as a result of the migration to GitHub. Please update this page with new documentation and remove this warning. If you are unsure how to proceed, please contact the GitHub migration team.

This page is kept for historical purposes, if you want to install Apertium on Windows easily, see: Apertium_VirtualBox

Dragon.jpg

hic·sunt·dracones
Here be dragons! To make improvements, please dive right in. It is highly recommended that you hang out on the IRC channel (irc.oftc.net #apertium) and join the mailing list.


Introductory notes[edit]

Apertium is Unix software (this includes Mac OS/X). It's not formally supported on Windows - though we would all like it to run on Windows, it's not supported. It seems that Apertium is a difficult beast to get working, which is odd for us, because in a unix environment, it "just works".

Cygwin, as a 'Unix-like' environment for Windows, is an easy target for us, because it fully supports wstring (wide-char strings, widely used by Apertium) since version 1.7.

Lttoolbox builds with Visual C++ without additional changes - provided that you follow the instructions in README.Win32.

You will also need a Unix-type shell, such as bash, to run it. Batch files simply will not work, as Windows does not have true support for program pipes ('|').[1]. A solution can be using bash under Cygwin.

Getting the code[edit]

The lttoolbox changes are in the main SVN branch. Current win32 changes for apertium are in the branch apertium/win32

See instructions for using SVN, or the simplest way:

Building under Cygwin[edit]

Cygwin is a Unix-like environment and command-line interface for Microsoft Windows, which provides native integration of Windows-based applications, data, and other system resources with applications, software tools, and data of the Unix-like environment. Therefore it is possible to launch Windows applications from the Cygwin environment, as well as to use Cygwin tools and applications within the Windows operating context.

LTToolbox and Apertium can be easily built under Cygwin 1.7: first make sure you have all the needed dependencies; you can checkout LTToolbox' and Apertium's sources directly from Apertium's SVN repository by using Cygwin's shell with the following commands: 

svn co http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/lttoolbox lttoolbox
svn co http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/apertium apertium

Then, by using Cygwin's bash shell, go in LTToolbox' sources directory: 

cd lttoolbox

And execute the following sequence of commands: 

./autogen.sh --prefix=/usr
make
make install

Then go in Apertium's source directory and do the same: 

cd ../apertium
./autogen.sh --prefix=/usr
make
make install

Now you're ready to download and build language pairs and use them under Cygwin's shell.

Building under Visual C++[edit]

Dependencies[edit]

You will need to get Windows versions of the following tools and libraries to compile the code under Windows :

  • Microsoft's C++ compiler - the compiler is part of Visual Studio 2008 Express, which is available free of charge.
  • CMake - a cross platform build tool, download cmake-2.6.4-win32-x86.exe, at installation make sure you have CMakeSetup.exe in the bin directory.
  • libxml - the Windows binaries are at http://xmlsoft.org/sources/win32/; you will need:
    • libxml2 (install it in C:\Program Files\LibXML) from libxml2-2.7.6.win32.zip
    • zlib (install it in C:\Program Files\zlib) from zlib-1.2.3.win32.zip
    • iconv (install it in C:\Program Files\iconv) from iconv-1.9.2.win32.zip
    • libxslt (install it in C:\Program Files\libxslt) from libxslt-1.1.26.win32.zip
  • pcre - Download pcre-7.9.zip and unzip it (in C:\pcre for instance), next read Building and installing pcre below.
    • KDE/Win32 have packages for MinGW/MSVC here -- I haven't checked them yet
  • flex for Windows - download and use the Windows installer version (http://sourceforge.net/projects/gnuwin32/files/flex/).

Insert the libxml2, zlib and iconv bin directories in the path variable, for information they are needed by xsltproc.exe.

Setting up the required tools and libraries[edit]

Installing Visual Studio[edit]

Be sure to start downloading Visual Studio 2008 first, since it is a fairly big download.

Installing CMake[edit]

Next, install the Windows version of CMake. Let the installer place CMake in your path, since it will be much easier to run CMake from the command line if you do this.

Installing the libxml libraries[edit]

The zip files of libxml2, zlib, iconv and libxslt all have the familiar UNIX layout: 

|- bin
|- include
|- lib
|- share

Unzip each of these archives and copy the bin, include, lib and share directories in each package into a common directory (C:\Program Files\LibXML by default).

Building and installing pcre[edit]

You have unziped the pcre archive to a directory C:\pcre-7.9 (likely), create a diretory C:\pcre\build (for instance).

Edit the file CMakeLists.txt and update the following by replacing OFF by ON: 

SET(PCRE_SUPPORT_UTF8 OFF CACHE BOOL
    "Enable support for the Unicode UTF-8 encoding.")

To compile it, you will need to have Visual Studio AND CMake already installed on your system. Open the Visual Studio 8.0 Command Prompt (Start -> Microsoft Visual C++ 2008 Express Edition -> Visual Studio Tools -> Visual Studio 2008 Command Prompt). In this command prompt, go to the pcre source directory and run: 

cmakesetup .

Set the source code location (C:\pcre-7.9) and the build location (C:\pcre-7.9\build). Click on "Configure" and select "NMake Makefiles", CMake will show you a number of build variables, all highlighted in red, click "Configure" once more, if everything goes well the "OK" button will become active, hit it: as a result your "build" directory will be populated with a number of files for compiling the software.

To compile pcre, go to your build directory (C:\pcre-7.9\build) and run: 

nmake

When this is done, copy the file pcred.lib in your final pcre directory (C:\Program Files\pcre\lib for instance).

Installing flex[edit]

If you have downloaded the binary archive, unzip them into something like C:\Program Files\Flex. If you downloaded the Windows installer, then just follow the installation wizard.

Building lttoolbox[edit]

Before you can build lttoolbox, you need to tell CMake where to find the libxml files (under Linux, CMake will use pkg-config to find where the packages are installed). Open lttoolbox\CMakeLists.txt and modify the lines 

SET (LIBXML2_BASE        "C:/Program\ Files/LibXML")
SET (LIBXML2_INCLUDE_DIR "${LIBXML2_BASE}/include")
SET (LIBXML2_LIBRARIES   "${LIBXML2_BASE}/lib/libxml2.lib")

to match your installation.

Now, in the Visual Studio command prompt (again, Start -> Microsoft Visual C++ 2008 Express Edition -> Visual Studio Tools -> Visual Studio 2008 Command Prompt) change the directory to lttoolbox and execute 

cmakesetup .

Click on the configure button. As with libpcre, CMake will ask you to choose the type of build file to generate. Again, choose "Nmake". If you get an error message about LIBXML, then it means that LIBXML2_INCLUDE_DIR or LIBXML2_LIBRARIES in lttoolbox\CMakeLists.txt has an incorrect value.

If everything completes without errors after the first time that you clicked the "Configure" button, you will again see a few lines in the appear in the cmakesetup window. They should be highlighted with red. You have the opportunity to change the installation directory of lttoolbox by modifying the build variable CMAKE_INSTALL_PREFIX. To continue, click a second time on "Configure"; if everything is correct, then everything that was highlighted with red should now be highlighted with gray and the OK button becomes active.

Next, click OK; CMake should generate NMake files. To compile, simply execute: 

nmake

To install lttoolbox, run: 

nmake install

Building apertium[edit]

In order to build apertium, CMake needs to use the following tools and libraries:

  • libxml
  • libpcre
  • lttoolbox
  • xsltproc (you should have this since it is distributed with libxslt)
  • flex

At this point you should have all of the above and you simply need to set the correct CMake variables in order to build apertium. Open apertium/CMakeLists.txt (from the trunk directory) and find the section which should resemble the following: 

IF (WIN32)
        SET (WIN32_DIR ${PROJECT_SOURCE_DIR}/apertium/win32)
        INCLUDE_DIRECTORIES (${WIN32_DIR}; C:/apertium/trunk/lttoolbox; C:/pcre)
        LIST (APPEND EXTRA_SOURCES ${WIN32_DIR}/getopt.c ${WIN32_DIR}/libgen.c)

        SET(FLEX_EXECUTABLE        "${PROGRAM_FILES_PATH}/GnuWin32/bin/flex.exe")

        SET(LIBXML2_BASE_DIR       "${LIBXML2_BASE_DIR}/LibXML")
        SET(LIBXML2_INCLUDE_DIR    "${LIBXML2_BASE_DIR}/include")
        SET(LIBXML2_LIBRARIES      "${LIBXML2_BASE_DIR}/lib/libxml2.lib")
        
        SET(LIBXSLT_BASE_DIR       "${PROGRAM_FILES_PATH}/libxslt")
        SET(LIBXSLT_INCLUDE_DIR    "${LIBXSLT_BASE_DIR}/include")
        SET(LIBXSLT_LIBRARIES      "${LIBXSLT_BASE_DIR}/lib/libxslt.lib")
        SET(XSLTPROC_EXECUTABLE    "${LIBXSLT_BASE_DIR}/bin/xsltproc.exe")

        SET(LTTOOLBOX3_BASE_DIR    "${PROGRAM_FILES_PATH}/apertium-3.1")
        SET(LTTOOLBOX3_INCLUDE_DIR "${LTTOOLBOX3_BASE_DIR}/include/lttoolbox-3.1")
        SET(LTTOOLBOX3_LIBRARIES   "${LTTOOLBOX3_BASE_DIR}/lib/lttoolbox3.lib")

        SET(LIBPCRE_BASE_DIR       "${PROGRAM_FILES_PATH}/pcre")
        SET(LIBPCRE_INCLUDE_DIR    "${LIBPCRE_BASE_DIR}/include")
        SET(LIBPCRE_LIBRARIES      "${LIBPCRE_BASE_DIR}/lib/pcred.lib")

        ADD_DEFINITIONS (/D _CRT_SECURE_NO_WARNINGS /D STDC_HEADERS /D PCRE_STATIC)
ENDIF (WIN32)

The only variables you may need to adapt are:

  • INCLUDE_DIRECTORIES
  • FLEX_EXECUTABLE
  • LIBXML2_BASE_DIR
  • LIBXSLT_BASE_DIR
  • LTTOOLBOX3_BASE_DIR
  • LIBPCRE_BASE_DIR

These variables point to the base directories where the respective required tools and libraries are installed.

To build apertium, open the Visual Studio command prompt (as described earlier in this document) and change the directory to apertium. Now as before, run : 

cmakesetup .

Follow all of the build steps as described in the previous section which details how lttoolbox is built.

After the build completes, you should be able to install apertium by typing : 

nmake install

Testing apertium[edit]

Go to the directory occitan-catalan directory (likely to be C:\Apertium\trunk\apertium-oc-ca) and do the following tests: 

lt-comp lr apertium-oc-ca.ca.dix ca-oc.automorf.bin

expected results: 

apostrophes@postblank 1001 1543
final@inconditional 28 537
main@standard 36360 66910

try now: 

echo "Això és una prova" | lt-proc ca-oc.automorf.bin

expected results: 

^Això/Això<prn><tn><p3><nt>$ ^és/ser<vbser><pri><p3><sg>$ ^una/un<num><f><sp>/un<det><ind><f><sg>$ ^prova/prova<n><f><sg>/provar<vblex><imp><p2><sg>/provar<vblex><pri><p3><sg>$

next do: 

echo "Això és una prova" | lt-proc ca-oc.automorf.bin  | apertium-tagger -g ca-oc.prob

expected results: 

^Això<prn><tn><p3><nt>$ ^ser<vbser><pri><p3><sg>$ ^un<det><ind><f><sg>$ ^prova<n><f><sg>$

Notes[edit]

  1. ↑ Rather than pipe directly between processes, the 'DOS box' uses temporary files to emulate it: ls|more becomes the equivalent of ls>tmp;more<tmp
Retrieved from "https://wiki.apertium.org/w/index.php?title=Apertium_on_Windows&oldid=65981"