Difference between revisions of "Talk:Apertium on Windows"

From Apertium
Jump to navigation Jump to search
 
(69 intermediate revisions by 5 users not shown)
Line 23: Line 23:


:::I'm merging the win32 stuff now... -- [[User:Jimregan|Jimregan]] 15:01, 25 November 2008 (UTC)
:::I'm merging the win32 stuff now... -- [[User:Jimregan|Jimregan]] 15:01, 25 November 2008 (UTC)


I have installed apertium on windows and it's working fine... i used the installer at http://sourceforge.net/projects/apertium/files/apertium-win32-installer/ApertiumSetup.exe/download
The following guide was really helpful: wiki.apertium.org/wiki/Apertium_guide_for_Windows_users -- [[User:anu1328|anu1328]] 03:10 09 November 2011 (IST)


== Windows not supported ? ==
== Windows not supported ? ==
Line 202: Line 206:
== Execution ==
== Execution ==


I compiled everything now (using the objs instead of the lib),
I compiled everything now (linking with the obj files instead of apertium3.lib),


$ lt-comp lr apertium-oc-ca.ca.dix ca-oc.automorf.bin
$ lt-comp lr apertium-oc-ca.ca.dix ca-oc.automorf.bin
Line 209: Line 213:


Is this a wrong dll ? [[User:Mlavaud|Mlavaud]] 00:42, 27 November 2008 (UTC)
Is this a wrong dll ? [[User:Mlavaud|Mlavaud]] 00:42, 27 November 2008 (UTC)

:I'm not sure if there is an equivalent in Windows, but in *nix to find the symbols of a library you can do:

<pre>
$ nm libxml2.a | grep xmlTextReaderGetParserLineNumber
00000000000008b0 T xmlTextReaderGetParserLineNumber
</pre>

:Try that. The libxml2 I'm using is 2.6.27. - [[User:Francis Tyers|Francis Tyers]] 06:39, 27 November 2008 (UTC)
::Thanks Francis, I need to compile it myself I think, I read somewhere that the options used make a difference, I'll do that later this week. [[User:Mlavaud|Mlavaud]] 11:37, 27 November 2008 (UTC)

:::Ok. Btw, I found an old version of lttoolbox compiled for Windows [http://ilazki.thinkgeek.co.uk/~spectre/lttoolbox-3.0.zip here], I'd forgotten about it, but it might work... - [[User:Francis Tyers|Francis Tyers]] 12:17, 27 November 2008 (UTC)
::::My lt-comp works with your libxml2.lib, this is interesting, thank you. [[User:Mlavaud|Mlavaud]] 22:52, 27 November 2008 (UTC)

:::::Can you show us the output of the compilation? and of running <code>lt-proc</code> with the test phrase as above? - [[User:Francis Tyers|Francis Tyers]] 07:52, 28 November 2008 (UTC)

== Crash ==
apertium_tagger is crashing in crtexe.c
<pre>
if (__native_startup_state == __initializing)
{
_initterm( __xc_a, __xc_z );
__native_startup_state = __initialized;
}
</pre>

I guess I need to setup a VS project to debug it ? [[User:Mlavaud|Mlavaud]] 23:37, 27 November 2008 (UTC)

:Hmm, how are you trying to run it? - [[User:Francis Tyers|Francis Tyers]] 07:53, 28 November 2008 (UTC)
::I ran you example above, it failed, also when I run it with no parameters. [[User:Mlavaud|Mlavaud]] 10:13, 28 November 2008 (UTC)

:::What was the output of lt-proc ? - [[User:Francis Tyers|Francis Tyers]] 10:33, 28 November 2008 (UTC)
::::Same as in your example, only apertium-tagger failed. [[User:Mlavaud|Mlavaud]] 11:12, 28 November 2008 (UTC)

Whoah. That's a pretty fundamental stage to get a crash in. The more details you can provide, the better. -- [[User:Jimregan|Jimregan]] 11:37, 28 November 2008 (UTC)
:It seems to be an initialization problem, I'll run it with the debugger to tell you what it is. [[User:Mlavaud|Mlavaud]] 11:39, 28 November 2008 (UTC)

::Yes, it is. That's the part of the CRT that takes care of initialising stdio, C++ objects etc. -- [[User:Jimregan|Jimregan]] 12:02, 28 November 2008 (UTC)

-- Didn't cmake set one up for you??? -- [[User:Jimregan|Jimregan]] 11:37, 28 November 2008 (UTC)
: Good question, I am not familiar with the command tools, I'll check this, anyway I see that useful to have a real VS project, it is also very convenient if someone wants to do some dev on it. [[User:Mlavaud|Mlavaud]] 11:52, 28 November 2008 (UTC)
:: CMake has a GUI :)

:: The first step in trying to compile Apertium in VC is to run CMake, which then generates the VS project files for you. Are we talking about something different, or have you skipped this step? -- [[User:Jimregan|Jimregan]] 12:02, 28 November 2008 (UTC)
:::I did this and it's compiling and linking now, but it's not exactly a VS project, it's just an [http://www.jurassicpark.com/ old style] make environment. [[User:Mlavaud|Mlavaud]] 12:12, 28 November 2008 (UTC)

What version of visual studio are you using? Because of limitations in Windows, you need to use a version that uses MSVCRT7. -- [[User:Jimregan|Jimregan]] 19:07, 28 November 2008 (UTC)

::::I will try to find a tool that converts a makefile to a DSW project (specific project file that gives access to all the good features of the VS IDE), otherwise I will continue to build it myself. <i>Endavant los Dracs !</i> [[User:Mlavaud|Mlavaud]] 16:18, 28 November 2008 (UTC)

== Debugging ==
CMake does that. Read the Wiki; that's all I did when I took over the Windows work -- [[User:Jimregan|Jimregan]] 19:07, 28 November 2008 (UTC)
:Yes the makefile works fine, could (I mean did you have a chance to) you run a debugging session and how exactly ? [[User:Mlavaud|Mlavaud]] 10:25, 29 November 2008 (UTC)

::Ok; here's the disconnect, I think. I use CMake to generate VS project files, not a makefile: that's the method Wynand set up for, it's the method I followed, and it's the only method anyone else who has shown interest in Win32 has - until you :) - tried to use, so it's the only way we can support. CMake comes with a GUI, just use the list to select Visual Studio project files. If you're version of VS isn't supported, we most likely can't help you - CMake generates project files for every version of VS that's able to compile Apertium. -- [[User:Jimregan|Jimregan]] 14:17, 29 November 2008 (UTC)
::: I understand that, I don't need a VS version of Apertium, I just need a functional exe to start building my fr-oc files... Did someone show any interest in debugging the exe under Windows when it's going wrong ? I have no time to build a VS version so either I find a debugger that lets me find where the problem is (likely to be in my quick changes related to the array dimensions), either I try some [http://en.wikipedia.org/wiki/Amoeba very old style] "printf debugging". No way I resign, this translator looks too good and it will be perfect as soon as it gets its fr-oc implementation. ;) [[User:Mlavaud|Mlavaud]] 15:03, 29 November 2008 (UTC)

== Running ==
Ok Francis, Jim, now the 3 tests given by Francis work and produce the good results, it was only a mistake (of mine) in the makefile, what should I test now ? Thanks. [[User:Mlavaud|Mlavaud]] 00:11, 30 November 2008 (UTC)

<pre>
echo "Fe no es esperar" | lt-proc ca-oc.automorf.bin | apertium-tagger -g ca-oc.prob
"^Fe<n><f><sg>$ ^no<adv>$ ^es<prn><pro><ref><p3><mf><sp>$ ^esperar<vblex><inf>$"
</pre>

:<code>apertium-transfer</code> is the next part of the pipeline, and the next necessary part. It should work, unless I forgot to win32-ise it -- [[User:Jimregan|Jimregan]] 12:39, 30 November 2008 (UTC)
:: Thank you, I see that exciting to start building the oc stuff, an more later accumulate all exceptions we can express with the software to make translations as good as possible. A suggestion for the parameters to use with apertium-transfer ? [[User:Mlavaud|Mlavaud]] 13:57, 30 November 2008 (UTC)

:Great! Here you are (note: This looks more complicated than normal because the language pairs including Occitan allow for both oc and oc@aran):

<pre>
$ xsltproc --stringparam alt oc alt.xsl apertium-oc-ca.oc-ca.dix > oc-ca.dix # Extract the "general Occitan" parts from the Occitan--Catalan dictionary
$ lt-comp rl oc-ca.dix ca-oc.autobil.bin # Compile the dictionary to make a Catalan->Occitan transducer
$ xsltproc --stringparam alt oc alt.xsl apertium-oc-ca.ca-oc.t1x > ca-oc.t1x # Extract the "general Occitan" parts from the Catalan->Occitan transfer file
$ apertium-preprocess-transfer ca-oc.t1x ca-oc.t1x.bin # Perform pre-processing on the transfer file.

$ echo "Fe no es esperar" | lt-proc ca-oc.automorf.bin | apertium-tagger -g ca-oc.prob | apertium-transfer ca-oc.t1x ca-oc.t1x.bin ca-oc.autobil.bin
</pre>

:Output:

<pre>
^nom<SN><f><sg><ndf>{^Fe<n><2><3>$}$ ^adv<adv>{^pas<adv>$}$ ^pro<prnpro><ref><p3><mf><sp>{^se<prn><pro><ref><p3><mf><sp>$}$ ^inf<SN><vblex><inf><sg>{^esperar<vblex><inf>$}$
</pre>

:- [[User:Francis Tyers|Francis Tyers]] 17:05, 30 November 2008 (UTC)
:: in the example above "oc-ca.t1x" should be "oc-ca.dix" is that right ? apertium-preprocess-transfer fails on "pcre compiled with no utf8 support" whatever the pre.lib I try, so I need to go back to the compilation of pcre and activate the corresponding flag. [[User:Mlavaud|Mlavaud]] 11:30, 1 December 2008 (UTC)

:::Nope, the example above is correct. I'll add comments. - [[User:Francis Tyers|Francis Tyers]] 12:16, 1 December 2008 (UTC)
::::It says that the file 'oc-ca.t1x' doesn't exist, can you check again ? I was not able to improve my version of pcre, lazy me I searched for a version already compiled and I found one here: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/, I downloaded and compiled pcre-7.8.zip which has every files needed, I only set the flag PCRE_SUPPORT_UTF8 to ON, it produces a pcred.lib that works fine. [[User:Mlavaud|Mlavaud]] 22:53, 1 December 2008 (UTC)

So line 3 fails because 'oc-ca.t1x' doesn't exist, but line 4 gives the correct result. [[User:Mlavaud|Mlavaud]] 23:16, 1 December 2008 (UTC)

<pre>
echo "cal que neixin flors a cada instant" | lt-proc ca-oc.automorf.bin | apertium-tagger -g ca-oc.prob | apertium-transfer ca-oc.t1x ca-oc.t1x.bin ca-oc.autobil.bin

"^verb<SV><vbmod><pri><p3><sg>{^caler<vbmod><3><4><5>$}$ ^que<querel><mf><s
p>{^que<rel><an><mf><sp>$}$ ^verb<SV><vblex><imp><p2><pl>{^@néixer<vblex><3><4>
<5>$}$ ^nom<SN><f><pl><ndf>{^flor<n><2><3>$}$ ^a<PREP>{^a<pr>$}$ ^det_nom<SN><m>
<sg><ndf>{^cada<det><ind><mf><sp>$ ^instant<n><2><3>$}$"
</pre>

I updated the section <b>Building and installing pcre</b>. [[User:Mlavaud|Mlavaud]] 00:55, 2 December 2008 (UTC)

:Aha, I missed that, it should have been '''ca-oc.t1x''' in place of '''oc-ca.t1x'''. The output looks ok, although there appears to be an error with encodings: <code>^@néixer<vblex><3><4><5>$</code>. The next step will be:

<pre>
$ xsltproc --stringparam alt oc alt.xsl apertium-oc-ca.ca-oc.t2x > ca-oc.t2x
$ apertium-preprocess-transfer ca-oc.t2x ca-oc.t2x.bin

$ echo "Fe no es esperar" | lt-proc ca-oc.automorf.bin | apertium-tagger -g ca-oc.prob | apertium-transfer ca-oc.t1x ca-oc.t1x.bin ca-oc.autobil.bin | apertium-interchunk ca-oc.t2x ca-oc.t2x.bin
</pre>

:- [[User:Francis Tyers|Francis Tyers]] 06:21, 2 December 2008 (UTC)

== MinGW (note to self) ==

I'm using the version of MinGW here: http://www.tdragon.net/recentgcc/
(because it supports GCC 4); specifically, 'On-Demand Installer: [tdm-mingw-1.812.0-webdl.exe]', with the option 'TDM-GCC Recommended, C/C++' selected.

Grab MSYS and MsysDTK from http://www.mingw.org (MSYS-1.0.10.exe, msysDTK-1.0.1.exe)

Grab PCRE [http://sourceforge.net/project/showfiles.php?group_id=214730&package_id=261982 here], and extract in <code>/mingw</code>

Grab pkgconfig, libxml2 and libxslt [http://sourceforge.net/project/showfiles.php?group_id=204414 here]. pkgconfig also needs glib and gettext installed, so grab them too. xsltproc needs ''their'' version of iconv, zlib, libgcrypt, and libgpg-error so install those (even if Msys already has them!!)

'''Do not install pkg-config in /bin''' (the Msys default) -- it simply won't work (a basic security precaution, presumably -- it refuses to process command line arguments).

I needed to add this command:
<code>export PKG_CONFIG_PATH=/lib/pkgconfig</code>

The pcre package from KDE/Win32 is missing a .pc file, so you need to save this as /lib/pkgconfig/pcre.pc

<pre>
# Package Information for pkg-config
prefix=/usr/i686-pc-mingw32/sys-root/mingw
exec_prefix=/usr/i686-pc-mingw32/sys-root/mingw
libdir=/usr/i686-pc-mingw32/sys-root/mingw/lib
includedir=/usr/i686-pc-mingw32/sys-root/mingw/include
modules=1

Name: libpcre
Description: PCRE - Perl compatible regular expressions C library
Version: 7.8
Libs: -L/mingw/lib -lpcre
Cflags: -I/mingw/include
</pre>

<code>/bin/install</code> doesn't work on MSYS, at least, not for me, so after you run configure and make on lttoolbox:

<pre>
cp lttoolbox/liblttoolbox3.a /lib
cp lttoolbox/liblttoolbox3.lai /lib/liblttoolbox3.la
mkdir /include/lttoolbox
cp lttoolbox/*.h /include/lttoolbox
cp lttoolbox-3.1.pc /lib/pkgconfig
</pre>

Latest revision as of 21:37, 8 November 2011

Packaging a windows release?[edit]

Perhaps Cygwin license[1] would allow a packging of apertium and the required cygwin items to create a windows release?

Ote 22:55, 9 November 2007 (UTC)

It should be possible, but cygwin distribution must be optimized for size. I've tried compiling apertium with cygwin last week, but it gave some errors and could not advance further. I would also appreciate if someone had packaged cygwin with apertium and also supplied a batch file which enables translation with a simple double click under Windows environment.--Msalperen 09:14, 10 November 2007 (UTC)

I'm sorry, but you're wrong. It's not possible to build Apertium with any currently existing version of Cygwin. Cygwin's great, but it's not 100%. Cygwin 1.7 reputedly has the missing pieces of Unicode support that we need, but it doesn't as yet use GCC 4, which we also need. As soon as Cygwin 1.7 is available with GCC 4, I'll look into it again. -- Jimregan 18:05, 22 November 2008 (UTC)
Oh, and to answer the original question; the licence of Cygwin is not a problem. Apertium hasn't been packaged for cygwin due to technical issues -- Jimregan 18:07, 22 November 2008 (UTC)

Compilation[edit]

Is "valgrind" the correct version to use for Windows ? But next, directory Win32 is missing, on the other hand "branches\apertium" has a Win32 dir. but no lttoolbox. The page can be updated depending on the answer.

All of the Windows stuff is now merged into the trunk. We should really delete the branch[es]. - Francis Tyers 12:51, 22 November 2008 (UTC)
Correction: the win32 work on lttoolbox has all been merged - it should build without problems in VC++. The work on Apertium hasn't all been merged, and it's not yet complete. I'll get to it, I'm just not majorly motivated, because I despise Windows :) -- Jimregan 18:05, 22 November 2008 (UTC)
I think all the necessary stuff has been merged no ? A couple of weeks ago we guided a guy through compiling it on Windows, and he managed it without any patches. It might be worth looking at the mailing list posts, look up the thread [Apertium-stuff] Help on errors in Windows build. - Francis Tyers 18:10, 22 November 2008 (UTC)
I remember that thread. Nope, no magical fixes slipped in under my nose. Same problems exist. -- Jimregan 22:08, 22 November 2008 (UTC)
I'm merging the win32 stuff now... -- Jimregan 15:01, 25 November 2008 (UTC)


I have installed apertium on windows and it's working fine... i used the installer at http://sourceforge.net/projects/apertium/files/apertium-win32-installer/ApertiumSetup.exe/download The following guide was really helpful: wiki.apertium.org/wiki/Apertium_guide_for_Windows_users -- anu1328 03:10 09 November 2011 (IST)

Windows not supported ?[edit]

But the code I am trying to compile now will work right ? Mlavaud 18:32, 23 November 2008 (UTC)

Yes. What "not supported" means is that we don't really know _how_ it works, and can't offer the same level of support as on *nix. :) At least two or three people have got it working under Windows, so it is possible. - Francis Tyers 18:35, 23 November 2008 (UTC)
I will help fix things with pleasure if I can. Mlavaud 18:38, 23 November 2008 (UTC)
Cool thanks :) I think that at the moment the main thing we're lacking is documentation. So any details you can add or clarify we'd be very happy :) - Francis Tyers 18:54, 23 November 2008 (UTC)

First I need a few parameters to be correctly positioned: I have "#define MB_CUR_MAX ___mb_cur_max_func()" activated in stdlib.h and it is not compatible with "char symbol[MB_CUR_MAX+1];" in apertium_deshtml.cc -> "error constant needed", also I have "wchar_t cad[filename.size()];" in the same file, but an array definition needs a constant, or is there a parameter somewhere in Visual C++ for that ? Mlavaud 19:47, 23 November 2008 (UTC)

I'm not aware of any parameter for that, although... the deformatters aren't 100% necessary for the correct functioning of the system. I would recommend leaving them until last. - Francis Tyers 19:53, 23 November 2008 (UTC)
I got that stuff working in the win32 branch, no? I have a function to fix, because I did completely the wrong thing, but the deformatters should work. -- Jimregan 13:13, 25 November 2008 (UTC)

Fixed -- Jimregan 17:58, 26 November 2008 (UTC)

apertium_desodt.cc also has a line "wchar_t cad[filename.size()];" with same error (constant waited), probably many files will have the same problem. Mlavaud 20:07, 23 November 2008 (UTC)

I think only the files which are generated from flex (e.g. the de/re-formatters) will have this. - Francis Tyers 20:16, 23 November 2008 (UTC)
A problem at linking, how do I include the libraries needed ? Mlavaud 20:11, 23 November 2008 (UTC)
"apertium_desodt.cc
Linking CXX executable apertium-desodt.exe
apertium_desodt.cc.obj : error LNK2019: symbole externe non rÚsolu _regcomp rÚfÚ
rencÚ dans la fonction "void __cdecl init_escape(void)" (?init_escape@@YAXXZ)
apertium_desodt.cc.obj : error LNK2019: symbole externe non rÚsolu _regexec rÚfÚ
rencÚ dans la fonction "class std::basic_string<wchar_t,struct std::char_traits<
wchar_t>,class std::allocator<wchar_t> > __cdecl escape(class std::basic_string<
char,struct std::char_traits<char>,class std::allocator<char> > const &)" (?esca
pe@@YA?AV?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@ABV?$ba
sic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@2@@Z)
apertium-desodt.exe : fatal error LNK1120: 2 externes non résolus"
Can you show me which libraries it is linking (or trying to link) to ? - Francis Tyers
It needs regcomp and regexec, so I guess it needs a regex lib ? Well my \pcre\lib is empty, that is the problem I think. Mlavaud 20:20, 23 November 2008 (UTC)
Yes that could well be the problem. Did you install libpcre from source or from a binary package ? - Francis Tyers 20:24, 23 November 2008 (UTC)
A package found at http://gnuwin32.sourceforge.net/packages/pcre.htm, but no file pcrecpp.lib is provided, only a pcre-bcc.lib and it doesn't work more. Mlavaud 20:55, 23 November 2008 (UTC)
Try building PCRE from source then I guess... - Francis Tyers 20:58, 23 November 2008 (UTC)
I removed the ref. to pcrecpp.lib and it linked. Mlavaud 15:18, 24 November 2008 (UTC)
Ok, so long as it was in the de/re-formatters or tagger and not any of the transfer modules, it should work fine. - Francis Tyers 17:48, 24 November 2008 (UTC)

An idea to fix this error ? Mlavaud 19:33, 24 November 2008 (UTC)

C:\Apertium\trunk\apertium\apertium\collection.cc(70) : error C2440: 'initialisation' : impossible de convertir de 'std::_Tree<_Traits>::const_iterator' en 'std::_Tree<_Traits>::iterator' with
[ _Traits=std::_Tset_traits<int,std::less<int>,std::allocator<int>,false> ] Aucun constructeur n'a pu prendre le type de source, ou la résolution de la surcharge du constructeur était ambiguë
C:\Apertium\trunk\apertium\apertium\collection.cc(71) : error C2440: 'initialisation' : impossible de convertir de 'std::_Tree<_Traits>::const_iterator' en 'std::_Tree<_Traits>::iterator' with 
[ _Traits=std::_Tset_traits<int,std::less<int>,std::allocator<int>,false> ] Aucun constructeur n'a pu prendre le type de source, ou la résolution de la surcharge du constructeur était ambiguë
set<int>::const_iterator it = element[i]->begin() works, you know what ? I "despise" STL. ;) Mlavaud 20:52, 24 November 2008 (UTC)
Haha :) Did it work finally? - Francis Tyers 22:00, 24 November 2008 (UTC)
not yet, searching for isinf() definition now. Mlavaud 22:04, 24 November 2008 (UTC)
This will do I hope. Mlavaud 22:33, 24 November 2008 (UTC)
#define isinf(x) ((_fpclass(x) == _FPCLASS_PINF) || (_fpclass(x) == _FPCLASS_NINF))
#define isnan(x) _isnan(x)
This, IIRC, is also fixed in the win32 branch. You're not the first person to suggest something like this - though, also IIRC, there is already an equivalent of isinf() in Windows land. And by the way, the correct way to add those kinds of definitions is to add an #ifdef MSCVER or whatever :P --Jimregan 13:13, 25 November 2008 (UTC)

Fixed -- Jimregan 17:58, 26 November 2008 (UTC)

If someone ever compiled this program with Visual Studio, he should tell me how to allow vector<wstring> translation_window[window.size()]; because there too many arrays defined with a variable instead of a constant and I doubt I will be able to fix them all. Mlavaud 22:55, 24 November 2008 (UTC)

I've emailed one of the guys who managed it before, hopefully he will reply here soon... although I suspect that he changed them all, unless there is a flag to Visual Studio? - Francis Tyers 06:27, 25 November 2008 (UTC)
Yes, a VS flag would be nice. Mlavaud 08:55, 25 November 2008 (UTC)
I try this: vector<wstring> *translation_window = new vector<wstring>[window.size()]; [...] delete translation_window;

"if (wcstok(news, delim, &ptr))" -> "if (ptr = wcstok(news, delim))"

strcasecmp() -> strcmpi()

Those are also fixed in the win32 branch. There are still other problems remaining though. One thing is that Windows does something stupid with wstring, and has to be told specifically to open files as unicode. That's fixed in a lot of parts of Apertium, but I'm not sure if I got them all. The other thing is that something gets broken in the tagger in the changes from arrays to vectors, and I still don't know exactly what's happening there -- Jimregan 13:13, 25 November 2008 (UTC)

Fixed -- Jimregan 18:35, 26 November 2008 (UTC)

4 exe compile now but the next one needs the full pcre to link I think, I will have to recompile it first.

Great! Which ones compile ? It would be interesting to see if those that compile work as expected as they have done in the past. - Francis Tyers 10:57, 25 November 2008 (UTC)
The first four in the makefile, I'll check this tonight and I'll do the testing, can you suggest something ? Mlavaud 11:18, 25 November 2008 (UTC)

By the four files you mean: apertium-pretransfer apertium-tagger apertium-preprocess-transfer apertium-filter-ambiguity ? PS. Looking at the Makefile you can remove apertium-lextor, it isn't used at the moment. For testing, how about trying analysis and tagging. Grab the SVN of oc-ca, and do:

$ lt-comp lr apertium-oc-ca.ca.dix ca-oc.automorf.bin
apostrophes@postblank 1001 1543
final@inconditional 28 537
main@standard 36360 66910

$ echo "Això és una prova" | lt-proc ca-oc.automorf.bin 
^Això/Això<prn><tn><p3><nt>$ ^és/ser<vbser><pri><p3><sg>$ ^una/un<num><f><sp>/un<det><ind><f><sg>$ ^prova/prova<n><f><sg>/provar<vblex><imp><p2><sg>/provar<vblex><pri><p3><sg>$

$ echo "Això és una prova" | lt-proc ca-oc.automorf.bin  | apertium-tagger -g ca-oc.prob 
^Això<prn><tn><p3><nt>$ ^ser<vbser><pri><p3><sg>$ ^un<det><ind><f><sg>$ ^prova<n><f><sg>$

- Francis Tyers 11:23, 25 November 2008 (UTC)

Grandmercé l'amic, aquò m'agrada fòrça. Mlavaud 11:40, 25 November 2008 (UTC)
de res ;) - Francis Tyers 11:56, 25 November 2008 (UTC)
I have got apertium-desodt.exe, apertium-desrtf.exe and apertium-destxt.exe so far, I compiled pcre in order to get pcrecpp.lib, and now the link of Apertium3.dll fails on finding "Compression" functions and many other ones, though lttoolbox3.lib is compiled and included, what can I do ? Mlavaud 00:03, 26 November 2008 (UTC)
Not sure it was included in the link, I added C:/Apertium/trunk/lttoolbox/lttoolbox/lttoolbox3.lib somewhere in the Apertium3.dll makefile and it goes much better, now _xmlFreeDoc is missing,maybe libxml2.lib ? Mlavaud 00:26, 26 November 2008 (UTC)
Yes, now missing pcre, tomorrow, enough today for the Drac. Mlavaud 00:31, 26 November 2008 (UTC)

Very nice Jim your fixes ! :) I added libs in the make file for apertium3.dll to compile:

/out:apertium3.dll /implib:apertium3.lib /pdb:C:\Apertium\trunk\apertium\apertium\apertium3.pdb /dll /version:0.0  /MANIFEST /STACK:10000000 /machine:I386 /debug /INCREMENTAL:YES kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib C:/Apertium/trunk/lttoolbox/lttoolbox/lttoolbox3.lib C:/LibXML/lib/libxml2.lib C:/pcre/lib/pcre.lib C:/pcre/lib/pcrecpp.lib C:/pcre/lib/pcreposix.lib

and below I also added:

apertium\CMakeFiles\apertium3.dir\requires: C:/Apertium/trunk/lttoolbox/lttoolbox/lttoolbox3.lib
apertium\CMakeFiles\apertium3.dir\requires: C:/LibXML/lib/libxml2.lib
apertium\CMakeFiles\apertium3.dir\requires: C:/pcre/lib/pcrecpp.lib
apertium\CMakeFiles\apertium3.dir\requires: C:/pcre/lib/pcreposix.lib
apertium\CMakeFiles\apertium3.dir\requires: C:/pcre/lib/pcre.lib

The dll was created but no apertium3.lib was generated so I used a tool to extract it from the dll. Mlavaud 21:02, 26 November 2008 (UTC)

I tried to compile apertium-filter-ambiguity.exe but the link doesn't find TSXReader and HMM for example (total 9 unresolved), they should be in apertium3.lib I think. How can I check the dll and lib files ? How can I have the makefile generate the lib automatically ? Mlavaud 23:03, 26 November 2008 (UTC)
Try to compile the tagger first (apertium-tagger). apertium-filter-ambiguity isn't used in most packages. - Francis Tyers 00:04, 27 November 2008 (UTC)

Dive right in[edit]

Dead link "dive right in" at top of page ? http://wiki.apertium.org/w/index.php?title={{{PAGENAME}}}&action=edit Mlavaud 12:37, 25 November 2008 (UTC)

Arreglat, gràcies! - Francis Tyers 13:41, 25 November 2008 (UTC)

.cc .cc.yy[edit]

I don't understand how it works, .cc.yy is built from .cc, but where does .cc come from and how can I make permanent changes in the code once for all (only locally on my PC I mean) ? Mlavaud 10:59, 26 November 2008 (UTC)

The .cc is generated using xsltproc and flex, for example:
$ /usr/bin/xsltproc deformat.xsl txt-format.xml > apertium_destxt.cctmp
$ /usr/bin/flex -Cfer -oapertium_destxt.cc apertium_destxt.cctmp
$ rm apertium_destxt.cctmp
- Francis Tyers 11:56, 26 November 2008 (UTC)
va plan ! Mlavaud 12:02, 26 November 2008 (UTC)

pcre[edit]

pcre is far from easy to compile, you could easily end by "despising" it ;) I suggest that someone at Apertium keeps a binary version of it for the users wanting to save time (time... my precious), I also update the page to explain how to compile it. Mlavaud 12:01, 26 November 2008 (UTC)

Ok, no problem. Just give me a link to the compiled package, and I'll host it here. Part of the problem is that none of us has Windows, so we can't do this ourselves. - Francis Tyers 12:08, 26 November 2008 (UTC)
òc-ben I'll do that after I check my pcre version on Apertium, now soon I hope. Mlavaud 16:12, 26 November 2008 (UTC)
Actually, I did have Windows... until my monitor blew up yesterday. Not that it was making much of a difference, because for some reason, VC++ refused to install. I have access to another Windows box, where it is installed, but in the meantime, I'm installing the mingw cross compiler in Linux, so we at least have a workable means of distributing windows binaries. -- Jimregan 17:53, 26 November 2008 (UTC)

There are precompiled versions of pcre here and here -- it'd be better to try those first, rather than hosting our own versions. -- Jimregan 18:13, 26 November 2008 (UTC)

Yes indeed, I'll try out your first link, the second one didn't work for me (not containing pcrecpp.lib). Mlavaud 20:10, 26 November 2008 (UTC)

Jim, if you can send the binaries for Windows, I am very interested ! Mlavaud 20:51, 26 November 2008 (UTC)

Execution[edit]

I compiled everything now (linking with the obj files instead of apertium3.lib),

$ lt-comp lr apertium-oc-ca.ca.dix ca-oc.automorf.bin

says "Entry point of xmlTextReaderGetParserLineNumber not found in libxml2.dll"

Is this a wrong dll ? Mlavaud 00:42, 27 November 2008 (UTC)

I'm not sure if there is an equivalent in Windows, but in *nix to find the symbols of a library you can do:
$ nm libxml2.a | grep xmlTextReaderGetParserLineNumber
00000000000008b0 T xmlTextReaderGetParserLineNumber
Try that. The libxml2 I'm using is 2.6.27. - Francis Tyers 06:39, 27 November 2008 (UTC)
Thanks Francis, I need to compile it myself I think, I read somewhere that the options used make a difference, I'll do that later this week. Mlavaud 11:37, 27 November 2008 (UTC)
Ok. Btw, I found an old version of lttoolbox compiled for Windows here, I'd forgotten about it, but it might work... - Francis Tyers 12:17, 27 November 2008 (UTC)
My lt-comp works with your libxml2.lib, this is interesting, thank you. Mlavaud 22:52, 27 November 2008 (UTC)
Can you show us the output of the compilation? and of running lt-proc with the test phrase as above? - Francis Tyers 07:52, 28 November 2008 (UTC)

Crash[edit]

apertium_tagger is crashing in crtexe.c

if (__native_startup_state == __initializing)
{
   _initterm( __xc_a, __xc_z );
   __native_startup_state = __initialized;
}

I guess I need to setup a VS project to debug it ? Mlavaud 23:37, 27 November 2008 (UTC)

Hmm, how are you trying to run it? - Francis Tyers 07:53, 28 November 2008 (UTC)
I ran you example above, it failed, also when I run it with no parameters. Mlavaud 10:13, 28 November 2008 (UTC)
What was the output of lt-proc ? - Francis Tyers 10:33, 28 November 2008 (UTC)
Same as in your example, only apertium-tagger failed. Mlavaud 11:12, 28 November 2008 (UTC)

Whoah. That's a pretty fundamental stage to get a crash in. The more details you can provide, the better. -- Jimregan 11:37, 28 November 2008 (UTC)

It seems to be an initialization problem, I'll run it with the debugger to tell you what it is. Mlavaud 11:39, 28 November 2008 (UTC)
Yes, it is. That's the part of the CRT that takes care of initialising stdio, C++ objects etc. -- Jimregan 12:02, 28 November 2008 (UTC)

-- Didn't cmake set one up for you??? -- Jimregan 11:37, 28 November 2008 (UTC)

Good question, I am not familiar with the command tools, I'll check this, anyway I see that useful to have a real VS project, it is also very convenient if someone wants to do some dev on it. Mlavaud 11:52, 28 November 2008 (UTC)
CMake has a GUI :)
The first step in trying to compile Apertium in VC is to run CMake, which then generates the VS project files for you. Are we talking about something different, or have you skipped this step? -- Jimregan 12:02, 28 November 2008 (UTC)
I did this and it's compiling and linking now, but it's not exactly a VS project, it's just an old style make environment. Mlavaud 12:12, 28 November 2008 (UTC)

What version of visual studio are you using? Because of limitations in Windows, you need to use a version that uses MSVCRT7. -- Jimregan 19:07, 28 November 2008 (UTC)

I will try to find a tool that converts a makefile to a DSW project (specific project file that gives access to all the good features of the VS IDE), otherwise I will continue to build it myself. Endavant los Dracs ! Mlavaud 16:18, 28 November 2008 (UTC)

Debugging[edit]

CMake does that. Read the Wiki; that's all I did when I took over the Windows work -- Jimregan 19:07, 28 November 2008 (UTC)

Yes the makefile works fine, could (I mean did you have a chance to) you run a debugging session and how exactly ? Mlavaud 10:25, 29 November 2008 (UTC)
Ok; here's the disconnect, I think. I use CMake to generate VS project files, not a makefile: that's the method Wynand set up for, it's the method I followed, and it's the only method anyone else who has shown interest in Win32 has - until you :) - tried to use, so it's the only way we can support. CMake comes with a GUI, just use the list to select Visual Studio project files. If you're version of VS isn't supported, we most likely can't help you - CMake generates project files for every version of VS that's able to compile Apertium. -- Jimregan 14:17, 29 November 2008 (UTC)
I understand that, I don't need a VS version of Apertium, I just need a functional exe to start building my fr-oc files... Did someone show any interest in debugging the exe under Windows when it's going wrong ? I have no time to build a VS version so either I find a debugger that lets me find where the problem is (likely to be in my quick changes related to the array dimensions), either I try some very old style "printf debugging". No way I resign, this translator looks too good and it will be perfect as soon as it gets its fr-oc implementation. ;) Mlavaud 15:03, 29 November 2008 (UTC)

Running[edit]

Ok Francis, Jim, now the 3 tests given by Francis work and produce the good results, it was only a mistake (of mine) in the makefile, what should I test now ? Thanks. Mlavaud 00:11, 30 November 2008 (UTC)

echo "Fe no es esperar" | lt-proc ca-oc.automorf.bin  | apertium-tagger -g ca-oc.prob
"^Fe<n><f><sg>$ ^no<adv>$ ^es<prn><pro><ref><p3><mf><sp>$ ^esperar<vblex><inf>$"
apertium-transfer is the next part of the pipeline, and the next necessary part. It should work, unless I forgot to win32-ise it -- Jimregan 12:39, 30 November 2008 (UTC)
Thank you, I see that exciting to start building the oc stuff, an more later accumulate all exceptions we can express with the software to make translations as good as possible. A suggestion for the parameters to use with apertium-transfer ? Mlavaud 13:57, 30 November 2008 (UTC)
Great! Here you are (note: This looks more complicated than normal because the language pairs including Occitan allow for both oc and oc@aran):
$ xsltproc --stringparam alt oc alt.xsl apertium-oc-ca.oc-ca.dix > oc-ca.dix # Extract the "general Occitan" parts from the Occitan--Catalan dictionary
$ lt-comp rl oc-ca.dix ca-oc.autobil.bin # Compile the dictionary to make a Catalan->Occitan transducer
$ xsltproc --stringparam alt oc alt.xsl apertium-oc-ca.ca-oc.t1x > ca-oc.t1x # Extract the "general Occitan" parts from the Catalan->Occitan transfer file
$ apertium-preprocess-transfer ca-oc.t1x ca-oc.t1x.bin # Perform pre-processing on the transfer file.

$ echo "Fe no es esperar" | lt-proc ca-oc.automorf.bin  | apertium-tagger -g ca-oc.prob | apertium-transfer ca-oc.t1x ca-oc.t1x.bin ca-oc.autobil.bin
Output:
^nom<SN><f><sg><ndf>{^Fe<n><2><3>$}$ ^adv<adv>{^pas<adv>$}$ ^pro<prnpro><ref><p3><mf><sp>{^se<prn><pro><ref><p3><mf><sp>$}$ ^inf<SN><vblex><inf><sg>{^esperar<vblex><inf>$}$ 
- Francis Tyers 17:05, 30 November 2008 (UTC)
in the example above "oc-ca.t1x" should be "oc-ca.dix" is that right ? apertium-preprocess-transfer fails on "pcre compiled with no utf8 support" whatever the pre.lib I try, so I need to go back to the compilation of pcre and activate the corresponding flag. Mlavaud 11:30, 1 December 2008 (UTC)
Nope, the example above is correct. I'll add comments. - Francis Tyers 12:16, 1 December 2008 (UTC)
It says that the file 'oc-ca.t1x' doesn't exist, can you check again ? I was not able to improve my version of pcre, lazy me I searched for a version already compiled and I found one here: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/, I downloaded and compiled pcre-7.8.zip which has every files needed, I only set the flag PCRE_SUPPORT_UTF8 to ON, it produces a pcred.lib that works fine. Mlavaud 22:53, 1 December 2008 (UTC)

So line 3 fails because 'oc-ca.t1x' doesn't exist, but line 4 gives the correct result. Mlavaud 23:16, 1 December 2008 (UTC)

echo "cal que neixin flors a cada instant" | lt-proc ca-oc.automorf.bin  | apertium-tagger -g ca-oc.prob | apertium-transfer ca-oc.t1x ca-oc.t1x.bin ca-oc.autobil.bin

"^verb<SV><vbmod><pri><p3><sg>{^caler<vbmod><3><4><5>$}$ ^que<querel><mf><s
p>{^que<rel><an><mf><sp>$}$ ^verb<SV><vblex><imp><p2><pl>{^@néixer<vblex><3><4>
<5>$}$ ^nom<SN><f><pl><ndf>{^flor<n><2><3>$}$ ^a<PREP>{^a<pr>$}$ ^det_nom<SN><m>
<sg><ndf>{^cada<det><ind><mf><sp>$ ^instant<n><2><3>$}$"

I updated the section Building and installing pcre. Mlavaud 00:55, 2 December 2008 (UTC)

Aha, I missed that, it should have been ca-oc.t1x in place of oc-ca.t1x. The output looks ok, although there appears to be an error with encodings: ^@néixer<vblex><3><4><5>$. The next step will be:
$ xsltproc --stringparam alt oc alt.xsl apertium-oc-ca.ca-oc.t2x > ca-oc.t2x 
$ apertium-preprocess-transfer ca-oc.t2x ca-oc.t2x.bin

$ echo "Fe no es esperar" | lt-proc ca-oc.automorf.bin  | apertium-tagger -g ca-oc.prob | apertium-transfer ca-oc.t1x ca-oc.t1x.bin ca-oc.autobil.bin | apertium-interchunk ca-oc.t2x ca-oc.t2x.bin
- Francis Tyers 06:21, 2 December 2008 (UTC)

MinGW (note to self)[edit]

I'm using the version of MinGW here: http://www.tdragon.net/recentgcc/ (because it supports GCC 4); specifically, 'On-Demand Installer: [tdm-mingw-1.812.0-webdl.exe]', with the option 'TDM-GCC Recommended, C/C++' selected.

Grab MSYS and MsysDTK from http://www.mingw.org (MSYS-1.0.10.exe, msysDTK-1.0.1.exe)

Grab PCRE here, and extract in /mingw

Grab pkgconfig, libxml2 and libxslt here. pkgconfig also needs glib and gettext installed, so grab them too. xsltproc needs their version of iconv, zlib, libgcrypt, and libgpg-error so install those (even if Msys already has them!!)

Do not install pkg-config in /bin (the Msys default) -- it simply won't work (a basic security precaution, presumably -- it refuses to process command line arguments).

I needed to add this command: export PKG_CONFIG_PATH=/lib/pkgconfig

The pcre package from KDE/Win32 is missing a .pc file, so you need to save this as /lib/pkgconfig/pcre.pc

# Package Information for pkg-config
prefix=/usr/i686-pc-mingw32/sys-root/mingw
exec_prefix=/usr/i686-pc-mingw32/sys-root/mingw
libdir=/usr/i686-pc-mingw32/sys-root/mingw/lib
includedir=/usr/i686-pc-mingw32/sys-root/mingw/include
modules=1

Name: libpcre
Description: PCRE - Perl compatible regular expressions C library
Version: 7.8
Libs: -L/mingw/lib -lpcre
Cflags: -I/mingw/include

/bin/install doesn't work on MSYS, at least, not for me, so after you run configure and make on lttoolbox:

cp lttoolbox/liblttoolbox3.a /lib
cp lttoolbox/liblttoolbox3.lai /lib/liblttoolbox3.la
mkdir /include/lttoolbox
cp lttoolbox/*.h /include/lttoolbox
cp lttoolbox-3.1.pc /lib/pkgconfig