<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Dear Gabriele,<br>
<br>
Bastien Kindt is reputed to lemmatize, and probably pos tag, Old
Greek texts (see contact below).<br>
He may accept to send you back graciously the lemmatized version of
the texts you would send him.<br>
We redirected at least 3 different TXM users to him but I don't know
the results.<br>
Please get in touch if you would like to discuss about that, we are
looking for a communautary<br>
solution to this need.<br>
<br>
All the best,<br>
Serge<br>
<br>
---<br>
Bastien KINDT<br>
Projet de Recherche en Lexicologie Grecque<br>
Institut orientaliste<br>
Collège Érasme<br>
Place Blaise Pascal, 1<br>
B-1348 Louvain-la-Neuve<br>
BELGIQUE<br>
tel.: 00 32 10 47 44 16<br>
<a class="moz-txt-link-abbreviated" href="mailto:bastien.kindt@student.uclouvain.be">bastien.kindt@student.uclouvain.be</a><br>
<a class="moz-txt-link-abbreviated" href="mailto:bastien.kindt@brepols.net">bastien.kindt@brepols.net</a><br>
<a class="moz-txt-link-freetext" href="http://tpg.fltr.ucl.ac.be">http://tpg.fltr.ucl.ac.be</a><br>
----<br>
<br>
<br>
Le 11/03/2015 05:58, Gabriele Brandolini a écrit :<br>
<span style="white-space: pre;">> <br>
> Dear Ruprecht, Andrew and Stefan<br>
> <br>
> I followed your issue about encoding Old Greek texts.<br>
> <br>
> I also would like to cwb encode texts in this language
expecially old<br>
> texts of the Fathers of the Church. But I've not yet got a
PoS tagger<br>
> for such a language. We just planned to work on it to train<br>
> TreeTagger. But as I know it isn't ready yet.<br>
> <br>
> Do you, Ruprecht, know if there is any available?<br>
> <br>
> About your list of greek words in your email of 14 31 I
noticed that<br>
> they are mostly uncorrect. As the initial letter (alfa or eta
or<br>
> epsilon) were dropped out with its accent and spirit. I don't
know if<br>
> this has something to do with the encoding error messages you
get. <br>
> Just I wanted to point out it, maybe it can be of any help.<br>
> <br>
> Good work and good luck!<br>
> <br>
> Gabriele<br>
> <br>
> Il 10/mar/2015 14:31 "Ruprecht von Waldenfels"<br>
> <<a class="moz-txt-link-abbreviated" href="mailto:ruprecht.waldenfels@gmx.net">ruprecht.waldenfels@gmx.net</a>
<a class="moz-txt-link-rfc2396E" href="mailto:ruprecht.waldenfels@gmx.net"><mailto:ruprecht.waldenfels@gmx.net></a>> ha<br>
> scritto:<br>
> <br>
> Dear List, so my second problem, this time with Ancient
Greek. I<br>
> cannot easily reproduce this with a 2-line corpus, because I
don't<br>
> know where the culprit is. I am posting the CWB Output
instead, maybe<br>
> this is already enough.<br>
> <br>
> What I am trying to do: I am trying to align three documents,
one<br>
> Greek and two Slavic texts, using the aligVerse structural
element.<br>
> The two Slavic ones align fine, the Greek gives me the
following<br>
> error: rvw@rvw-Latitude-E6410:/data/PROIEL$<br>
> /opt/CWBUTF8/cwb/utils/cwb-align -r /data/PROIEL/Registry -S<br>
> aligVerse -o out.align NTESTAMENT_GR NTESTAMENT_MN aligVerse
OPENING<br>
> NTESTAMENT_GR [147613 tokens, 7497 <aligVerse> regions]
OPENING<br>
> NTESTAMENT_MN [71935 tokens, 7497 <aligVerse> regions]
OPENING<br>
> prealignment [NTESTAMENT_GR.aligVerse: 7497 regions,<br>
> NTESTAMENT_MN.aligVerse: 7497 regions] LEXICON SIZE: 18085 /
10132 <br>
> FEATURE: character count, weight=1 ... [1] FEATURE: Shared
words,<br>
> threshold=40.0%, weight=50 ... [0] FEATURE: 3-grams, weight=3
... CL:<br>
> major error, invalid UTF8 string passed to
cl_string_canonical... CL:<br>
> major error, invalid UTF8 string passed to
cl_string_canonical... CL:<br>
> major error, invalid UTF8 string passed to
cl_string_canonical... <br>
> [21952] FEATURE: 4-grams, weight=4 ... CL: major error,
invalid UTF8<br>
> string passed to cl_string_canonical... CL: major error,
invalid UTF8<br>
> string passed to cl_string_canonical... CL: major error,
invalid UTF8<br>
> string passed to cl_string_canonical... CL: major error,
invalid UTF8<br>
> string passed to cl_string_canonical... [614656] [636609
features<br>
> allocated] [520402 entries in source text feature map]
[246622<br>
> entries in target text feature map] PASS 2: Setting character
count<br>
> weight. PASS 2: Processing shared words (th=40.0%). PASS 2:<br>
> Processing 3-grams. CL: major error, invalid UTF8 string
passed to<br>
> cl_string_canonical... CL: major error, invalid UTF8 string
passed to<br>
> cl_string_canonical... PASS 2: Processing 4-grams. CL: major
error,<br>
> invalid UTF8 string passed to cl_string_canonical... CL:
major error,<br>
> invalid UTF8 string passed to cl_string_canonical... PASS 2:
Creating<br>
> character counts. [checking pointers] ERROR: fcount1[1387]=24<br>
> r->w2f1[1388]-r->w2f1[1387]=22 w=``ἥξουσιν'' ERROR:
fcount1[1388]=50<br>
> r->w2f1[1389]-r->w2f1[1388]=52 w=``ἀνακλιθήσονται''
ERROR:<br>
> fcount1[1783]=24 r->w2f1[1784]-r->w2f1[1783]=22
w=``θάνατον'' ERROR:<br>
> fcount1[1784]=50 r->w2f1[1785]-r->w2f1[1784]=52
w=``ἐπαναστήσονται'' <br>
> ERROR: fcount1[3037]=20 r->w2f1[3038]-r->w2f1[3037]=16
w=``δυνατά'' <br>
> ERROR: fcount1[3039]=48 r->w2f1[3040]-r->w2f1[3039]=52<br>
> w=``ἀκολουθήσαντές'' ERROR: fcount1[3784]=20<br>
> r->w2f1[3785]-r->w2f1[3784]=18 w=``ἤλθατε'' ERROR:
fcount1[3785]=50<br>
> r->w2f1[3786]-r->w2f1[3785]=52 w=``ἀποκριθήσονται''
ERROR:<br>
> fcount1[4459]=32 r->w2f1[4460]-r->w2f1[4459]=30
w=``ἐπιθυμίαι'' <br>
> ERROR: fcount1[4460]=50 r->w2f1[4461]-r->w2f1[4460]=52<br>
> w=``εἰσπορευόμεναι'' ERROR: fcount1[4998]=20<br>
> r->w2f1[4999]-r->w2f1[4998]=18 w=``Ἤρξατο'' ERROR:
fcount1[4999]=46<br>
> r->w2f1[5000]-r->w2f1[4999]=48 w=``ἠκολουθήκαμέν''
ERROR:<br>
> fcount1[5038]=36 r->w2f1[5039]-r->w2f1[5038]=34
w=``ἐγγίζουσιν'' <br>
> ERROR: fcount1[5039]=50 r->w2f1[5040]-r->w2f1[5039]=52<br>
> w=``εἰσπορευόμενοι'' ERROR: fcount1[7009]=32<br>
> r->w2f1[7010]-r->w2f1[7009]=30 w=``πλουσίους'' ERROR:<br>
> fcount1[7010]=46 r->w2f1[7011]-r->w2f1[7010]=48
w=``ἀντικαλέσωσίν'' <br>
> ERROR: fcount1[8582]=20 r->w2f1[8583]-r->w2f1[8582]=18
w=``ἐξάγει'' <br>
> ERROR: fcount1[8583]=50 r->w2f1[8584]-r->w2f1[8583]=52<br>
> w=``ἀκολουθήσουσιν'' ERROR: fcount1[9942]=20<br>
> r->w2f1[9943]-r->w2f1[9942]=24 w=``ἅρματι'' ERROR:
fcount1[9943]=56<br>
> r->w2f1[9944]-r->w2f1[9943]=52 w=``ἀναγινώσκοντος''
ERROR:<br>
> fcount1[10119]=48 r->w2f1[10120]-r->w2f1[10119]=44<br>
> w=``μεταπέμψασθαί'' ERROR: fcount1[10120]=48<br>
> r->w2f1[10121]-r->w2f1[10120]=52 w=``εἰσκαλεσάμενος''
ERROR:<br>
> fcount1[10553]=28 r->w2f1[10554]-r->w2f1[10553]=24
w=``ἐτάραξαν'' <br>
> ERROR: fcount1[10554]=48
r->w2f1[10555]-r->w2f1[10554]=52<br>
> w=``ἀνασκευάζοντες'' ERROR: fcount1[10622]=24<br>
> r->w2f1[10623]-r->w2f1[10622]=20 w=``Τρῳάδος'' ERROR:<br>
> fcount1[10623]=48 r->w2f1[10624]-r->w2f1[10623]=52<br>
> w=``εὐθυδρομήσαμεν'' ERROR: fcount1[11159]=48<br>
> r->w2f1[11160]-r->w2f1[11159]=44 w=``ἀποσπασθέντας''
ERROR:<br>
> fcount1[11160]=52 r->w2f1[11161]-r->w2f1[11160]=56<br>
> w=``εὐθυδρομήσαντες'' ERROR: fcount1[12054]=20<br>
> r->w2f1[12055]-r->w2f1[12054]=18 w=``πλάνης'' ERROR:<br>
> fcount1[12055]=50 r->w2f1[12056]-r->w2f1[12055]=52<br>
> w=``ἀπολαμβάνοντες'' ERROR: fcount1[12422]=12<br>
> r->w2f1[12423]-r->w2f1[12422]=10 w=``νοός'' ERROR:
fcount1[12423]=50<br>
> r->w2f1[12424]-r->w2f1[12423]=52 w=``αἰχμαλωτίζοντά''
ERROR:<br>
> fcount1[14334]=40 r->w2f1[14335]-r->w2f1[14334]=38
w=``ἐπαιρόμενον'' <br>
> ERROR: fcount1[14335]=54
r->w2f1[14336]-r->w2f1[14335]=56<br>
> w=``αἰχμαλωτίζοντες'' ERROR: fcount1[14641]=40<br>
> r->w2f1[14642]-r->w2f1[14641]=38 w=``κεκυρωμένην''
ERROR:<br>
> fcount1[14642]=50 r->w2f1[14643]-r->w2f1[14642]=52<br>
> w=``ἐπιδιατάσσεται'' ERROR: fcount1[14878]=32<br>
> r->w2f1[14879]-r->w2f1[14878]=34 w=``προέγραψα'' ERROR:<br>
> fcount1[14879]=54 r->w2f1[14880]-r->w2f1[14879]=52<br>
> w=``ἀναγινώσκοντες'' ERROR: fcount1[15698]=36<br>
> r->w2f1[15699]-r->w2f1[15698]=34 w=``ἐπιστεύθην''
ERROR:<br>
> fcount1[15699]=46 r->w2f1[15700]-r->w2f1[15699]=48<br>
> w=``ἐνδυναμώσαντί'' ERROR: fcount1[16170]=32<br>
> r->w2f1[16171]-r->w2f1[16170]=30 w=``ἀνέξονται'' ERROR:<br>
> fcount1[16171]=50 r->w2f1[16172]-r->w2f1[16171]=52<br>
> w=``ἐπισωρεύσουσιν'' ERROR: fcount1[16815]=32<br>
> r->w2f1[16816]-r->w2f1[16815]=30 w=``ἐνυβρίσας'' ERROR:<br>
> fcount1[16816]=50 r->w2f1[16817]-r->w2f1[16816]=52<br>
> w=``Ἀναμιμνῄσκεσθε'' ERROR: fcount1[17621]=40<br>
> r->w2f1[17622]-r->w2f1[17621]=42 w=``ἀπεσταλμένα''
ERROR:<br>
> fcount1[17622]=56 r->w2f1[17623]-r->w2f1[17622]=54
w=``εἴκοσι<br>
> τέσσαρες'' ERROR: fcount1[17793]=28
r->w2f1[17794]-r->w2f1[17793]=29<br>
> w=``μάρτυσίν'' ERROR: fcount1[17794]=93<br>
> r->w2f1[17795]-r->w2f1[17794]=92 w=``χιλίας διακοσίας
ἑξήκοντα'' <br>
> ERROR: fcount1[17937]=24
r->w2f1[17938]-r->w2f1[17937]=26<br>
> w=``χαλινῶν'' ERROR: fcount1[17938]=60<br>
> r->w2f1[17939]-r->w2f1[17938]=58 w=``χιλίων ἑξακοσίων''
ERROR:<br>
> fcount1[17967]=36 r->w2f1[17968]-r->w2f1[17967]=34
w=``καυματίσαι'' <br>
> ERROR: fcount1[17968]=50
r->w2f1[17969]-r->w2f1[17968]=52<br>
> w=``ἐκαυματίσθησαν''<br>
> <br>
> <br>
> Again, I would be very thankful for help.<br>
> <br>
> Best! Ruprecht<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Am 10.03.2015 um 12:07 schrieb Ruprecht von Waldenfels:<br>
>> Hi Andrew, YES! This does solve the problem. I was
thinking this<br>
>> setting would only concern tokens, not the lemma
attribute, but now<br>
>> I understand that this was a wrong assumption. Thank you!
I will<br>
>> now look at the other problem - because that, as it turns
out, is<br>
>> unrelated. Thanks A LOT! Ruprecht Am 10.03.2015 um 12:02
schrieb<br>
>> Hardie, Andrew:<br>
>>> <br>
>>> Is the context size measured in characters? If so,
that would<br>
>>> explain the problem, since “characters” = bytes
still.<br>
>>> <br>
>>> <br>
>>> <br>
>>> If changing the context width to a given number of
words fixes<br>
>>> the issue, then that is the solution.<br>
>>> <br>
>>> <br>
>>> <br>
>>> I have been working on a patch to fix this, but have
not<br>
>>> completed it yet.<br>
>>> <br>
>>> <br>
>>> <br>
>>> Andrew.<br>
>>> <br>
>>> <br>
>>> <br>
>>> *From:*cwb-bounces@sslmit.unibo.it<br>
>>> <a class="moz-txt-link-rfc2396E" href="mailto:cwb-bounces@sslmit.unibo.it"><mailto:cwb-bounces@sslmit.unibo.it></a><br>
>>> [<a class="moz-txt-link-freetext" href="mailto:cwb-bounces@sslmit.unibo.it">mailto:cwb-bounces@sslmit.unibo.it</a>] *On Behalf Of
*Ruprecht von<br>
>>> Waldenfels *Sent:* 10 March 2015 09:54 *To:*
<a class="moz-txt-link-abbreviated" href="mailto:cwb@sslmit.unibo.it">cwb@sslmit.unibo.it</a><br>
>>> <a class="moz-txt-link-rfc2396E" href="mailto:cwb@sslmit.unibo.it"><mailto:cwb@sslmit.unibo.it></a> *Subject:* [CWB]
unicode problems<br>
>>> with Greek and OCS<br>
>>> <br>
>>> <br>
>>> <br>
>>> Dear List,<br>
>>> <br>
>>> I am using CWB 3.4.8 on 64 bit Ubuntu 14.10. After
encoding a<br>
>>> text in Old Church Slavonic, I get invalid UTF-8
character<br>
>>> errors; I seem to get them only in sgml mode (I also
get them<br>
>>> during alignment with the Ancient Greek translation
source, which<br>
>>> might be a related problem, but I am not sure.)<br>
>>> <br>
>>> In order to pinpoint the problem with the Old Church
Slavonic<br>
>>> text, I have reduced the text in question to two
bible verses.<br>
>>> The text can be found here:
<a class="moz-txt-link-abbreviated" href="http://www.parasolcorpus.org/test.txt">www.parasolcorpus.org/test.txt</a><br>
>>> <a class="moz-txt-link-rfc2396E" href="http://www.parasolcorpus.org/test.txt"><http://www.parasolcorpus.org/test.txt></a><br>
>>> <br>
>>> I encode the corpus with the following commands: <br>
>>> /opt/CWBUTF8/cwb/utils/cwb-encode -d
Data/ntestament_tt -f<br>
>>> test.txt -R /data/PROIEL/Registry/ntestament_tt -c
utf8 -xsB -P<br>
>>> lemma -P id -P alig -P pos -P tag -S aligVerse:0 <br>
>>> /opt/CWBUTF8/cwb/utils/cwb-makeall -r
/data/PROIEL/Registry<br>
>>> NTESTAMENT_TT<br>
>>> <br>
>>> There is no problem in text mode:<br>
>>> <br>
>>> <br>
>>> <br>
>>> However, in sgml mode, some lemmas get truncated and
do not<br>
>>> contain valid utf8 anymore. For example, the lemma of
"с҃вщаѩи"<br>
>>> is such a token. This problem does NOT appear if I
search for<br>
>>> this token itself, it ONLY and consistently appears
if I search<br>
>>> for a different token and the problematic token is in
the result<br>
>>> set:<br>
>>> <br>
>>> <br>
>>> To sum up: I get the problem only if I search for a
neighboring<br>
>>> token in sgml mode. I don't get it if I search for
the token<br>
>>> itself, and I don't get it in text mode. I have
reduced the<br>
>>> problem to w 50-token text, and the problem persists.<br>
>>> <br>
>>> Any help would be greatly appreciated! Best, Ruprecht<br>
>>> <br>
>>> <br>
>>> <br>
>>> <br>
>>> <br>
>>> _______________________________________________ CWB
mailing list <br>
>>> <a class="moz-txt-link-abbreviated" href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a>
<a class="moz-txt-link-rfc2396E" href="mailto:CWB@sslmit.unibo.it"><mailto:CWB@sslmit.unibo.it></a> <br>
>>> <a class="moz-txt-link-freetext" href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
>> <br>
>> <br>
>> <br>
>> _______________________________________________ CWB
mailing list <br>
>> <a class="moz-txt-link-abbreviated" href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a> <a class="moz-txt-link-rfc2396E" href="mailto:CWB@sslmit.unibo.it"><mailto:CWB@sslmit.unibo.it></a> <br>
>> <a class="moz-txt-link-freetext" href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
> <br>
> <br>
> _______________________________________________ CWB mailing
list <br>
> <a class="moz-txt-link-abbreviated" href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a> <a class="moz-txt-link-rfc2396E" href="mailto:CWB@sslmit.unibo.it"><mailto:CWB@sslmit.unibo.it></a> <br>
> <a class="moz-txt-link-freetext" href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
> <br>
> <br>
> <br>
> _______________________________________________ CWB mailing
list <br>
> <a class="moz-txt-link-abbreviated" href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a> <br>
> <a class="moz-txt-link-freetext" href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a></span><br>
<br>
<br>
-- <br>
Dr. Serge Heiden, <a class="moz-txt-link-abbreviated" href="mailto:slh@ens-lyon.fr">slh@ens-lyon.fr</a>, <a class="moz-txt-link-freetext" href="http://textometrie.ens-lyon.fr">http://textometrie.ens-lyon.fr</a><br>
ENS de Lyon/CNRS - ICAR UMR5191, Institut de Linguistique Française<br>
15, parvis René Descartes 69342 Lyon BP7000 Cedex, tél. +33622003883<br>
<br>
</body>
</html>