<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Teresa,<br>
<br>
You need to know how your corpus has been tokenized (segmented
into a sequence of tokens and punctuation marks to use your
terminology), which is a process done before and outside of CQP.<br>
If your corpus provides word properties giving information about
punctuation status or equivalent you should also be able to access
such information.<br>
If your corpus has no documentation about that, you should ask the
provider of the corpus.<br>
As a last resort, as an approximation at least for roman
languages, you can search your corpus for frequent words with a
short form.<br>
For example the most frequent words matching [word="."] are
globally punctuation marks, with some mix of grammatical words
(auxiliary, pronouns...).<br>
Then you can explore frequent words of length two: [word=".."],
etc.<br>
This is why I suggested to search for words of length longer than
one character: [word!="."]<br>
<br>
Best,<br>
Serge<br>
<br>
Le 18/10/2014 12:44, Teresa Molés Cases a écrit :<br>
</div>
<blockquote
cite="mid:238E2159-C903-4434-A4D9-54D672B202BC@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Hi Serge,
<div><br>
</div>
<div>Thank you a lot for your answer, but this query does not seem
to work in my corpus. Could you please tell me how can I get the
information about the surface forms of punctuations in my
corpus? If it is not much effort, of course.</div>
<div><br>
</div>
<div>Thanks a lot! Best,</div>
<div><br>
</div>
<div>Teresa</div>
<div><br>
</div>
<div><br>
<div>
<div>El 17/10/2014, a las 23:10, Serge Heiden <<a
moz-do-not-send="true" href="mailto:slh@ens-lyon.fr">slh@ens-lyon.fr</a>>
escribió:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi,<br>
<br>
Le 17/10/2014 20:28, Teresa Molés Cases a écrit :<br>
</div>
<blockquote
cite="mid:25135581-1827-4799-8A17-5BFA30E13CA2@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div>I have a question regarding the counting of tokens
in CQP. I know that the exact query would be <span
style="font-size: 11px;">DICKENS> Q1 = []; size
Q1;</span></div>
<div><br>
</div>
<div>But I have also read that this search would count
not only tokens but also punctuation marks. Is that
right?</div>
</blockquote>
Yes<br>
<blockquote
cite="mid:25135581-1827-4799-8A17-5BFA30E13CA2@gmail.com"
type="cite">
<div> Is it possible in CQP to count just tokens (not
including punctuation marks)?<br>
</div>
</blockquote>
Sure, just ask for something different from a punctuation
mark in your query instead of any "word"/token.<br>
For example : <span style="font-size: 11px;">DICKENS>
Q1 = [word!="."&word!="''|``"|word="[ai]"%c]; size
Q1;</span><br>
(to formulate such a query, you need to know the surface
forms of punctuations in your corpus)<br>
<br>
Of course it would be better if you run a tagger or a
syntactic analyzer on your sources before CQP to tel it<br>
what property could be used to filter punctuations (and
not only 'word' forms).<br>
<br>
You can also filter punctuations from the sources before
CQP encode and makeall, in which case your original query
will work.<br>
But a corpus without punctuation is difficult to read.
Another strategy is to have two versions of your corpus:
one with<br>
punctuations and one without, depending on the queries you
need to run.<br>
<br>
Best,<br>
Serge<br>
<pre class="moz-signature" cols="72">--
Dr. Serge Heiden, <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:slh@ens-lyon.fr">slh@ens-lyon.fr</a>, <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://textometrie.ens-lyon.fr/">http://textometrie.ens-lyon.fr</a>
ENS de Lyon/CNRS - ICAR UMR5191, Institut de Linguistique Française
15, parvis René Descartes 69342 Lyon BP7000 Cedex, tél. +33622003883</pre>
</div>
_______________________________________________<br>
CWB mailing list<br>
<a moz-do-not-send="true" href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a><br>
<a class="moz-txt-link-freetext" href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
</blockquote>
</div>
<br>
<div apple-content-edited="true">
<span class="Apple-style-span" style="border-collapse:
separate; color: rgb(0, 0, 0); font-family: Helvetica;
font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal;
orphans: 2; text-align: -webkit-auto; text-indent: 0px;
text-transform: none; white-space: normal; widows: 2;
word-spacing: 0px; -webkit-border-horizontal-spacing: 0px;
-webkit-border-vertical-spacing: 0px;
-webkit-text-decorations-in-effect: none;
-webkit-text-size-adjust: auto; -webkit-text-stroke-width:
0px; "><span class="Apple-style-span"
style="border-collapse: separate; color: rgb(0, 0, 0);
font-family: Helvetica; font-style: normal; font-variant:
normal; font-weight: normal; letter-spacing: normal;
line-height: normal; orphans: 2; text-align: -webkit-auto;
text-indent: 0px; text-transform: none; white-space:
normal; widows: 2; word-spacing: 0px;
-webkit-border-horizontal-spacing: 0px;
-webkit-border-vertical-spacing: 0px;
-webkit-text-decorations-in-effect: none;
-webkit-text-size-adjust: auto; -webkit-text-stroke-width:
0px; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode:
space; -webkit-line-break: after-white-space; ">
<div>Teresa Molés Cases</div>
<div>Traductora EN/DE/FR > ES/CAT</div>
<div><a moz-do-not-send="true"
href="mailto:teresamoles@gmail.com">teresamoles@gmail.com</a></div>
<div>667848390</div>
<div><br>
</div>
</div>
</span><br class="Apple-interchange-newline">
</span><br class="Apple-interchange-newline">
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
CWB mailing list
<a class="moz-txt-link-abbreviated" href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a>
<a class="moz-txt-link-freetext" href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Dr. Serge Heiden, <a class="moz-txt-link-abbreviated" href="mailto:slh@ens-lyon.fr">slh@ens-lyon.fr</a>, <a class="moz-txt-link-freetext" href="http://textometrie.ens-lyon.fr">http://textometrie.ens-lyon.fr</a>
ENS de Lyon/CNRS - ICAR UMR5191, Institut de Linguistique Française
15, parvis René Descartes 69342 Lyon BP7000 Cedex, tél. +33622003883</pre>
</body>
</html>