<div dir="ltr">Thank you Andrew.<div>Or, as a third choice, prevent users from viewing frequency lists. Is this possible in Cqpweb? I mean, setting privileges so as users can make queries and use all the CQP functions, except viewing frequency lists?</div><div>Thank you again,</div><div>Stefania</div><div class="gmail_extra"><br><div class="gmail_quote">2015-06-20 15:14 GMT+02:00 Hardie, Andrew <span dir="ltr"><<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">Hi Stefania,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">This is a known problem which arises from the fact that the available MySQL collations for sorting-and-merging strings considered “equal” even if they are not
do not match the case/diacritic folding in CWB.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">There are two choices made possible by the available collations: the behaviour you have currently, in which all accents and case distinctions are ignored when
collating; OR, a collation which doesn’t merge <i>anything</i>, i.e. it treats accented characters as distinct, but also treats case distinctions as significant.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">You can engage the latter mode under the “Corpus settings” option in the main screen menu. If you set “</span><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:black;background:#d5d5d5">Corpus
requires case-sensitive collation for string comparison and searches<span> </span></span><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">” to “yes”, you will switch the collation over to the case/diacritic-sensitive
mode. Please note well the warning about the need to rebuild all frequency tables.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">I have a long-term idea for a solution to this but unfortunately (a) I don’t know yet whether it will work, (b) even if does, it will take a long time to implement.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">The solution in question involves MySQL custom collations and the big open question is the impact they have on performance. If anyone has experience with custom
collations, your input here would be welcome.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">best<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">Andrew.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> [mailto:<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a>]
<b>On Behalf Of </b>Stefania Spina<br>
<b>Sent:</b> 20 June 2015 10:21<br>
<b>To:</b> cwb<br>
<b>Subject:</b> [CWB] problems with Cqpweb and frequency lists<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hello,<u></u><u></u></p>
<div>
<p class="MsoNormal">I have an Italian corpus indexed in Cqpweb (v3.1.13); the corpus is encoded in iso-8859-1.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">When I use frequency lists, it seems that accented and non-accented characters are not properly distinguished. For example, in the word frequency list, the word "è" combines the frequency values of "è" and "e", and the unaccented word "e"
is not included in the frequency list.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">This does not happen in the queries, where accented and non accented characters are perfectly distinguished.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Is there a way I can solve this problem?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thank you for your help,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Stefania<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Stefania Spina<br>
Università per Stranieri di Perugia<br>
Dipartimento di Scienze Umane e Sociali<br>
<a href="mailto:stefania.spina@unistrapg.it" target="_blank">stefania.spina@unistrapg.it</a><br>
<a href="https://unistrapg.academia.edu/StefaniaSpina" target="_blank">https://unistrapg.academia.edu/StefaniaSpina</a><u></u><u></u></p>
</div>
</div>
</div>
</div></div></div>
</div>
<br>_______________________________________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a><br>
<a href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb" rel="noreferrer" target="_blank">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Stefania Spina<br>Università per Stranieri di Perugia<br>Dipartimento di Scienze Umane e Sociali<br><a href="mailto:stefania.spina@unistrapg.it" target="_blank">stefania.spina@unistrapg.it</a><br><a href="https://unistrapg.academia.edu/StefaniaSpina" target="_blank">https://unistrapg.academia.edu/StefaniaSpina</a><br></div>
</div></div>