I understand. <div>I just wanted to know if there were any available solutions..<div><br></div><div>As for better hardware, I am currently running cqp on pretty high-end hardware (xeon, 12gb ram) and, exept investing in SSDs in order to speed it up some more, there is not much room for improvement...</div>
<div><br></div><div>But still, I am aware that querying 1.5G words in 3-4 minutes is allready pretty cool and I thank you for making this the tool</div><div><br></div><div>regards</div><div>NT </div><div> </div><div><div>
<div><br><br><div class="gmail_quote">On Sun, Nov 11, 2012 at 10:16 PM, Hardie, Andrew <span dir="ltr"><<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">Better hardware?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">I know this sounds glib, but re-engineering CWB to make it multithreaded or to use ancillary database indexes would be a huge undertaking. Throwing better hardware
at the problem will almost certainly cost you less than the programmer time to rewrite large chunks of CWB from the ground up.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">best<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d">Andrew.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> [mailto:<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a>]
<b>On Behalf Of </b>Nikola Tulechki<br>
<b>Sent:</b> 11 November 2012 07:58<br>
<b>To:</b> Open source development of the Corpus WorkBench<br>
<b>Subject:</b> [CWB] cqp and very large corpora<u></u><u></u></span></p>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Hello<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I am using cqp with the *WAC corpora (1.5G words) and, while not prohibiting, response times are still in the minutes range. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Are there any ways to further speed-up the tool?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Multithreading? Indexes stored in RAM, in DB? <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">NT<u></u><u></u></p>
</div>
</div></div></div>
</div>
<br>_______________________________________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a><br>
<a href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
<br></blockquote></div><br></div></div></div></div>