<div dir="ltr">Hi,<div><br></div><div>just adding my 2 cents - obviously these are just opinions of someone who has</div><div>built things on top of CQP's API</div><div><br></div><div>1) horizontal stability: I've seen time and again that this is necessary for any kind of fast access.</div><div><br></div><div>2) developing your own storage engine: My gut feeling says that it may be a good thing to rely</div><div> on existing work (e.g. Apache Parquet, MonetDB, WiredTiger etc.) if the goal is to support more complex</div><div> data structures than currently possible.</div><div> See e.g. this overview on techniques on columnar databases </div><div> <a href="http://www.cs.yale.edu/homes/dna/papers/abadi-column-stores.pdf">http://www.cs.yale.edu/homes/dna/papers/abadi-column-stores.pdf</a></div><div> WiredTiger is a high-performance embeddable database</div><div> <a href="http://source.wiredtiger.com/2.6.1/index.html">http://source.wiredtiger.com/2.6.1/index.html</a></div><div> There are also freely available implementations of query compilers, such as this one,</div><div> if you want to design the storage layer yourself.</div><div> <a href="https://github.com/uwescience/raco">https://github.com/uwescience/raco</a></div><div><br></div><div>3) scriptability: I'm not sure whether everyone really wants to learn Lua.</div><div> What I would like to see, though, is to have some public API that both allows</div><div> access to the query compiler/execution engine and to raw data columns (or</div><div> lists/vectors of structures), and that can be wrapped using the FFI of your</div><div> favorite language.</div><div><br></div><div>Best wishes,</div><div>Yannick</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 9, 2015 at 12:35 PM, Hardie, Andrew <span dir="ltr"><<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">Hi everyone,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">I’d like to draw your attention to a recent addition to the website, namely:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><a href="http://cwb.sourceforge.net/cwb4.php" target="_blank">http://cwb.sourceforge.net/cwb4.php</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">We’ve added this page as a point of public information for the work toward CWB v 4.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">As of now we would like to invite comments, questions and suggestions on the plans we have outlined for the future.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">The first proposal is for the new Ziggurat database engine. At the link above is posted an outline for how we intend this to work, including an explanation of the data model
and summary of the file format that implements it.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">If you’d to chip in on any aspect of the plan, please do so to this list (or, if you’d rather not speak publicly, to mine & Stefan’s email off-list). Plus any other questions
about where all this is going are welcome too. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">There is going to be a period of (probably) some months before serious work begins, due to the need to get all components to a stable point that we can branch before we break
things in the main line of development, so there is ample time for discussion!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">best<span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span></span></p><span class="HOEnZb"><font color="#888888">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">Andrew.<u></u><u></u></span></p>
</font></span></div>
</div>
<br>_______________________________________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a><br>
<a href="http://devel.sslmit.unibo.it/mailman/listinfo/cwb" rel="noreferrer" target="_blank">http://devel.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
<br></blockquote></div><br></div>