http://www.openlinksw.com/blog/vdb/blog/
OpenLink Virtuoso (Product Blog)
A great place to track Virtuoso's rapid evolution.
kidehen@openlinksw.com
kidehen@openlinksw.com
2013-06-19T11:06:53Z
Virtuoso Universal Server 06.04.3136
http://www.openlinksw.com/weblog/public/images/vbloglogo.gif
RDF and Transactions
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-22#1690
2011-03-22T22:52:56Z
2011-03-22T17:44:21-04:00
<p>I will here talk about <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x249bc940">RDF</a> and transactions for developers in general. The next one talks about specifics and is for specialists.</p> <p>Transactions are certainly not the first thing that comes to mind when one hears "RDF". We have at times used a recruitment questionnaire where we ask applicants to define a transaction. Many vaguely remember that it is a unit of work, but usually not more than that. We sometimes get questions from users about why they get an error message that says "deadlock". "Deadlock" is what happens when multiple users concurrently update balances on multiple bank accounts in the wrong order. What does this have to do with RDF?</p> <p>There are in fact users who even use XA with a <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x22c8dbc8">Virtuoso</a>-based RDF application. <a class="auto-href" href="http://semanticweb.org/id/Franz_Inc" id="link-id0x27bd0c08">Franz</a> also has publicized their development of full <a class="auto-href" href="http://dbpedia.org/resource/ACID" id="link-id0x283985c8">ACID</a> capabilities for <a class="auto-href" href="http://semanticweb.org/id/AllegroGraph" id="link-id0x238ba438">AllegroGraph</a>. RDF is a database <a class="auto-href" href="http://dbpedia.org/resource/Database_schema" id="link-id0x2864fef8">schema</a> model, and transactions will inevitably become an issue in databases.</p> <p>At the same time, the developer population trained with <a class="auto-href" href="http://dbpedia.org/resource/MySQL" id="link-id0x284d2d80">MySQL</a> and <a class="auto-href" href="http://dbpedia.org/resource/PHP" id="link-id0x237230e8">PHP</a> is not particularly transaction-aware. Transactions have gone out of style, declares the No-<a class="auto-href" href="http://dbpedia.org/resource/SQL" id="link-id0x2920cc88">SQL</a> crowd. Well, it is not so much SQL they object to but ACID, i.e., transactional guarantees. We will talk more about this in the next post. The <a class="auto-href" href="http://dbpedia.org/resource/SPARQL" id="link-id0x283f0588">SPARQL</a> language and protocol do not go into transactions, except for expressing the wish that an <code>UPDATE</code> request to an end-point be atomic. But beware -- atomicity is a gateway drug, and soon one finds oneself on full ACID. </p> <p>If one says that a thing will either happen <i>in its entirety</i> or <i>not at all,</i> which is what (A) atomicity means, then the question arises of (I) isolation; that is, what happens if somebody else does something to the same <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x238280f8">data</a> at the same time? Then comes the question of whether a thing, once having happened, will stay that way; i.e., (D) durability. Finally, there is (<a class="auto-href" href="http://dbpedia.org/resource/C%2B%2B" id="link-id0x276714b8">C</a>) consistency, which means that the transaction's result must not contradict restrictions the database is supposed to enforce. RDF usually has no restrictions; thus consistency mostly means that the internal state of the DBMS must be consistent, e.g., different indices on triples/quads should contain the same data.</p> <p>There are, of course, database-like consistency criteria that one can express in RDF Schema and <a class="auto-href" href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x28625a90">OWL</a>, concerning data types, mandatory presence of properties, or restrictions on cardinality (i.e., one may only have one spouse at a time, and the like). </p> <p>If one indeed did enforce them all, then RDF would be very like the relational model -- with all the restrictions, but without the 40 years of work on <a class="auto-href" href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x249bf4f8">RDBMS</a> performance. For this reason, RDF use tends to involve data that is not structured enough to be a good fit for RDBMS.</p> <p>There is of course the OWL side, where consistency is important but is defined in such complex ways that they again are not a good fit for RDBMS. RDF could be seen to be split between the schema-last world and the <a class="auto-href" href="http://dbpedia.org/resource/Knowledge" id="link-id0x249504f8">knowledge</a> representation world. I will here focus on the schema-last side.</p> <p>Transactions are relevant in RDF in two cases: 1. If data is trickle loaded in small chunks, one likes to know that the chunks do not get lost or corrupted; 2. If the application has any semantics that reserve resources, then these operations need transactions. The latter is not so common with RDF but examples include read-write situations, like checking if a seat is available and then reserving it. Transactionality guarantees that the same seat does not get reserved twice.</p> <p>Web people argue with some justification that since the four cardinal virtues of database never existed on the web to begin with, applying strict ACID to web data is beside the point, like locking the stable after the horse has long since run away. This may be so; yet the systems used for processing data, whether that data is dirty or not, benefit from predictable operation under concurrency and from not losing data.</p> <p>Analytics workloads are not primarily about transactions, but still need to specify what happens with updates. Analyzing data from measurements may not have concurrent updates, but there the transaction issue is replaced by the question of making explicit how the data was acquired and what processing has been applied to it before storage.</p> <p>As mentioned before, the <a class="auto-href" href="http://lod2.eu/" id="link-id0x27d952d0">LOD2</a> project is at the crossroads of RDF and database. I construe its mission to be the making of RDF into a respectable database discipline. Database respectability in turn is as good as inconceivable without addressing the very bedrock on which this science was founded: transactions.</p> <p>As previously argued, we need well-defined and auditable benchmarks. This again brings up the topic of transactions. Once we embark on the database benchmark route, there is no way around this. <a class="auto-href" href="http://www.tpc.org/" id="link-id0x2359d2d0">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/TPC-H" id="link-id0x28edb770">H</a> mandates that the system under test support transactions, and the audit involves a test for this. We can do no less.</p> <p>This has led me to more closely examine the issue of RDF and transactions, and whether there exist differences between transactions applied to RDF and to relational data. </p> <p>As concerns Virtuoso, our position has been that one can get full ACID in Virtuoso, whether in SQL or SPARQL, by using a connected client (e.g., <a class="auto-href" href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0x23a55698">ODBC</a>, <a class="auto-href" href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0x235cecf0">JDBC</a>, or the <a class="auto-href" href="http://jena.sourceforge.net/" id="link-id0x23213900">Jena</a> or <a class="auto-href" href="http://sourceforge.net/projects/sesame/" id="link-id0x277874d0">Sesame</a> frameworks), and setting the isolation options on the connection. Having taken this step, one then must take the next step, which consists of dealing with deadlocks; i.e., with concurrent utilization, it may happen that the database at any time notifies the client that the transaction got aborted and the client must retry.</p> <p>Web developers especially do not like this, because this is not what MySQL has taught them to expect. MySQL does have transactional back-ends like InnoDB, but often gets used without transactions.</p> <p>With the March 2011 Virtuoso releases, we have taken a closer look at transactions with RDF. It is more practical to reduce the possibility of errors than to require developers to pay attention. For this reason we have automated isolation settings for RDF, greatly reduced the incidence of deadlocks, and even incorporated automatic deadlock retries where applicable.</p> <p>If all users lock resources they need in the same order, there will be no deadlocks. This is what we do with RDF load in Virtuoso 7; thus any mix of concurrent <code>INSERTs</code> and <code>DELETEs</code>, if these are under a certain size (normally 10000 quads) are guaranteed never to fail due to locking. These could still fail due to running out of space, though. With previous versions, there always was a possibility of having an <code>INSERT</code> or <code>DELETE</code> fail because of deadlock with multiple users. Vectored <code>INSERT</code> and <code>DELETE</code> are sufficient for making web crawling or archive maintenance practically deadlock free, since there the primary transaction is the <code>INSERT</code> or <code>DELETE</code> of a small graph. </p> <p>Furthermore, since the <a class="auto-href" href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id0x23eadf50">SPARQL protocol</a> has no way of specifying transactions consisting of multiple client-server exchanges, the SPARQL end-point may deal with deadlocks by itself. If all else fails, it can simply execute requests one after the other, thus eliminating any possibility of locking. We note that many statements will be intrinsically free of deadlocks by virtue of always locking in key order, but this cannot be universally guaranteed with arbitrary size operations; thus concurrent operations might still sometimes deadlock. Anyway, vectored execution as introduced in Virtuoso 7, besides getting easily double-speed random access, also greatly reduces deadlocks by virtue of ordering operations.</p> <p>In the next post we will talk about what transactions mean with RDF and whether there is any difference with the relational model.</p>
Benchmarks, Redux (part 15): BSBM Test Driver Enhancements
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-22#1688
2011-03-22T22:32:28Z
2011-03-22T17:04:43-04:00
<p>This article covers the changes we have made to the <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x2361bf18">BSBM</a> test driver during our series of experiments.</p> <ul> <li> <p> <b>Drill-down mode</b> - For queries that have a product type as parameter, the test driver will invoke the query multiple times with each time a random subtype of the product type of the previous invocation. The starting point of the drill-down is an a random type from a settable level in the hierarchy. The rationale for the drill-down mode is that depending on the parameter choice, there can be 1000x differences in query run time. Thus run times of consecutive query mixes will be incomparable unless we guarantee that each mix has a predictable number of queries with a product type from each level in the hierarchy.</p> </li> <li> <b>Permutation of query mix</b> - In the BI workload, the queries are run in a random order on each thread in multiuser mode. Doing exactly the same thing on many threads is not realistic for large queries. The <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x2834cec8">data</a> access patterns must be spread out in order to evaluate how bulk IO is organized with differing concurrent demands. The permutations are deterministic on consecutive runs and do not depend on the non-deterministic timing of concurrent activities. For queries with a drill-down, the individual executions that make up the drill-down are still consecutive.</li> <li> <p> <b>New metrics</b> - The BI Power is the geometric mean of query run times scaled to queries per hour and multiplied by the scale factor, where 100 Mt is considered the unit scale. The BI Throughput is the arithmetic mean of the run times scaled to QPH and adjusted to scale as with the Power metric. These are analogous to the <a class="auto-href" href="http://www.tpc.org/" id="link-id0x236c5158">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/TPC-H" id="link-id0x28814950">H</a> Power and Throughput metrics. </p> <p>The <i>Power</i> is defined as</p> <blockquote>(scale_factor / 284826) * 3600 / ((t0 * t1 * ... * tn) ^(1 / n)) </blockquote> <p>The <i>Throughput</i> is defined as</p> <blockquote>(scale_factor / 284826) * 3600 / ((t0 + t2 + ... + tn) / n)</blockquote> <p>The magic number 284826 is the scale that generates approximately 100 million triples (100 Mt). We consider this "scale one." The reason for the multiplication is that scores at different scales should get similar numbers, otherwise 10x larger scale would result roughly in 10x lower throughput with the BI queries.</p> <p>We also show the percentage each query represents from the total time the test driver waits for responses. </p> </li> <li> <p> <b>Deadlock retry</b> - When running update mixes, it is possible that a transaction gets aborted by a deadlock. We have made a retry logic for this.</p> </li> <li> <p> <b>Cluster mode</b> - Cluster databases may have multiple interchangeable <a class="auto-href" href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x240f9008">HTTP</a> listeners. With this mode, one can specify multiple end-points so a multi-user workload can divide itself evenly over these.</p> </li> <li> <p> <b>Identifying matter</b> - A version number was added to test driver output. Use of the new switches is also indicated in the test driver output.</p> </li> <li> <p> <b>SUT <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x249b7208">CPU</a></b> - In comparing results it is crucial to differentiate between in memory runs and IO bound runs. To make this easier, we have added an option to report server CPU times over the timed portion (excluding warm-ups). A pluggable self-script determines the CPU times for the system; thus clusters can be handled, too. The time is given as a sum of the time the server processes have aged during the run and as a percentage over the wall-clock time.</p> </li> </ul> <p>These changes will soon be available <a href="http://blogs.usnet.private:8893/RPC2" id="link-id0x1f9a57c0">as a diff</a> and <a href="http://blogs.usnet.private:8893/RPC2" id="link-id0x1f2fea08">as a source tree</a>. This version is labeled <b><code>BSBM Test Driver 1.1-opl</code></b>; the <b><code>-opl</code></b> signifies OpenLink additions. </p> <p>We invite FU Berlin to include these enhancements into their Source Forge repository of the BSBM test driver. There is more precise documentation of these options in the README file in the above distribution.</p> <p>The next planned upgrade of the test driver concerns adding support for "<a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x2865ac68">RDF</a>-H", the RDF adaptation of the industry standard TPC-H decision support benchmark for <a class="auto-href" href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x23597bb0">RDBMS</a>.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1db2be00">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1dfcc038">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x197c26d0">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1d149cf0">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1ab69450">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1e67d688">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1dad87c8">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1cc73830">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1d6879a8">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1dfae510">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1ef052a0">Benchmarks, Redux (part 11): The Substance of Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1dadddb0">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e662ef0">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1df6fa70">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> Benchmarks, Redux (part 15): BSBM Test Driver Enhancements <i>(this post)</i> </li> </ul>
Benchmarks, Redux (part 14): BSBM BI Mix
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-22#1687
2011-03-22T22:31:32Z
2011-03-22T17:04:38-04:00
<p>In this post, we look at how we run the <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x236dcda8">BSBM</a>-BI mix. We consider the 100 Mt and 1000 Mt scales with <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x284893c0">Virtuoso</a> 7 using the same hardware and software as in the previous posts. The changes to workload and metric are given in the previous post.</p> <p>Our intent here is to look at whether the metric works, and to see what results will look like in general. We are as much testing the benchmark as we are testing the system-under-test (SUT). The results shown here will likely not be comparable with future ones because we will most likely change the composition of the workload since it seems a bit out of balance. Anyway, for the sake of disclosure, we attach the query templates. The test driver we used will be made available soon, so the interested may still try a comparison with their systems. If you practice with this workload for the coming races, the effort will surely not be wasted.</p> <p>Once we have come up with a rules document, we will redo all that we have published so far by-the-book, and have it audited as part of the <a class="auto-href" href="http://lod2.eu/" id="link-id0x23724860">LOD2</a> service we plan for this (see previous posts in this series). This will introduce comparability; but before we get that far with the BI workload, the workload needs to evolve a bit.</p> <p>Below we show samples of test driver output; the whole output is <a href="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/BenchmarksReduxSupportingFiles/br.tar.gz" id="link-id0x1b703ad8">downloadable</a>.</p> <p>100 Mt Single User</p> <blockquote> <code><pre> bsbm/testdriver -runs 1 -w 0 -idir /bs/1 -drill \ -ucf bsbm/usecases/businessIntelligence/<a class="auto-href" href="http://dbpedia.org/resource/SPARQL" id="link-id0x2385eb48">sparql</a>.txt \ -dg <a class="auto-href" href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x22e2f508">http</a>://bsbm.org http://localhost:8604/sparql </pre> </code> </blockquote> <blockquote> <code><pre> 0: 43348.14ms, total: 43440ms Scale factor: 284826 Explore Endpoints: 1 Update Endpoints: 1 Drilldown: on Number of warmup runs: 0 Seed: 808080 Number of query mix runs (without warmups): 1 times min/max Querymix runtime: 43.3481s / 43.3481s Elapsed runtime: 43.348 seconds QMpH: 83.049 query mixes per hour CQET: 43.348 seconds average runtime of query mix CQET (geom.): 43.348 seconds geometric mean runtime of query mix AQET (geom.): 0.492 seconds geometric mean runtime of query Throughput: 1494.874 BSBM-BI throughput: qph*scale BI Power: 7309.820 BSBM-BI Power: qph*scale (geom) </pre> </code> </blockquote> <p>100 Mt 8 User </p> <blockquote> <code><pre> Thread 6: query mix 3: 195793.09ms, total: 196086.18ms Thread 8: query mix 0: 197843.84ms, total: 198010.50ms Thread 7: query mix 4: 201806.28ms, total: 201996.26ms Thread 2: query mix 5: 221983.93ms, total: 222105.96ms Thread 4: query mix 7: 225127.55ms, total: 225317.49ms Thread 3: query mix 6: 225860.49ms, total: 226050.17ms Thread 5: query mix 2: 230884.93ms, total: 231067.61ms Thread 1: query mix 1: 237836.61ms, total: 237959.11ms Benchmark run completed in 237.985427s Scale factor: 284826 Explore Endpoints: 1 Update Endpoints: 1 Drilldown: on Number of warmup runs: 0 Number of clients: 8 Seed: 808080 Number of query mix runs (without warmups): 8 times min/max Querymix runtime: 195.7931s / 237.8366s Total runtime (sum): 1737.137 seconds Elapsed runtime: 1737.137 seconds QMpH: 121.016 query mixes per hour CQET: 217.142 seconds average runtime of query mix CQET (geom.): 216.603 seconds geometric mean runtime of query mix AQET (geom.): 2.156 seconds geometric mean runtime of query Throughput: 2178.285 BSBM-BI throughput: qph*scale BI Power: 1669.745 BSBM-BI Power: qph*scale (geom) </pre> </code> </blockquote> <p>1000 Mt Single User</p> <blockquote> <code><pre> 0: 608707.03ms, total: 608768ms Scale factor: 2848260 Explore Endpoints: 1 Update Endpoints: 1 Drilldown: on Number of warmup runs: 0 Seed: 808080 Number of query mix runs (without warmups): 1 times min/max Querymix runtime: 608.7070s / 608.7070s Elapsed runtime: 608.707 seconds QMpH: 5.914 query mixes per hour CQET: 608.707 seconds average runtime of query mix CQET (geom.): 608.707 seconds geometric mean runtime of query mix AQET (geom.): 5.167 seconds geometric mean runtime of query Throughput: 1064.552 BSBM-BI throughput: qph*scale BI Power: 6967.325 BSBM-BI Power: qph*scale (geom) </pre> </code> </blockquote> <p>1000 Mt 8 User </p> <blockquote> <code><pre> bsbm/testdriver -runs 8 -mt 8 -w 0 -idir /bs/10 -drill \ -ucf bsbm/usecases/businessIntelligence/sparql.txt \ -dg http://bsbm.org http://localhost:8604/sparql </pre> </code> </blockquote> <blockquote> <code><pre> Thread 3: query mix 4: 2211275.25ms, total: 2211371.60ms Thread 4: query mix 0: 2212316.87ms, total: 2212417.99ms Thread 8: query mix 3: 2275942.63ms, total: 2276058.03ms Thread 5: query mix 5: 2441378.35ms, total: 2441448.66ms Thread 6: query mix 7: 2804001.05ms, total: 2804098.81ms Thread 2: query mix 2: 2808374.66ms, total: 2808473.71ms Thread 1: query mix 6: 2839407.12ms, total: 2839510.63ms Thread 7: query mix 1: 2889199.23ms, total: 2889263.17ms Benchmark run completed in 2889.302566s Scale factor: 2848260 Explore Endpoints: 1 Update Endpoints: 1 Drilldown: on Number of warmup runs: 0 Number of clients: 8 Seed: 808080 Number of query mix runs (without warmups): 8 times min/max Querymix runtime: 2211.2753s / 2889.1992s Total runtime (sum): 20481.895 seconds Elapsed runtime: 20481.895 seconds QMpH: 9.968 query mixes per hour CQET: 2560.237 seconds average runtime of query mix CQET (geom.): 2544.284 seconds geometric mean runtime of query mix AQET (geom.): 13.556 seconds geometric mean runtime of query Throughput: 1794.205 BSBM-BI throughput: qph*scale BI Power: 2655.678 BSBM-BI Power: qph*scale (geom) Metrics for Query: 1 Count: 8 times executed in whole run Time share 2.120884% of total execution time AQET: 54.299656 seconds (arithmetic mean) AQET(geom.): 34.607302 seconds (geometric mean) QPS: 0.13 Queries per second minQET/maxQET: 11.71547600s / 148.65379700s Metrics for Query: 2 Count: 8 times executed in whole run Time share 0.207382% of total execution time AQET: 5.309462 seconds (arithmetic mean) AQET(geom.): 2.737696 seconds (geometric mean) QPS: 1.34 Queries per second minQET/maxQET: 0.78729800s / 25.80948200s Metrics for Query: 3 Count: 8 times executed in whole run Time share 17.650472% of total execution time AQET: 451.893890 seconds (arithmetic mean) AQET(geom.): 410.481088 seconds (geometric mean) QPS: 0.02 Queries per second minQET/maxQET: 171.07262500s / 721.72939200s Metrics for Query: 5 Count: 32 times executed in whole run Time share 6.196565% of total execution time AQET: 39.661685 seconds (arithmetic mean) AQET(geom.): 6.849882 seconds (geometric mean) QPS: 0.18 Queries per second minQET/maxQET: 0.15696500s / 189.00906200s Metrics for Query: 6 Count: 8 times executed in whole run Time share 0.119916% of total execution time AQET: 3.070136 seconds (arithmetic mean) AQET(geom.): 2.056059 seconds (geometric mean) QPS: 2.31 Queries per second minQET/maxQET: 0.41524400s / 7.55655300s Metrics for Query: 7 Count: 40 times executed in whole run Time share 1.577963% of total execution time AQET: 8.079921 seconds (arithmetic mean) AQET(geom.): 1.342079 seconds (geometric mean) QPS: 0.88 Queries per second minQET/maxQET: 0.02205800s / 40.27761500s Metrics for Query: 8 Count: 40 times executed in whole run Time share 72.126818% of total execution time AQET: 369.323481 seconds (arithmetic mean) AQET(geom.): 114.431863 seconds (geometric mean) QPS: 0.02 Queries per second minQET/maxQET: 5.94377300s / 1824.57867400s </pre> </code> </blockquote> <p>The <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x2809d998">CPU</a> for the multiuser runs stays above 1500% for the whole run. The CPU for the single user 100 Mt run is 630%; for the 1000 Mt run, this is 574%. This can be improved since the queries usually have a lot of <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x22cf75b8">data</a> to work on. But final <a class="auto-href" href="http://dbpedia.org/resource/Program_optimization" id="link-id0x238b94c8">optimization</a> is not our goal yet; we are just surveying the race track. The difference between a warm single user run and a cold single user run is about 15% with data on SSD; with data on disk, this would be more. The numbers shown are with warm <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x23ad8c08">cache</a>. The single-user and multi-user Throughput difference, 1064 single-user vs. 1794 multi-user, is about what one would expect from the CPU utilization.</p> <p>With these numbers, the CPU does not appear badly memory-bound, else the increase would be less; also core multi-threading seems to bring some benefit. If the single-user run was at 800%, the Throughput would be 1488. The speed in excess of this may be attributed to core multi-threading, although we must remember that not every query mix is exactly the same length, so the figure is not exact. Core multi-threading does not seem to hurt, at the very least. Comparison of the same numbers with the column store will be interesting since it misses the cache a lot less and accordingly has better SMP scaling. The <a class="auto-href" href="http://dbpedia.org/resource/Intel_Corporation" id="link-id0x23568308">Intel</a> Nehalem memory subsystem is really pretty good.</p> <p> </p> <p>For reference, we show a run with Virtuoso 6 at 100Mt. </p> <blockquote> <code><pre> 0: 424754.40ms, total: 424829ms Scale factor: 284826 Explore Endpoints: 1 Update Endpoints: 1 Drilldown: on Number of warmup runs: 0 Seed: 808080 Number of query mix runs (without warmups): 1 times min/max Querymix runtime: 424.7544s / 424.7544s Elapsed runtime: 424.754 seconds QMpH: 8.475 query mixes per hour CQET: 424.754 seconds average runtime of query mix CQET (geom.): 424.754 seconds geometric mean runtime of query mix AQET (geom.): 1.097 seconds geometric mean runtime of query Throughput: 152.559 BSBM-BI throughput: qph*scale BI Power: 3281.150 BSBM-BI Power: qph*scale (geom) </pre> </code> </blockquote> <p>and 8 user </p> <blockquote> <code><pre> Thread 5: query mix 3: 616997.86ms, total: 617042.83ms Thread 7: query mix 4: 625522.18ms, total: 625559.09ms Thread 3: query mix 7: 626247.62ms, total: 626304.96ms Thread 1: query mix 0: 629675.17ms, total: 629724.98ms Thread 4: query mix 6: 667633.36ms, total: 667670.07ms Thread 8: query mix 2: 674206.07ms, total: 674256.72ms Thread 6: query mix 5: 695020.21ms, total: 695052.29ms Thread 2: query mix 1: 701824.67ms, total: 701864.91ms Benchmark run completed in 701.909341s Scale factor: 284826 Explore Endpoints: 1 Update Endpoints: 1 Drilldown: on Number of warmup runs: 0 Number of clients: 8 Seed: 808080 Number of query mix runs (without warmups): 8 times min/max Querymix runtime: 616.9979s / 701.8247s Total runtime (sum): 5237.127 seconds Elapsed runtime: 5237.127 seconds QMpH: 41.031 query mixes per hour CQET: 654.641 seconds average runtime of query mix CQET (geom.): 653.873 seconds geometric mean runtime of query mix AQET (geom.): 2.557 seconds geometric mean runtime of query Throughput: 738.557 BSBM-BI throughput: qph*scale BI Power: 1408.133 BSBM-BI Power: qph*scale (geom) </pre> </code> </blockquote> <p>Having the numbers, let us look at the metric and its scaling. We take the geometric mean of the single-user Power and the multiuser Throughput.</p> <blockquote> <code><pre> 100 Mt: sqrt ( 7771 * 2178 ); = 4114 1000 Mt: sqrt ( 6967 * 1794 ); = 3535 </pre> </code> </blockquote> <p>Scaling seems to work; the results are in the same general ballpark. The real times for the 1000 Mt run are a bit over 10x the times for the 100Mt run, as expected. The relative percentages of the queries are about the same on both scales, with the drill-down in Q8 alone being 77% and 72% respectively. The Q8 drill-down starts at the root of the product hierarchy. If we made this start one level from the top, its share would drop. This seems reasonable.</p> <p>Conversely, Q2 is out of place, with far too little share of the time. It takes a product as a starting point and shows a list of products with common features, sorted by descending count of common features. This would more appropriately be applied to a leaf product category instead, measuring how many of the products in the category have the top 20 features found in this category, to name an example.</p> <p>Also there should be more queries.</p> <p>At present it appears that BSBM-BI is definitely runnable, but a cursory look suffices to show that the workload needs more development and variety. We remember that I dreamt up the business questions last fall without much analysis, and that these questions were subsequently translated to SPARQL by FU Berlin. So, on one hand, BSBM-BI is of crucial importance because it is the first attempt at doing a benchmark with long running queries in SPARQL. On the other hand, BSBM-BI is not very good as a benchmark; <a class="auto-href" href="http://www.tpc.org/" id="link-id0x23872a10">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/TPC-H" id="link-id0x28487d98">H</a> is a lot better. This stands to reason, as TPC-H has had years and years of development and participation by many people.</p> <p>Benchmark queries are trick questions: For example, TPC-H Q18 cannot be done without changing an <code>IN</code> into a <code>JOIN</code> with the <code>IN</code> subquery in the outer loop and doing streaming aggregation. Q13 cannot be done without a well-optimized <code><a class="auto-href" href="http://dbpedia.org/resource/Hash_join" id="link-id0x24974830">HASH JOIN</a></code> which besides must be partitioned at the larger scales.</p> <p>Having such trick questions in an important benchmark eventually results in everybody doing the optimizations that the benchmark clearly calls for. Making benchmarks thus entails a responsibility ultimately to the end user, because an irrelevant benchmark might in the worst case send developers chasing things that are beside the point.</p> <p>In the following, we will look at what BSBM-BI requires from the database and how these requirements can be further developed and extended.</p> <p>BSBM-BI does not have any clear trick questions, at least not premeditatedly. BSBM-BI just requires a cost model that can guess the fanout of a <code>JOIN</code> and the cardinality of a <code>GROUP BY</code>; it is enough to distinguish smaller from greater; the guess does not otherwise have to be very good. Further, the queries are written in the benchmark text so that joining from left to right would work, so not even a cost-based optimizer is strictly needed. I did however have to add some cardinality statistics to get reasonable <code>JOIN</code> order since we always reorder the query regardless of the source formulation.</p> <p>BSBM-BI does have variable selectivity from the drill-downs; thus these may call for different <code>JOIN</code> orders for different parameter values. I have not looked into whether this really makes a difference, though.</p> <p>There are places in BSBM-BI where using a <code>HASH JOIN</code> makes sense. We do not use <code>HASH JOINs</code> with <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x23cbf908">RDF</a> because there is an index for everything and making a <code>HASH JOIN</code> in the wrong place can have a large up-front cost, so one is more robust against cost model errors if one does not do <code>HASH JOINs</code>. This said, a <code>HASH JOIN</code> in the right place is a lot better than an index lookup. With TPC-H Q13, our best <code>HASH JOIN</code> is over 2x better than the best <code>INDEX</code>-based <code>JOIN</code>, both being well tuned. For questions like "count the hairballs made in <a class="auto-href" href="http://dbpedia.org/resource/Germany" id="link-id0x249d3e28">Germany</a> reviewed by Japanese Hello Kitty fans," where two ends of a <code>JOIN</code> path are fairly selective doing the other as a <code>HASH JOIN</code> is good. This can, if the <code>JOIN</code> is always cardinality-reducing, even be merged inside an <code>INDEX</code> lookup. We have such capabilities since we have been for a while gearing up for the relational races, but are not using any of these with BSBM-BI, although they would be useful.</p> <p>Let us see the profile for a single user 100 Mt run.</p> <p>The database activity summary is --</p> <p> <code>select db_activity (0, 'http');</code> </p> <p> <code> 161.3M rnd 210.2M seq   0 same seg  104.5M same pg 45.08M same par   0 disk    0 spec disk   0B /   0 messages 2.393K fork</code> </p> <p>See the post "<a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1b1f3068">What Does BSBM Explore Measure</a>" for an explanation of the numbers. We see that there is more sequential access than random and the random has fair locality with over half on the same page as the previous and a lot of the rest falling under the same parent. Funnily enough, the explore mix has more locality. Running with a longer vector size would probably increase performance by getting better locality. There is an optimization that adjusts vector size on the fly if locality is not sufficient but this is not being used here. So we manually set vector size to 100000 instead of the default 10000. We get --</p> <p> <code> 172.4M rnd 220.8M seq   0 same seg  149.6M same pg 10.99M same par   21 disk  861 spec disk   0B /   0 messages   754 fork</code> </p> <p>The throughput goes from 1494 to 1779. We see more hits on the same page, as expected. We do not make this setting a default since it raises the cost for small queries; therefore the vector size must be self-adjusting -- besides, expecting a DBA to tune this is not reasonable. We will just have to correctly tune the self-adjust logic, and we have again clear gains.</p> <p>Let us now go back to the first run with vector size 10000.</p> <p>The top of the CPU <code>oprofile</code> is as follows:</p> <blockquote> <code><pre> 722309 15.4507 cmpf_iri64n_iri64n 434791 9.3005 cmpf_iri64n_iri64n_anyn_iri64n 294712 6.3041 itc_next_set 273488 5.8501 itc_vec_split_search 203970 4.3631 itc_dive_transit 199687 4.2714 itc_page_rcf_search 181614 3.8848 dc_itc_append_any 173043 3.7015 itc_bm_vec_row_check 146727 3.1386 cmpf_int64n 128224 2.7428 itc_vec_row_check 113515 2.4282 dk_alloc 97296 2.0812 page_wait_access 62523 1.3374 qst_vec_get_int64 59014 1.2623 itc_next_set_parent 53589 1.1463 sslr_qst_get 48003 1.0268 ds_add 46641 0.9977 dk_free_tree 44551 0.9530 kc_var_col 43650 0.9337 page_col_cmp_1 35297 0.7550 cmpf_iri64n_iri64n_anyn_gt_lt 34589 0.7399 dv_compare 25864 0.5532 cmpf_iri64n_anyn_iri64n_iri64n_lte 23088 0.4939 dk_free </pre> </code> </blockquote> <p>The top 10 are all index traversal, with the key compare for two leading IRI keys in the lead, corresponding to a lookup with <code>P</code> and <code>S</code> given. The one after that is with all parts given, corresponding to an existence test. The existence tests could probably be converted to <code>HASH JOIN</code> lookups to good advantage. Aggregation and arithmetic are absent. We should probably add a query like TPC-H Q1 that does nothing but these two. Considering the overall profile, <code>GROUP BY</code> seems to be around 3%. We should probably put in a query that makes a very large number of groups and could make use of streaming aggregation, i.e., take advantage of a situation where aggregation input comes already grouped by the grouping columns.</p> <p>A BI use case should offer no problem with including arithmetic, but there are not that many numbers in the BSBM set. Some code sections in the queries with conditional execution and costly tests inside <code>ANDs</code> and <code>ORs</code> would be good. TPC-H has such in Q21 and Q19. An <code>OR</code> with existences where there would be gain from good guesses of a subquery's selectivity would be appropriate. Also, there should be conditional expressions somewhere with a lot of data, like the <code>CASE-WHEN</code> in TPC-H Q12.</p> <p>We can make BSBM-BI more interesting by putting in the above. Also we will have to see where we can profit from <code>HASH JOIN</code>, both small and large. There should be such places in the workload already so this is a matter of just playing a bit more.</p> <p>This post amounts to a cheat sheet for the BSBM-BI runs a bit farther down the road. By then we should be operational with the column store and Virtuoso 7 Cluster, though, so not everything is yet on the table.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1fd1d4e0">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1d5b07d8">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1dfe6c48">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x197fce30">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1fbf4210">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1beeb1e0">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1d7e1818">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1dfc1730">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1ea819a8">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1ec73da0">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1fbdce90">Benchmarks, Redux (part 11): The Substance of Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x19928618">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1f3d8710">Benchmarks, Redux (part 13): BSBM-BI Modifications </a> </li> <li> Benchmarks, Redux (part 14): BSBM-BI Mix <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e627400">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 13): BSBM BI Modifications
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-22#1686
2011-03-22T22:30:44Z
2011-03-22T17:04:34.000003-04:00
<p>In this post we introduce changes to the <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x234e0ca0">BSBM</a> BI queries and metric. These changes are motivated by prevailing benchmark practice and by our experiences in optimizing for the BSBM BI workload.</p> <p>We will publish results according to the definitions given here and recommend that any interested parties do likewise. The rationales are given in the text.</p> <h3>Query Mix</h3> <p>We have removed Q4 from the mix because it is quadratic to the scale factor. The other queries are roughly <code>n * log (n)</code>. </p> <h3>Parameter Substitution </h3> <p>All queries that take a product type as parameter are run in flights of several query invocations where the product type goes from broader to more specific. The initial product type specifies either the root product type or an immediate subtype of this, and the last in the drill-down is a leaf type.</p> <p>The rationale for this is that the choice of product type may make several orders of magnitude difference in the run time of a query. In order to make consecutive query mixes roughly comparable in execution time, all mixes should have a predictable number of query invocations with product types of each level.</p> <h3>Query Order </h3> <p>In the BI mix, when running multiple concurrent clients, each query mix is submitted in a random order. Queries which do drill-downs always have the steps of the drill-down as consecutive in the session, but the query templates are permuted. This is done so as to make less likely that there were two concurrent queries accessing exactly the same <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x23be8d28">data</a>. In this way, scans cannot be trivially shared between queries -- but there are still opportunities for reuse of results and adapting execution to working set, e.g., starting with what is in memory.</p> <h3>Metrics </h3> <p>We use a <a class="auto-href" href="http://www.tpc.org/" id="link-id0x238c81a0">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/TPC-H" id="link-id0x28c6bbd8">H</a>-like metric. This metric consists of a single-user part and a multi-user part, called respectively <i>Power</i> and <i>Throughput.</i> The <i>Power</i> metric is a geometric mean of query run-time. The <i>Throughput</i> is the total run-time divided by the number of queries completed. After taking the mean, the time is converted into queries-per-hour. This time is then multiplied by the scale factor divided by the scale factor for 100 Mt. In other words, we consider the 100 Mt data set as the unit scale.</p> <p>The <i>Power</i> is defined as</p> <blockquote>( scale_factor / 284826 ) * 3600 / ( ( t1 * t1 * ... * tn ) ^ ( 1 / n ) ) </blockquote> <p>The <i>Throughput</i> is defined as</p> <blockquote>( scale_factor / 284826 ) * 3600 / ( ( t1 + t2 + ... + tn ) / n ) </blockquote> <p>The magic number <b><code>284826</code></b> is the scale that generates approximately 100 million triples (100 Mt). We consider this scale "one". The reason for the multiplication is that scores at different scales should get similar numbers; otherwise 10x larger scale would result roughly in 10x lower throughput with the BI queries.</p> <p>The <i>Composite</i> metric is the geometric mean of the <i>Power</i> and <i>Throughput</i> metrics. A complete report shows both <i>Power</i> and <i>Throughput</i> metrics, as well as individual query times for all queries. The rationale for using a geometric mean is to give an equal importance to long and short queries. Halving the execution time of either a long query or a short query will have the same effect on the metric. This is good for encouraging research into all aspects of query processing. On the other hand, real-life users are more interested in halving the time of queries that take one hour than of queries that take one second; therefore, the throughput metric considers run times.</p> <p>Taking the geometric mean of the two metrics gives more weight to the lower of the two than an arithmetic mean, hence we pay more attention to the worse of the two.</p> <p>Single-user and multi-user metrics are separate because of the relative importance of intra-query parallelization in BI workloads: There may not be large numbers of concurrent users, yet queries are still complex, and it is important to have maximum parallelization. Therefore the metric rewards single-user performance.</p> <p>In the next post we will look at the use of this metric and the actual content of BSBM BI.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1b02d528">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1d65f740">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1a797860">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1d3538e0">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1e566f60">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1dedffd8">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1eb11528">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1db46c38">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1c8174e8">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1dfa9338">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1e6dd7b0">Benchmarks, Redux (part 11): The Substance of Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d154bb0">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> Benchmarks, Redux (part 13): BSBM BI Modifications <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1f242ae0">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ebf2f98">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-10#1680
2011-03-10T23:30:11Z
2011-03-14T19:37:28-04:00
<p>Let us talk about what ought to be benchmarked in the context of <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x2a84d3c0">RDF</a>.</p> <p>A point that often gets brought up by RDF-ers when talking about benchmarks is that there already exist systems which perform very well at <a class="auto-href" href="http://www.tpc.org/" id="link-id0x2a9758e8">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/TPC-H" id="link-id0x2a8fa2a0">H</a> and similar workloads, and therefore there is no need for RDF to go there. It is, as it were, somebody else's problem; besides, it is a solved one.</p> <p>On the other hand, being able to express what is generally expected of a query language might not be a core competence or a competitive edge, but it certainly is a checklist item.</p> <p> <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x29c75a30">BSBM</a> seems to be adopted as a de facto RDF benchmark, as there indeed is almost nothing else. But we should not lose sight of the fact that this is in fact a relational <a class="auto-href" href="http://dbpedia.org/resource/Database_schema" id="link-id0x2a0565b8">schema</a> and workload that has just been straightforwardly transformed to RDF. BSBM was made, after all, in part for measuring RDB to RDF mapping. Thus BSBM is no more RDF-ish than a trivially RDF-ized TPC-H would be. TPC-H is however a bit more difficult if also a better thought out benchmark than the BSBM BI Mix proposal. But I do not expect an RDF audience to have any enthusiasm for this as this is indeed a very tough race by now, and besides one in which RDB and <a class="auto-href" href="http://dbpedia.org/resource/SQL" id="link-id0x29c44d50">SQL</a> will keep some advantage. However, using this as a validation test is meaningful, as there exists a validation dataset and queries that we already have RDF-ized. We could publish these and call this "RDF-H". </p> <p>In the following I will outline what would constitute an RDF-friendly, scientifically interesting benchmark. The points are in part based on discussions with <a class="auto-href" href="http://nl.linkedin.com/in/peterboncz" id="link-id0x2ac282f0">Peter Boncz</a> of <a class="auto-href" href="http://dbpedia.org/resource/National_Research_Institute_for_Mathematics_and_Computer_Science" id="link-id0x2a1c9e10">CWI</a>.</p> <p>The <a class="auto-href" href="http://www.w3.org/wiki/Social_Network_Intelligence_BenchMark" id="link-id0x29e7d3d8">Social Network Intelligence Benchmark</a> (<a class="auto-href" href="http://www.w3.org/wiki/Social_Network_Intelligence_BenchMark" id="link-id0x2a70e3c0">SNIB</a>) takes the social web Facebook-style schema Ivan Mikhailov and I made last year under the name of Botnet BM. In <a class="auto-href" href="http://lod2.eu/" id="link-id0x2a9a70f0">LOD2</a>, CWI is presently working on this.</p> <p>The <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x2ad04408">data</a> includes <a class="auto-href" href="http://dbpedia.org/resource/DBpedia" id="link-id0x29d5eeb0">DBpedia</a> as a base component used for providing conversation topics, <a class="auto-href" href="http://dbpedia.org/resource/Information" id="link-id0x2ac97c40">information</a> about geographical locales of simulated users, etc. DBpedia is not very large, around 200M-300M triples, but it is diverse enough.</p> <p>The data will have correlations, e.g., people who talk about sports tend to know other people who talk about the same sport, and they are more likely to know people from their geographical area than from elsewhere. </p> <p>The bulk of the data consists of a rich history of interactions including messages to individuals and groups, linking to people, dropping links, joining and leaving groups, and so forth. The messages are tagged using real-world concepts from DBpedia, and there is correlation between tagging and textual content since both are generated from Dbpedia articles. Since there is such correlation, <a class="auto-href" href="http://dbpedia.org/resource/Natural_language_processing" id="link-id0x2ac359c0">NLP</a> techniques like <a class="auto-href" href="http://dbpedia.org/resource/Entity" id="link-id0x2a1c8ed0">entity</a> and relationship extraction can be used with the data even though this is not the primary thrust of SNIB.</p> <p>There is variation in frequency of online interaction, and this interaction consist of sessions. For example, one could analyze user behavior per time of day for online ad placement.</p> <p>The data probably should include propagating memes, fashions, and trends that travel on the social network. With this, one could query about their origin and speed of propagation.</p> <p>There should probably be cases of duplicate identities in the data, i.e., one real person using many online accounts to push an agenda. Resolving duplicate identities makes for nice queries.</p> <p>Ragged data with half-filled profiles and misspelled identifiers like person and place names are a natural part of the social web use case. The data generator should take this into account.</p> <ul> <li> <p>Distribution of popularity and activity should follow a power-law-like pattern; actual measures of popularity can be sampled from existing social networks even though large quantities of data cannot easily be extracted.</p> </li> <li> <p>The dataset should be predictably scalable. For the workload considered, the relative importance of the queries or other measured tasks should not change dramatically with the scale.</p> </li> </ul> <p>For example some queries are logarithmic to data size (e.g., find connections to a person), some are linear (e.g., find average online time of sports fans on Sundays), and some are quadratic or worse (e.g., find two extremists of the same ideology that are otherwise unrelated). Making a single metric from such parts may not be meaningful. Therefore, SNIB might be structured into different workloads.</p> <p>The first would be an online mix with typically short lookups and updates, around <code>O ( log ( n ) )</code>. </p> <p>The Business Intelligence Mix would be composed of queries around <code>OO ( n log ( n ) )</code>. Even so, with real data, choice of parameters will provide dramatic changes in query run-time. Therefore a run should be specified to have a predictable distribution of "hard" and "easy" parameter choices. In the BSBM BI mix modification, I did this by defining some to be drill downs from a more general to a more specific level of a hierarchy. This could be done here too in some cases; other cases would have to be defined with buckets of values. </p> <p>Both the real world and LOD2 are largely concerned with data integration. The SNIB workload can have aspects of this, for example, in resolving duplicate identities. These operations are more complex than typical database queries, as the attributes used for joining might not even match in the initial data.</p> <p>One characteristic of these is the production of sometimes large intermediate results that need to be materialized. Doing these operations in practice requires procedural control. Further, running algorithms like network analytics (e.g., Page rank, centrality, etc.) involves aggregation of intermediate results that is not very well expressible in a query language. Some basic graph operations like shortest path are expressible but then are not in unextended <a class="auto-href" href="http://dbpedia.org/resource/SPARQL" id="link-id0x29d26588">SPARQL</a> 1.1; as these would for example involve returning paths, which are explicitly excluded from the spec.</p> <p>These are however the areas where we need to go for a benchmark that is more than a repackaging of a relational BI workload.</p> <p>We find that such a workload will have procedural sections either in application code or stored procedures. Map-reduce is sometimes used for scaling these. As one would expect, many cluster databases have their own version of these control structures. Therefore some of the SNIB workload could even be implemented as map-reduce jobs alongside parallel database implementations. We might here touch base with the <a class="auto-href" href="http://www.larkc.eu/" id="link-id0x29b69640">LarKC</a> map-reduce work to see if it could be applied to SNIB workloads. </p> <p>We see a three-level structure emerging. There is an <i>Online</i> mix which is a bit like the BSBM <i>Explore</i> mix, and an <i>Analytics</i> mix which is on the same order of complexity as TPC-H. These may have a more-or-less fixed query formulation and test driver. Beyond these, yet working on the same data, we have a set of <i>Predefined Tasks</i> which the test sponsor may implement in a manner of their choice.</p> <p>We would finally get to the "raging conflict" between the "declarativists" and the "map reductionists." Last year's VLDB had a lot of map-reduce papers. I know of comparisons between <a class="auto-href" href="http://www.vertica.com/" id="link-id0x2a8c4510">Vertica</a> and map reduce for doing a fairly simple SQL query on a lot of data, but here we would be talking about much more complex jobs on more interesting (i.e., less uniform) data.</p> <p>We might even interest some of the cluster <a class="auto-href" href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x2995aaa8">RDBMS</a> players (<a class="auto-href" href="http://www.teradata.com/" id="link-id0x29c9af10">Teradata</a>, Vertica, <a class="auto-href" href="http://dbpedia.org/resource/Greenplum" id="link-id0x29c9af38">Greenplum</a>, <a class="auto-href" href="http://dbpedia.org/page/Oracle_Exadata" id="link-id0x29d48b78">Oracle Exadata</a>, <a class="auto-href" href="http://www.paraccel.com/" id="link-id0x29d48ba0">ParAccel</a>, and/or <a class="auto-href" href="http://www.asterdata.com/" id="link-id0x29bf8fb0">Aster Data</a>, to name a few) in running this workload using their map-reduce analogs.</p> <p>We see that as we get to topics beyond relational BI, we do not find ourselves in an RDF-only world but very much at a crossroads of many technologies, e.g., map-reduce and its database analogs, various custom built databases, graph libraries, data integration and cleaning tools, and so forth.</p> <p>There is not, nor ought there to be, a sheltered, RDF-only enclave. RDF will have to justify itself in a world of alternatives.</p> <p>This must be reflected in our benchmark development, so relational BI is not irrelevant; in fact, it is what everybody does. RDF cannot be a total failure at this, even if this were not RDF's claim to fame. The claim to fame comes after we pass this stage, which is what we intend to explore in SNIB.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1c9f7ab8">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1dd17b28">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1eb20620">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1f8a5ae8">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1ac14a08">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1d1f8d58">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1ea83308">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1b548028">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1c3d9c58">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1f5e6978">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1c082a28">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ec73578">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1eb25d48">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1b261958">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 10): LOD2 and the Benchmark Process
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-10#1679
2011-03-10T23:29:41Z
2011-03-14T19:37:14.000001-04:00
<p>I have in the previous posts generally argued for and demonstrated the usefulness of benchmarks.</p> <p>Here I will talk about how this could be organized in a way that is tractable, and takes vendor and end user interests into account. These are my views on the subject and do not represent a <a class="auto-href" href="http://lod2.eu/" id="link-id0x2acb0760">LOD2</a> members consensus, but have been discussed in the consortium. </p> <p>My colleague Ivan Mikhailov once proposed that the only way to get benchmarks run right is to package them as a single script that does everything, like instant noodles -- just add water! But even instant noodles can be abused: Cook too long, add too much water, maybe forget to light the stove, and complain that the result is unsatisfyingly hard and brittle, lacking the suppleness one has grown to expect from this delicacy. No, the answer lies at the other end of the culinary spectrum, in gourmet cooking. Let the best cooks show what they can do, and let them work at it; let those who in fact have capacity and motivation for creating <i>le chef d'oeuvre culinaire</i> ("the culinary masterpiece") create it. Even so, there are many value points along the dimensions of preparation time, cost, and esthetic layout, not to forget taste and nutritional values. Indeed, an intimate <a class="auto-href" href="http://dbpedia.org/resource/Knowledge" id="link-id0x2aca6a30">knowledge</a> <i>de la vie secrete du canard</i> ("the secret life of duck") is required in order to liberate the aroma that it might take flight and soar. In the previous, I have shed some light on how we prepare <i>le canard</i>, and if <i>le canard</i> be such then <i>la dinde</i> (turkey) might in some ways be analogous; who is to say?</p> <p>In other words, as a vendor, we want to have complete control over the benchmarking process, and have it take place in our environment at a time of our choice. In exchange for this, we are ready to document and observe possibly complicated rules, document how the runs are made, and let others monitor and repeat them on the equipment on which the results are obtained. This is the <a class="auto-href" href="http://www.tpc.org/" id="link-id0x2b847818">TPC</a> (Transaction Processing Performance Council) model.</p> <p>Another culture of doing benchmarks is the periodic challenge model used in TREC, the <a class="auto-href" href="http://challenge.semanticweb.org/" id="link-id0x2ac3a6f8">Billion Triples Challenge</a>, the Semantic Search Challenge and others. In this model, vendors prepare the benchmark submission and agree to joint publication.</p> <p>A third party performing benchmarks by itself is uncommon in databases. Licenses even often explicitly prohibit this, for understandable reasons.</p> <p>The LOD2 project has an outreach activity called Publink where we offer to help owners of <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x2aea5930">data</a> to publish it as <a class="auto-href" href="http://dbpedia.org/resource/Linked_Data" id="link-id0x2a790128">Linked Data</a>. Similarly, since FP 7s are supposed to offer a visible service to their communities, I proposed that LOD2 offer to serve a role in disseminating and auditing <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x29babb00">RDF</a> store benchmarks.</p> <p>One representative of an RDF store vendor I talked to, in relation to setting up a benchmark configuration of their product, told me that we could do this and that they would give some advice but that such an exercise was by its nature fundamentally flawed and could not possibly produce worthwhile results. The reason for this was that OpenLink engineers could not possibly learn enough about the other products nor unlearn enough of their own to make this a meaningful comparison.</p> <p>Isn't this the very truth? Let the chefs mix their own spices.</p> <p>This does not mean that there would not be comparability of results. If the benchmarks and processes are well defined, documented, and checked by a third party, these can be considered legitimate and not just one-off best-case results without further import.</p> <p>In order to stretch the envelope, which is very much a LOD2 goal, this benchmarking should be done on a variety of equipment -- whatever works best at the scale in question. Increasing the scale remains a stated objective. LOD2 even promised to run things with a trillion triples in another 3 years. </p> <p>Imagine that the unimpeachably impartial Berliners made house calls. Would this debase Justice to be a servant of mere show-off? Or would this on the contrary combine strict Justice with edifying Charity? Who indeed is in greater need of the light of objective evaluation than the vendor whose very nature makes a being of bias and prejudice?</p> <p>Even better, <a class="auto-href" href="http://dbpedia.org/resource/National_Research_Institute_for_Mathematics_and_Computer_Science" id="link-id0x2a21d108">CWI</a>, with its <a href="http://monetdb.cwi.nl/Development/Research/Articles/" id="link-id0x1d6479d0">stellar database pedigree</a>, agreed in principle to audit RDF benchmarks in LOD2. </p> <p>In this way one could get a stamp of approval for one's results regardless of when they were produced, and be free of the arbitrary schedule of third party benchmarking runs. On the relational side this is a process of some cost and complexity, but since the RDF side is still young and more on mutually friendly terms, the process can be somewhat lighter here. I did promise to draft some extra descriptions of process and result disclosure so that we could see how this goes.</p> <p>We could even do this unilaterally -- just publish <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x2a0d73d8">Virtuoso</a> results according to a predefined reporting and verification format. If others wished to publish by the same rules, LOD2 could use some of the benchmarking funds for auditing the proceedings. This could all take place over the <a class="auto-href" href="http://dbpedia.org/resource/.NET_Framework" id="link-id0x2a6b44a0">net</a>, so we are not talking about any huge cost or prohibitive amount of trouble. It would be in the FP7 spirit that LOD2 provide this service for free, naturally within reason.</p> <p>Then there is the matter of the <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x2a1722a8">BSBM</a> Business Intelligence (BI) mix. At present, it seems everybody has chosen to defer the matter to another round of BSBM runs in the summer. This seems to fit the pattern of a public challenge with a few months given for contenders to prepare their submissions. Here we certainly should look at bigger scales and more diverse hardware than in the Berlin runs published this time around. The BI workload is in fact fairly cluster friendly, with big joins and aggregations that parallelize well. There it would definitely make sense to reserve an actual cluster, and have all contenders set up their gear on it. If all have access to the run environment and to monitoring tools, we can be reasonably sure that things will be done in a transparent manner. </p> <p>(I will talk about the BI mix in more detail in <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1dfcc038">part 13</a> and <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1edaa388">part 14</a> of this series.)</p> <p>Once the BI mix has settled and there are a few interoperable implementations, likely in the summer, we could pass from the challenge model to a situation where vendors may publish results as they become available, with LOD2 offering its services for audit. </p> <p>Of course, this could be done even before then, but the content of the mix might not be settled. We likely need to check it on a few implementations first.</p> <p>For equipment, people can use their own, or LOD2 partners might on a case-by-case basis make some equipment available for running on the same hardware on which say the Virtuoso results were obtained. For example, FU Berlin could give people a login to get their recently published results fixed. Now this might or might not happen, so I will not hold my breath waiting for this but instead close with a proposal.</p> <p>As a unilateral diplomatic overture I put forth the following: If other vendors are interested in 1:1 comparison of their results with our publications, we can offer them a login to the same equipment. They can set up and tune their systems, and perform the runs. We will just watch. As an extra quid pro quo, they can try Virtuoso as configured for the results we have published, with the same data. Like this, both parties get to see the others' technology with proper tuning and installation. What, if anything, is reported about this activity is up to the owner of the technology being tested. We will publish a set of benchmark rules that can serve as a guideline for mutually comparable reporting, but we cannot force anybody to use these. This all will function as a catalyst for technological advance, all to the ultimate benefit of the end user. If you wish to take advantage of this offer, you may contact <a href="mailto:hwilliams@openlinksw.com?subject=Collaborative RDF Benchmark" id="link-id0x1c071100">Hugh Williams at OpenLink Software, and we will see how this can be arranged in practice.</a> </p> <p>The next post will talk about the <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x19933fd8">actual content of benchmarks</a>. The milestone after this will be when we publish the measurement and reporting protocols.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1c554800">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1ec159e8">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1dd5eb10">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x18f05940">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1ed5ef10">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1e9cb130">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1dfa79d8">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1eb6f478">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1de5a918">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> Benchmarks, Redux (part 10): LOD2 and the Benchmark Process <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1dae9060">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1f45fa10">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1f49d2b8">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e68e4c8">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e353858">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 9): BSBM With Cluster
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-09#1676
2011-03-09T22:54:50Z
2011-03-14T19:36:58-04:00
<p>This post is dedicated to our brothers in horizontal partitioning (or sharding), <a class="auto-href" href="http://freebase.com/guid/9202a8c04000641f8000000005c908d6" id="link-id0x2a1e9010">Garlik</a> and <a class="auto-href" href="http://www.systap.com/bigdata.htm" id="link-id0x2acd5218">Bigdata</a>.</p> <p>At first sight, the <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x2bb33648">BSBM</a> <i>Explore</i> mix appears very cluster-unfriendly, as it contains short queries that access <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x2b8fffb8">data</a> at random. There is every opportunity for latency and few opportunities for parallelism.</p> <p>For this reason we had not even run the BSBM mix with <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x2a84b780">Virtuoso</a> Cluster. We were not surprised to learn that <a href="http://steveharris.tumblr.com/post/3453040647/bsbm-v3-post-mortem" id="link-id0x1c4ef8d8">Garlik hadn't run BSBM either</a>. We have understood from <a class="auto-href" href="http://www.systap.com/" id="link-id0x2ad3d050">Systap</a> that their Bigdata BSBM experiments were on a single-process configuration.</p> <p>But the 4Store results in the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/V6/index.html" id="link-id0x1f8090f8">recent Berlin report</a> were with a distributed setup, as 4Store always runs a multiprocess configuration, even on a single server, so it seemed interesting to us to compare how Virtuoso Cluster compares with Virtuoso Single with this workload. These tests were run on a different box than the recent BSBM tests, so those 4Store figures are not directly comparable.</p> <p>The setup here consists of 8 partitions, each managed by its own process, all running on the same box. Any of these processes can have its <a class="auto-href" href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x2ac28380">HTTP</a> and <a class="auto-href" href="http://dbpedia.org/resource/SQL" id="link-id0x2bba8720">SQL</a> listener and can provide the same service. Most access to data goes over the interconnect, except when the data is co-resident in the process which is coordinating the query. The interconnect is Unix domain sockets since all 8 processes are on the same box.</p> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="4" align="center">6 Cluster - Load Rates and Times</th> </tr> <tr> <th align="center">Scale</th> <th align="center">Rate <br /> (quads per second)</th> <th align="center">Load time <br /> (seconds)</th> <th align="center">Checkpoint time <br /> (seconds)</th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 119,204 </td> <td align="center"> 749 </td> <td align="center"> 89 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 121,607 </td> <td align="center"> 1486 </td> <td align="center"> 157 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 102,694 </td> <td align="center"> 8737 </td> <td align="center"> 979 </td> </tr> </table> <br /> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="4" align="center">6 Single - Load Rates and Times</th> </tr> <tr> <th align="center">Scale</th> <th align="center">Rate <br /> (quads per second)</th> <th align="center">Load time <br /> (seconds)</th> <th align="center">Checkpoint time <br /> (seconds)</th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 74,713 </td> <td align="center"> 1192 </td> <td align="center"> 145 </td> </tr> </table> <p>The load times are systematically better than for 6 Single. This is also not bad compared to the 7 Single vectored load rates of 220 Kt/s or so. We note that loading is a cluster friendly operation, going at a steady 1400+% <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x296b03b8">CPU</a> utilization with an aggregate message throughput of 40MB/s. 7 Single is faster because of vectoring at the index level, not because the clusters were hitting communication overheads. 6 Cluster is faster than 6 Single because scale-out in this case diminishes contention, even on a single box.</p> <p>Throughput is as follows:</p> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="3" align="center"> 6 Cluster - Throughput <br /> (QMpH, query mixes per hour) </th> </tr> <tr> <th align="center">Scale</th> <th align="center"> Single User </th> <th align="center"> 16 User </th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 7318 </td> <td align="center"> 43120 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 6222 </td> <td align="center"> 29981 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 2526 </td> <td align="center"> 11156 </td> </tr> </table> <br /> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="3" align="center"> 6 Single - Throughput <br /> (QMpH, query mixes per hour) </th> </tr> <tr> <th align="center">Scale</th> <th align="center"> Single User </th> <th align="center"> 16 User </th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 7641 </td> <td align="center"> 29433 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 6017 </td> <td align="center"> 13335 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 1770 </td> <td align="center"> 2487 </td> </tr> </table> <p>Below is a snapshot of status during the 6 Cluster 100 Mt run.</p> <blockquote> <code><pre> Cluster 8 nodes, 15 s. 25784 m/s 25682 KB/s 1160% cpu 0% read 740% clw threads 18r 0w 10i buffers 1133459 12 d 4 w 0 pfs cl 1: 10851 m/s 3911 KB/s 597% cpu 0% read 668% clw threads 17r 0w 10i buffers 143992 4 d 0 w 0 pfs cl 2: 2194 m/s 7959 KB/s 107% cpu 0% read 9% clw threads 1r 0w 0i buffers 143616 3 d 2 w 0 pfs cl 3: 2186 m/s 7818 KB/s 107% cpu 0% read 9% clw threads 0r 0w 0i buffers 140787 0 d 0 w 0 pfs cl 4: 2174 m/s 2804 KB/s 77% cpu 0% read 10% clw threads 0r 0w 0i buffers 140654 0 d 2 w 0 pfs cl 5: 2127 m/s 1612 KB/s 71% cpu 0% read 9% clw threads 0r 0w 0i buffers 140949 1 d 0 w 0 pfs cl 6: 2060 m/s 544 KB/s 66% cpu 0% read 10% clw threads 0r 0w 0i buffers 141295 2 d 0 w 0 pfs cl 7: 2072 m/s 517 KB/s 65% cpu 0% read 11% clw threads 0r 0w 0i buffers 141111 1 d 0 w 0 pfs cl 8: 2105 m/s 522 KB/s 66% cpu 0% read 10% clw threads 0r 0w 0i buffers 141055 1 d 0 w 0 pfs </pre> </code> </blockquote> <p>The main meters for cluster execution are the messages-per-second (m/s), the message volume (KB/s), and the total CPU% of the processes. </p> <p>We note that CPU utilization is highly uneven and messages are short, about 1K on the average, compared to about 100K during the load. CPU would be evenly divided between the nodes if each got a share of the HTTP requests. We changed the test driver to round-robin requests between multiple end points. The work does then get evenly divided, but the speed is not affected. Also, this does not improve the message sizes since the workload consists mostly of short lookups. However, with the processes spread over multiple servers, the round-robin would be essential for CPU and especially for interconnect throughput. </p> <p>Then we try 6 Cluster at 1000 Mt. For Single User, we get 1180 m/s, 6955 KB/s, and 173% cpu. For 16 User, this is 6573 m/s, 44366 KB/s, 1470% cpu.</p> <p>This is a lot better than the figures with 6 Single, due to lower contention on the index tree, as discussed in <i><a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1e9a0b58">A Benchmarking Story</a></i>. Also Single User throughput on 6 Cluster outperforms 6 Single, due to the natural parallelism of doing the Q5 joins in parallel in each partition. The larger the scale, the more weight this has in the metric. We see this also in the average message size, i.e., the KB/s throughput is almost double while the messages/s is a bit under a third.</p> <p>The small-scale 6 Cluster run is about even with the 6 Single figure. Looking at the details, we see that the qps for Q1 in 6 Cluster is half of that on 6 Single, whereas the qps for Q5 on 6 Cluster is about double that of the 6 Single. This is as one might expect; longer queries are favored, and single row lookups are penalized.</p> <p>Looking further at the 6 Cluster status we see the cluster wait (<code>clw</code>) to be 740%. For 16 Users, this means that about half of the execution real time is spent waiting for responses from other partitions. A high figure means uneven distribution between partitions; a low figure means even. This is as expected, since many queries are concerned with just one S and its related objects.</p> <p>We will update this section once 7 Cluster is ready. This will implement vectored execution and column store inside the cluster nodes.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1d7894d0">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1e434888">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1f6b5260">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1dd29460">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1f0d78b8">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1f9a9670">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1c055370">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1dc06cd0">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> Benchmarks, Redux (part 9): BSBM With Cluster <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x18f04db0">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1ee729b8">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e2e76b8">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d75ef48">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ee518c0">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d9244b0">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 8): BSBM Explore and Update
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-09#1674
2011-03-09T17:32:47Z
2011-03-15T17:18:32-04:00
<p>We will here look at the <i>Explore and Update</i> scenario of <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1c064218">BSBM</a>. This presents us with a novel problem as the specification does not address any aspect of <a class="auto-href" href="http://dbpedia.org/resource/ACID" id="link-id0x1c1852b0">ACID</a>.</p> <p>A transaction benchmark ought to have something to say about this. The <a class="auto-href" href="http://dbpedia.org/page/SPARUL" id="link-id0x1dbca228">SPARUL</a> (also known as <a class="auto-href" href="http://dbpedia.org/resource/SPARQL" id="link-id0x1eaa4fd0">SPARQL</a>/<a class="auto-href" href="http://dbpedia.org/page/SPARUL" id="link-id0x1dd12bb0">Update</a>) language does not say anything about transactionality, but I suppose it is in the spirit of the SPARUL protocol to promise atomicity and durability.</p> <p>We begin by running <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1c5f4830">Virtuoso</a> 7 Single, with Single User and 16 User, each at scales of 100 Mt, 200 Mt, and 1000 Mt. The transactionality is default, meaning <code>SERIALIZABLE</code> isolation between <code>INSERTs</code> and <code>DELETEs</code>, and <code>READ COMMITTED</code> isolation between <code>READ</code> and any <code>UPDATE</code> transaction. (Figures for Virtuoso 6 will also be presented here in the near future, as they are the currently shipping production versions.)</p> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="3" align="center"> Virtuoso 7 Single, Full ACID <br /> (QMpH, query mixes per hour) </th> </tr> <tr> <th align="center">Scale</th> <th align="center"> Single User </th> <th align="center"> 16 User </th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 9,969 </td> <td align="center"> 65,537 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 8,646 </td> <td align="center"> 40,527 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 5,512 </td> <td align="center"> 17,293 </td> </tr> </table> <br /> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="3" align="center"> Virtuoso 6 Cluster, Full ACID <br /> (QMpH, query mixes per hour) </th> </tr> <tr> <th align="center"> Scale </th> <th align="center"> Single User </th> <th align="center"> 16 User </th> </tr> <tr> <th align="center"> 100 Mt </th> <td align="center"> 5604.520 </td> <td align="center"> 34079.019 </td> </tr> <tr> <th align="center"> 1000 Mt </th> <td align="center"> 2866.616 </td> <td align="center"> 10028.325 </td> </tr> </table> <br /> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="3" align="center"> Virtuoso 6 Single, Full ACID <br /> (QMpH, query mixes per hour) </th> </tr> <tr> <th align="center">Scale</th> <th align="center"> Single User </th> <th align="center"> 16 User </th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 7,152 </td> <td align="center"> 21,065 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 5,862 </td> <td align="center"> 16,895 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 1,542 </td> <td align="center"> 4,548 </td> </tr> </table> <p>Each run is preceded by a warm-up of 500 or 300 mixes (the exact number is not material), resulting in a warm <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x1d4f13d8">cache</a>; see <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1f8ac510">previous post on read-ahead</a> for details. All runs do 1000 <i>Explore and Update</i> mixes. The initial database is in the state following the <i>Explore</i> only runs.</p> <p>The results are in line with the <i>Explore</i> results. There is a fair amount of variability between consecutive runs; the 16 User run at 1000 Mt varies between 14K and 19K QMpH depending on the measurement. The smaller runs exhibit less variability.</p> <p>In the following we will look at transactions and at how the definition of the workload and reporting could be made complete.</p> <p>Full ACID means serializable semantic of concurrent insert and delete of the same quad. Non-transactional means that on concurrent insert and delete of overlapping sets of quads the result is undefined. Further if one logged such "transactions," the replay would give serialization although the initial execution did not, hence further confusing the issue. Considering the hypothetical use case of an e-commerce information portal, there is little chance of deletes and inserts actually needing serialization. An insert-only workload does not need serializability because an insert cannot fail. If the <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x1ec05c10">data</a> already exists the insert does nothing, if the quad does not previously exist it is created. The same applies to deletes alone. If a delete and insert overlap, serialization would be needed but the semantics implicit in the use case make this improbable.</p> <p>Read-only transactions (i.e., the <i>Explore</i> mix in the <i>Explore and Update</i> scenario) will be run as <code>READ COMMITTED</code>. These do not see uncommitted data and never block for lock wait. The reads may not be repeatable.</p> <p>Our first point of call is to determine the cost of ACID. We run 1000 mixes of <i>Explore and Update</i> at 1000 Mt. The throughput is 19214 after a warm-up of 500 mixes. This is pretty good in comparison with the diverse read-only results at this scale.</p> <p>We look at the pertinent statistics:</p> <p> <code></code> </p> <pre> SELECT TOP 5 * FROM sys_l_stat ORDER BY waits DESC; </pre> <blockquote> <code><pre> KEY_TABLE INDEX_NAME LOCKS WAITS WAIT_PCT DEADLOCKS LOCK_ESC WAIT_MSECS =============== ============= ====== ===== ======== ========= ======== ========== DB.DBA.<a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x180837c8">RDF</a>_QUAD RDF_QUAD_POGS 179205 934 0 0 0 35164 DB.DBA.RDF_IRI RDF_IRI 20752 217 1 0 0 16445 DB.DBA.RDF_QUAD RDF_QUAD_SP 9244 3 0 0 0 235 </pre> </code> </blockquote> <p>We see 934 waits with a total duration of 35 seconds on the index with the most contention. The run was 187 seconds, real time. The lock wait time is not real time since this is the total elapsed wait time summed over all threads. The lock wait frequency is a little over one per query mix, meaning a little over one per five locking transactions. </p> <p>We note that we do not get deadlocks since all inserts and deletes are in ascending key order due to vectoring. This guarantees the absence of deadlocks for single insert transactions, as long as the transaction stays within the vector size. This is always the case since the inserts are a few hundred triples at the maximum. The waits concentrate on POGS, because this is a bitmap index where the locking resolution is less than a row, and the values do not correlate with insert order. The locking behavior could be better with the column store, where we would have row level locking also for this index. This is to be seen. The column store would otherwise tend to have higher cost per random insert.</p> <p>Considering these results it does not seem crucial to "drop ACID," though doing so would save <i>some</i> time. We will now run measurements for all scales with 16 Users and ACID. </p> <p>Let us now see what the benchmark writes:</p> <p> <code></code> </p> <pre> SELECT TOP 10 * FROM sys_d_stat ORDER BY n_dirty DESC; </pre> <blockquote> <code><pre> KEY_TABLE INDEX_NAME TOUCHES READS READ_PCT N_DIRTY N_BUFFERS =========================== ============================ ========= ======= ======== ======= ========= DB.DBA.RDF_QUAD RDF_QUAD_POGS 763846891 237436 0 58040 228606 DB.DBA.RDF_QUAD RDF_QUAD 213282706 1991836 0 30226 1940280 DB.DBA.RDF_OBJ RO_VAL 15474 17837 115 13438 17431 DB.DBA.RO_START RO_START 10573 11195 105 10228 11227 DB.DBA.RDF_IRI RDF_IRI 61902 125711 203 7705 121300 DB.DBA.RDF_OBJ RDF_OBJ 23809053 3205963 13 636 3072517 DB.DBA.RDF_IRI DB_DBA_RDF_IRI_UNQC_RI_ID 3237687 504486 15 340 488797 DB.DBA.RDF_QUAD RDF_QUAD_SP 89995 70446 78 99 68340 DB.DBA.RDF_QUAD RDF_QUAD_OP 19440 47541 244 66 45583 DB.DBA.VTLOG_DB_DBA_RDF_OBJ VTLOG_DB_DBA_RDF_OBJ 3014 1 0 11 11 DB.DBA.RDF_QUAD RDF_QUAD_GS 1261 801 63 10 751 DB.DBA.RDF_PREFIX RDF_PREFIX 14 168 1120 1 153 DB.DBA.RDF_PREFIX DB_DBA_RDF_PREFIX_UNQC_RP_ID 1807 200 11 1 200 </pre> </code> </blockquote> <p>The most dirty pages are on the <code>POGS</code> index, which is reasonable; values are spread out at random. After this we have the <code>PSOG</code> index, likely because of random deletes. New IRIs tend to get consecutive numbers and do not make many dirty pages. Literals come next, with the index from leading string or hash of the literal to id leading, as one would expect, again because of values being distributed at random. After this come IRIs. The distribution of updates is generally as one would expect.</p> <p align="center">* * *</p> <p>Going back to BSBM, at least the following aspects of the benchmark have to be further specified:</p> <ul> <li> <p> <b>Disclosure of ACID properties.</b> If the benchmark required full ACID many would not run this at all. Besides full ACID is not necessarily an absolute requirement based on the hypothetical usage scenario of the benchmark. However, when publishing numbers the guarantees that go with the numbers must be made explicit. This includes logging, checkpoint frequency or equivalent etc.</p> </li> <li> <p> <b>Steady state.</b> The working set of the <i>Update</i> mix is different from that of the <i>Explore</i> mixes. This touches more indices than <i>Explore</i>. The <i>Explore</i> warm-up is in part good but does not represent steady state.</p> </li> <li> <p> <b>Checkpoint and sustained throughput.</b> Benchmarks involving update generally have rules for checkpointing the state and for sustained throughput. In specific, the throughput of an update benchmark cannot rely on never flushing to persistent storage. Even bulk load must be timed with a checkpoint guaranteeing durability at the end. A steady update stream should be timed with a test interval of sufficient length involving a few checkpoints; for example, a minimum duration of 30 minutes with no less than 3 completed checkpoints in the interval with at least 9 minutes between the end of one and the start of the next. Not all DBMSs work with logs and checkpoints, but if an alternate scheme is used then this needs to be described.</p> </li> <li> <p> <b>Memory and warm-up issues.</b>We have seen the test data generator run out of memory when trying to generate update streams of meaningful length. Also the test driver should allow running updates in timed and non-timed mode (warm-up).</p> </li> </ul> <p>With an update benchmark, many more things need to be defined, and the set-up becomes more system specific, than with a read-only workload. We will address these shortcomings in the measurement rules proposal to come. Especially with update workloads, the vendors need to provide tuning expertise; however, this will not happen if the benchmark does not properly set the expectations. If benchmarks serve as a catalyst for clearly defining how things are to be set up, then they will have served the end user.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1de61db8">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1f9f96f8">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1f89eeb0">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1ad83f30">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1de62178">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1b2ec018">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1ae6f028">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> Benchmarks, Redux (part 8): BSBM Explore and Update <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x132605c0">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1a9871b0">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1baa20f8">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e25a840">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1b53db20">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e7ce520">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1b18f400">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 7): What Does BSBM Explore Measure?
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-07#1672
2011-03-07T23:39:22Z
2011-03-14T17:57:20-04:00
<p>We will here analyze what the <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1db49f28">BSBM</a> Explore workload does. This is necessary in order to compare benchmark results at different scales. Historically, BSBM had a Query 6 whose share of the metric approached 100% as scale increased. The present mix does not have this query, but different queries still have different relative importance at different scales.</p> <p>We will here look at database-running statistics for BSBM at different scales. Finally, we look at <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x1f150460">CPU</a> profiles.</p> <p>But first, let us see what BSBM reads in general. The system is in steady state after around 1500 query mixes; after this the working set does not shift much. After several thousand query mixes, we have:</p> <p> <code>SELECT TOP 10 * FROM sys_d_stat ORDER BY reads DESC;</code> </p> <blockquote> <code><pre> KEY_TABLE INDEX_NAME TOUCHES READS READ_PCT N_DIRTY N_BUFFERS ================= ============================ ========== ======= ======== ======= ========= DB.DBA.<a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1ddb0b50">RDF</a>_OBJ RDF_OBJ 114105938 3302150 2 0 3171275 DB.DBA.RDF_QUAD RDF_QUAD 977426773 2041156 0 0 1970712 DB.DBA.RDF_IRI DB_DBA_RDF_IRI_UNQC_RI_ID 8250414 509239 6 15 491631 DB.DBA.RDF_QUAD RDF_QUAD_POGS 3677233812 183860 0 0 175386 DB.DBA.RDF_IRI RDF_IRI 32 99710 302151 5 95353 DB.DBA.RDF_QUAD RDF_QUAD_OP 30597 51593 168 0 48941 DB.DBA.RDF_QUAD RDF_QUAD_SP 265474 47210 17 0 46078 DB.DBA.RDF_PREFIX DB_DBA_RDF_PREFIX_UNQC_RP_ID 6020 212 3 0 212 DB.DBA.RDF_PREFIX RDF_PREFIX 0 167 16700 0 157 </pre> </code> </blockquote> <p>The first column is the table, then the index, then the number of times a row was found. The fourth number is the count of disk pages read. The last number is the count of 8K buffer pool pages in use for caching pages of the index in question. Note that the index is clustered, i.e., there is no table <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x1d4f9808">data</a> structure separate from the index. Most of the reads are for strings or RDF literals. After this comes the <code>PSOG</code> index for getting a property value given the subject. After this, but much lower, we have lookups of IRI strings given the ID. The index from object value to subject is used the most but the number of pages is small; only a few properties seem to be concerned. The rest is minimal in comparison.</p> <p>Now let us reset the counts and see what the steady state I/O profile is.</p> <p> <code>SELECT key_stat (key_table, name_part (key_name, 2), 'reset') FROM sys_keys WHERE key_migrate_to IS NULL;</code> </p> <p> <code>SELECT TOP 10 * FROM sys_d_stat ORDER BY reads DESC;</code> </p> <blockquote> <code><pre> KEY_TABLE INDEX_NAME TOUCHES READS READ_PCT N_DIRTY N_BUFFERS ================= ============================ ========== ======= ======== ======= ========= DB.DBA.RDF_OBJ RDF_OBJ 30155789 79659 0 0 3191391 DB.DBA.RDF_QUAD RDF_QUAD 259008064 8904 0 0 1948707 DB.DBA.RDF_QUAD RDF_QUAD_SP 68002 7730 11 0 53360 DB.DBA.RDF_IRI RDF_IRI 12 5415 41653 6 98804 DB.DBA.RDF_QUAD RDF_QUAD_POGS 975147136 1597 0 0 173459 DB.DBA.RDF_IRI DB_DBA_RDF_IRI_UNQC_RI_ID 2213525 1286 0 17 485093 DB.DBA.RDF_QUAD RDF_QUAD_OP 7999 904 11 0 48568 DB.DBA.RDF_PREFIX DB_DBA_RDF_PREFIX_UNQC_RP_ID 1494 1 0 0 213 </pre> </code> </blockquote> <p>Literal strings dominate. The <code>SP</code> index is used only for situations where the <code>P</code> is not specified, i.e., the <code>DESCRIBE</code> query. Based on this, I/O seems to be attributable mostly to this. The first <code>RDF_IRI</code> represents translations from string to IRI id; the second represents translations from IRI id to string. The touch count for the first <code>RDF_IRI</code> is not properly recorded, hence the miss % is out of line. We see <code>SP</code> missing the <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x17d2e670">cache</a> the most since its use is infrequent in the mix.</p> <p>We will next look at query processing statistics. For this we introduce a new meter.</p> <p>The <code>db_activity</code> <a class="auto-href" href="http://dbpedia.org/resource/SQL" id="link-id0x1d4915b8">SQL</a> function provides a session-by-session cumulative statistic of activity. The fields are: </p> <ul> <li> <b><code>rnd</code> </b> - Count of <i>random index lookups</i>. Each first row of a select or insert counts as one, regardless of whether something was found.</li> <li> <b><code>seq</code> </b> - Count of <i>sequential rows</i>. Every move to next row on a cursor counts as 1, regardless of whether conditions match.</li> <li> <b><code>same seg</code> </b> - For column store only; counts how many times the next row in a vectored join using an index falls in the <i>same segment</i> as the previous random access. A segment is the stretch of rows between entries in the sparse top level index on the column projection.</li> <li> <b><code>same pg</code> </b> - Counts how many times a vectored index join finds the next match on the <i>same page</i> as the previous one.</li> <li> <b><code>same par</code> </b> - Counts how many times the next lookup in a vectored index join falls on a different page than the previous but still under the <i>same parent</i>.</li> <li> <b><code>disk</code> </b> - Counts how many <i>disk reads</i> were made, including any speculative reads initiated.</li> <li> <b><code>spec disk</code> </b> - Counts <i>speculative disk reads</i>.</li> <li> <b><code>messages</code> </b> - Counts <i>cluster interconnect messages</i> </li> <li> <b><code>B (KB, MB, GB)</code> </b> - is the <i>total length</i> of the cluster interconnect messages.</li> <li> <b><code>fork</code> </b> - Counts how many times a <i>thread was forked (started)</i> for query parallelization.</li> </ul> <p>The numbers are given with 4 significant digits and a scale suffix. G is 10^9 (1,000,000,000); M is 10^6 (1,000,000), K is 10^3 (1,000).</p> <p>We run 2000 query mixes with 16 Users. The special <code><a class="auto-href" href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x1bf7f318">http</a></code> account keeps a cumulative account of all activity on web server threads.</p> <blockquote> <p> <code>SELECT db_activity (2, 'http');</code> </p> <p> <code>1.674G rnd 3.223G seq   0 same seg 1.286G same pg 314.8M same par 6.186M disk 6.461M spec disk   0B /   0 messages 298.6K fork</code> </p> </blockquote> <p>We see that random access dominates. The <code>seq</code> number is about twice the <code>rnd</code> number, meaning that the average random lookup gets two rows. Getting a row at random obviously takes more time than getting the next row. Since the index used is row-wise, the <code>same seg</code> is 0; the <code>same pg</code> indicates that 77% of the random accesses fall on the same page as the previous random access; most of the remaining random accesses fall under the same parent as the previous one.</p> <p>There are more speculative reads than disk reads which is an artifact of counting some concurrently speculated reads twice. This does indicate that speculative reads dominate. This is because a large part of the run was in the warm-up state with aggressive speculative reading. We reset the counts and run another 2000 mixes.</p> <p>Now let us look at the same reading after 2000 mixes, 16 user at 100Mt.</p> <blockquote> <p> <code>234.3M rnd 420.5M seq   0 same seg  188.8M same pg 29.09M same par 808.9K disk 919.9K spec disk   0B /   0 messages 76K fork</code> </p> </blockquote> <p>We note that the ratios between the random and sequential and same page/parent counts are about the same. The sequential number looks to be even a bit smaller in proportion. The count of random accesses for the 100Mt run is 14% of the count for the 1000Mt run. The count of query parallelization threads is also much lower since it is worthwhile to schedule a new thread only if there are at least a few thousand operations to perform on it. The precise criterion for making a thread is that according to the cost model guess, the thread must have at least 5ms worth of work.</p> <p>We note that the 100 Mt throughput is a little over three-times that of the 1000 Mt throughput, as reported before. We might justifiably ask why the 100 Mt run is not seven-times faster instead, for this much less work. </p> <p>We note that for one-off random access, it makes no real difference whether the tree has 100 M or 1000 M rows; this translates to roughly 27 vs 30 comparisons, so the depth of the tree is not a factor <i>per se</i>. Besides, vectoring makes the tree often look only one or two levels deep, so the total row count matters even less there.</p> <p>To elucidate this last question, we look at the CPU profiles. We take an <a href="http://oprofile.sourceforge.net/about/" id="link-id0x1efb3360">oprofile</a> of 100 Single User mixes at both scales.</p> For 100 Mt: <blockquote> <code><pre> 61161 10.1723 cmpf_iri64n_iri64n_anyn_gt_lt 31321 5.2093 box_equal 19027 3.1646 sqlo_parse_tree_has_node 15905 2.6453 dk_alloc 15647 2.6024 itc_next_set_neq 12702 2.1126 itc_vec_split_search 12487 2.0768 itc_dive_transit 11450 1.9044 itc_bm_vec_row_check 10646 1.7706 itc_page_rcf_search 9223 1.5340 id_hash_get 9215 1.5326 gen_qsort 8867 1.4748 sqlo_key_part_best 8807 1.4648 itc_param_cmp 8062 1.3409 cmpf_iri64n_iri64n 6820 1.1343 sqlo_in_list 6005 0.9987 dc_iri_id_cmp 5905 0.9821 dk_free_tree 5801 0.9648 box_hash 5509 0.9163 dks_esc_write 5444 0.9054 sql_tree_hash_1 </pre> </code> </blockquote> For 1000 Mt <blockquote> <code><pre> 754331 31.4149 cmpf_iri64n_iri64n_anyn_gt_lt 146165 6.0872 itc_vec_split_search 144795 6.0301 itc_next_set_neq 131671 5.4836 itc_dive_transit 110870 4.6173 itc_page_rcf_search 66780 2.7811 gen_qsort 66434 2.7667 itc_param_cmp 58450 2.4342 itc_bm_vec_row_check 55213 2.2994 dk_alloc 47793 1.9904 cmpf_iri64n_iri64n 44277 1.8440 dc_iri_id_cmp 39489 1.6446 cmpf_int64n 36880 1.5359 dc_append_bytes 36601 1.5243 dv_compare 31286 1.3029 dc_any_value_prefetch 25457 1.0602 itc_next_set 20852 0.8684 box_equal 19895 0.8285 dk_free_tree 19698 0.8203 itc_page_insert_search 19367 0.8066 dc_copy </pre> </code> </blockquote> <p>The top function in both is the compare for an equality of two leading IRIs and a range for the trailing any. This corresponds to the range check in Q5. At the larger scale this is three times more important. At the smaller scale, the share of query <a class="auto-href" href="http://dbpedia.org/resource/Program_optimization" id="link-id0x1bf8ca38">optimization</a> is about 6.5 times greater. The top function in this category is <code>box_equal</code> with 5.2% vs 0.87%. The remaining SQL compiler functions are all in proportion to this, totaling 14.3% of the 100 Mt top-20 profile.</p> <p>From this sample it appears ten times more scale is seven times more database operations. This is not taken into account in the metric. Query compilation is significant at the small end, and no longer significant at 1000 Mt. From these numbers, we could say that <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1be12350">Virtuoso</a> is about two times more efficient in terms of database operation throughput at 1000 Mt than at 100 Mt.</p> <p>We may conclude that different BSBM scales measure different things. The <a class="auto-href" href="http://www.tpc.org/" id="link-id0x17eb98a0">TPC</a> workloads are relatively better in that they have a balance between metric components that stay relatively constant across a large range of scales.</p> <p>This is not necessarily something that should be fixed in the BSBM Explore mix. We must however take these factors better into account in developing the BI mix.</p> <p>Let us also remember that BSBM Explore is a relational workload. Future posts in this series will outline how we propose to make RDF-friendlier benchmarks. </p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1a9bcff8">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1d3e5470">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1de94770">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1ea66470">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1f1118d8">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1d1c0cd8">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> Benchmarks, Redux (part 7): What Does BSBM Explore Measure? <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1aaf4180">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1a957610">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x127e75c8">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1c9400f0">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d2c1d68">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ea1fb40">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1c073a10">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1c5541e8">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 6): BSBM and I/O, continued
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-07#1670
2011-03-07T22:36:24Z
2011-03-14T17:57:06.000001-04:00
<p>In the words of Jim Gray, disks have become tapes. By this he means that a disk is really only good for sequential access. For this reason, the SSD extent read ahead was incomparably better. We note that in the experiment, every page in the general area of the database the experiment touched would in time be touched, and that the whole working set would end up in memory. Therefore no speculative read would be wasted. Therefore it stands to reason to read whole extents.</p> <p>So I changed the default behavior to use a very long window for triggering read-ahead as long as the buffer pool was not full. After the initial filling of the buffer pool, the read ahead would require more temporal locality before kicking in. </p> <p>Still, the scheme was not really good since the rest of the extent would go for background-read and the triggering read would be done right then, leading to extra seeks. Well, this is good for latency but bad for throughput. So I changed this too, going to an "elevator only" scheme where reads that triggered read-ahead would go with the read-ahead batch. Reads that did not trigger read-ahead would still be done right in place, thus favoring latency but breaking any sequentiality with its attendant 10+ ms penalty.</p> <p>We keep in mind that the test we target is <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x17c88010">BSBM</a> warm-up time, which is purely a throughput business. One could have timeouts and could penalize queries that sacrificed too much latency to throughput.</p> <p>We note that even for this very simple metric, just reading the allocated database pages from start to end is not good since a large number of pages in fact never get read during a run.</p> <p>We further note that the vectored read-ahead without any speculation will be useful as-is for cases with few threads and striping, since at least one thread's random I/Os get to go to multiple threads. The benefit is less in multiuser situations where disks are randomly busy anyhow. </p> <p>In the previous I/O experiments, we saw that with vectored read ahead and no speculation, there were around 50 pages waiting for I/O at all times. With an easily-triggered extent read-ahead, there were around 4000 pages waiting. The more pages are waiting for I/O, the greater the benefit from the elevator algorithm of servicing I/O in order of file offset. </p> <p>In <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1c51fae0">Virtuoso</a> 5 we had a trick that would, if the buffer pool was not full, speculatively read every uncached sibling of every index tree node it visited. This filled the <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x1d6a0cf0">cache</a> quite fast, but was useless after the cache was full. The extent read ahead first implemented in 6 was less aggressive, but would continue working with full cache and did in fact help with shifts in the working set.</p> <p>The next logical step is to combine the vector and extent read-ahead modes. We see what pages we will be getting, then take the distinct extents; if we have been to this extent within the time window, we just add all the uncached allocated pages of the extent to the batch.</p> <p>With this setting, especially at the start of the run, we get large read-ahead batches and maintain I/O queues of 5000 to 20000 pages. The SSD starting time drops to about 120 seconds from cold start to reach 1200% <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x1d295448">CPU</a>. We see transfer rates of up to 150 MB/s per SSD. With HDDs, we see transfer rates around 14 MB/s per drive, mostly reading chunks of an average of seventy-one (71) 8K pages.</p> <p>The BSBM workload does not offer better possibilities for <a class="auto-href" href="http://dbpedia.org/resource/Program_optimization" id="link-id0x1aca8b40">optimization</a>, short of pre-reading the whole database, which is not practical at large scales. </p> <h2>Some Details</h2> <p>First we start from cold disk, with and without mandatory read of the whole extent on the touch.</p> <p>Without any speculation but with vectored read-ahead, here are the times for the first 11 query mixes:</p> <blockquote> <code><pre> 0: 151560.82 ms, total: 151718 ms 1: 179589.08 ms, total: 179648 ms 2: 71974.49 ms, total: 72017 ms 3: 102701.73 ms, total: 102729 ms 4: 58834.41 ms, total: 58856 ms 5: 65926.34 ms, total: 65944 ms 6: 68244.69 ms, total: 68274 ms 7: 39197.15 ms, total: 39215 ms 8: 45654.93 ms, total: 45674 ms 9: 34850.30 ms, total: 34878 ms 10: 100061.30 ms, total: 100079 ms </pre> </code> </blockquote> <p>The average CPU during this time was 5%. The best read throughput was 2.5 MB/s; the average was 1.35 MB/s. The average disk read was 16 ms. </p> <p>With vectored read-ahead and full extents only, i.e., max speculation:</p> <blockquote> <code><pre> 0: 178854.23 ms, total: 179034 ms 1: 110826.68 ms, total: 110887 ms 2: 19896.11 ms, total: 19941 ms 3: 36724.43 ms, total: 36753 ms 4: 21253.70 ms, total: 21285 ms 5: 18417.73 ms, total: 18439 ms 6: 21668.92 ms, total: 21690 ms 7: 12236.49 ms, total: 12267 ms 8: 14922.74 ms, total: 14945 ms 9: 11502.96 ms, total: 11523 ms 10: 15762.34 ms, total: 15792 ms ... 90: 1747.62 ms, total: 1761 ms 91: 1701.01 ms, total: 1714 ms 92: 1300.62 ms, total: 1318 ms 93: 1873.15 ms, total: 1886 ms 94: 1508.24 ms, total: 1524 ms 95: 1748.15 ms, total: 1761 ms 96: 2076.92 ms, total: 2090 ms 97: 2199.38 ms, total: 2212 ms 98: 2305.75 ms, total: 2319 ms 99: 1771.91 ms, total: 1784 ms Scale factor: 2848260 Number of warmup runs: 0 Seed: 808080 Number of query mix runs (without warmups): 100 times min/max Querymix runtime: 1.3006s / 178.8542s Elapsed runtime: 872.993 seconds QMpH: 412.374 query mixes per hour </pre> </code> </blockquote> <p>The peak throughput is 91 MB/s, with average around 50 MB/s; CPU average around 50%.</p> <p>We note that the latency of the first query mix is hardly greater than in the non-speculative run, but starting from mix 3 the speed is clearly better. </p> <p>Then the same with cold SSDs. First with no speculation:</p> <blockquote> <code><pre> 0: 5177.68 ms, total: 5302 ms 1: 2570.16 ms, total: 2614 ms 2: 1353.06 ms, total: 1391 ms 3: 1957.63 ms, total: 1978 ms 4: 1371.13 ms, total: 1386 ms 5: 1765.55 ms, total: 1781 ms 6: 1658.23 ms, total: 1673 ms 7: 1273.87 ms, total: 1289 ms 8: 1355.19 ms, total: 1380 ms 9: 1152.78 ms, total: 1167 ms 10: 1787.91 ms, total: 1802 ms ... 90: 1116.25 ms, total: 1128 ms 91: 989.50 ms, total: 1001 ms 92: 833.24 ms, total: 844 ms 93: 1137.83 ms, total: 1150 ms 94: 969.47 ms, total: 982 ms 95: 1138.04 ms, total: 1149 ms 96: 1155.98 ms, total: 1168 ms 97: 1178.15 ms, total: 1193 ms 98: 1120.18 ms, total: 1132 ms 99: 1013.16 ms, total: 1025 ms Scale factor: 2848260 Number of warmup runs: 0 Seed: 808080 Number of query mix runs (without warmups): 100 times min/max Querymix runtime: 0.8201s / 5.1777s Elapsed runtime: 127.555 seconds QMpH: 2822.321 query mixes per hour </pre> </code> </blockquote> <p>The peak I/O is 45 MB/s, with average 28.3 MB/s; CPU average is 168%.</p> <p>Now, SSDs with max speculation.</p> <blockquote> <code><pre> 0: 44670.34 ms, total: 44809 ms 1: 18490.44 ms, total: 18548 ms 2: 7306.12 ms, total: 7353 ms 3: 9452.66 ms, total: 9485 ms 4: 5648.56 ms, total: 5668 ms 5: 5493.21 ms, total: 5511 ms 6: 5951.48 ms, total: 5970 ms 7: 3815.59 ms, total: 3834 ms 8: 4560.71 ms, total: 4579 ms 9: 3523.74 ms, total: 3543 ms 10: 4724.04 ms, total: 4741 ms ... 90: 673.53 ms, total: 685 ms 91: 534.62 ms, total: 545 ms 92: 730.81 ms, total: 742 ms 93: 1358.14 ms, total: 1370 ms 94: 1098.64 ms, total: 1110 ms 95: 1232.20 ms, total: 1243 ms 96: 1259.57 ms, total: 1273 ms 97: 1298.95 ms, total: 1310 ms 98: 1156.01 ms, total: 1166 ms 99: 1025.45 ms, total: 1034 ms Scale factor: 2848260 Number of warmup runs: 0 Seed: 808080 Number of query mix runs (without warmups): 100 times min/max Querymix runtime: 0.4725s / 44.6703s Elapsed runtime: 269.323 seconds QMpH: 1336.683 query mixes per hour </pre> </code> </blockquote> <p>The peak I/O is 339 MB/s, with average 192 MB/s; average CPU is 121%.</p> <p>The above was measured with the read-ahead thread doing single-page reads. We repeated the test with merging reads with small differences. The max IO was 353 MB/s, and average 173 MB/s; average CPU 113%.</p> <p>We see that the start latency is quite a bit longer than without speculation and the CPU % is lower due to higher latency of individual I/O. The I/O rate is fair. We would expect more throughput however. </p> <p>We find that a supposedly better use of the API, doing single requests of up to 100 pages instead of consecutive requests of 1 page, does not make a lot of difference. The peak I/O is a bit higher; overall throughput is a bit lower.</p> <p>We will have to retry these experiments with a better controller. We have at no point seen anything like the 50K 4KB random I/Os promised for the SSDs by the manufacturer. We know for a fact that the controller gives about 700 MB/s sequential read with <code>cat file /dev/null</code> and two drives busy. With 4 drives busy, this does not get better. The best 30 second stretch we saw in a multiuser BSBM warm-up was 590 MB/s, which is consistent with the <code>cat</code> to <code>/dev/null</code> figure. We will later test with 8 SSDs with better controllers. </p> <p>Note that the average I/O and CPU are averages over 30 second measurement windows; thus for short running tests, there is some error from the window during which the activity ended. </p> <p>Let us now see if we can make a BSBM instance warm up from disk in a reasonable time. We run 16 users with max speculation. We note that after reading 7,500,000 buffers we are not entirely free of disk. The max speculation read-ahead filled the cache in 17 minutes, with an average of 58 MB/s. After the cache is filled, the system shifts to a more conservative policy on extent read-ahead; one which in fact never gets triggered with the BSBM <i>Explore</i> in steady state. The vectored read-ahead is kept on since this by itself does not read pages that are not needed. However, the vectored read-ahead does not run either, because the <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x1c9bca60">data</a> that is accessed in larger batches is already in memory. Thus there remains a trickle of an average 0.49 MB/s from disk. This keeps CPU around 350%. With SSDs, the trickle is about 1.5 MB/s and CPU is around 1300% in steady state. Thus SSDs give approximately triple the throughput in a situation where there is a tiny amount of continuous random disk access. The disk access in question is 80% for retrieving <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1c05e280">RDF</a> literal strings, presumably on behalf of the <code>DESCRIBE</code> query in the mix. This query touches things no other query touches and does so one subject at a time, in a way that can neither be anticipated nor optimized.</p> <p>The Virtuoso 7 column store will deal with this better because it is more space efficient overall. If we apply stream-compression to literals, these will go in under half the space, while quads will go in maybe one-quarter the space. Thus 3000 Mt all from memory should be possible with 72 GB RAM. 1000 Mt row-wise did fit in in 72 GB RAM except for the random literals accessed by the the <code>DESCRIBE</code>. This alone drops throughput to under a third of the memory-only throughput if using HDDs. SSDs, on the other hand, can largely neutralize this effect.</p> <h2>Conclusions</h2> <p>We have looked at basics of I/O. SSDs have been found to be a readily available solution to I/O bottlenecks without need for reconfiguration or complex I/O policies. We have been able to get a decent read rate under conditions of server warm-up or shift of working set even with HDDs.</p> <p>More advanced I/O matters will be covered with the column store. We note that the techniques discussed here apply identically to rows and columns.</p> <p>As concerns BSBM, it seems appropriate to include a warm-up time. In practice, this means that the store just must eagerly pre-read. This is not hard to do and can be quite useful.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1b4342b0">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1d3e7388">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x153c7ba8">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1da11d98">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1d25d630">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs</a> </li> <li> Benchmarks, Redux (part 6): BSBM and I/O, continued <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1f1f5ee8">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1cd44938">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1d51f848">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x13d333c0">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1e77a5e8">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ea1fb40">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e7786c8">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1f8a37f8">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1c69e018">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-07#1668
2011-03-07T19:17:36Z
2011-03-14T17:56:52-04:00
<p>In the context of database benchmarks we cannot ignore I/O, as pretty much has been done so far by <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1ea17348">BSBM</a>. </p> <p>There are two approaches:</p> <ol> <li> <p>run twice or otherwise make sure one runs from memory and forget about I/O, or</p> </li> <li> <p>make rules and metrics for warm-up.</p> </li> </ol> <p>We will see if the second is possible with BSBM.</p> <p>From this starting point, we look at various ways of scheduling I/O in <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x125c4f90">Virtuoso</a> using a 1000 Mt BSBM database on sets of each of HDDs (hard disk devices) and SSDs (solid-state storage devices). We will see that SSDs in this specific application can make a significant difference. </p> <p>In this test we have the same 4 stripes of a 1000 Mt BSBM database on each of two storage arrays.</p> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="9" align="center">Storage Arrays</th> </tr> <tr> <th align="center"> Type </th> <th align="center"> Quantity </th> <th align="center"> Maker </th> <th align="center"> Size </th> <th align="center"> Speed </th> <th align="center"> Interface speed </th> <th align="center"> Controller </th> <th align="center"> Drive <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x1cab1358">Cache</a> </th> <th align="center"> RAID </th> </tr> <tr> <td align="center"> SSD </td> <td align="center"> 4 </td> <td align="center"> Crucial </td> <td align="center"> 128 GB </td> <td align="center"> N/A </td> <td align="center"> 6Gbit SATA </td> <td align="center"> RocketRaid 640 </td> <td align="center"> 128 MB </td> <td align="center"> None </td> </tr> <tr> <td align="center"> HDD </td> <td align="center"> 4 </td> <td align="center"> Samsung </td> <td align="center"> 1000 GB </td> <td align="center"> 7200 RPM </td> <td align="center"> 3Gbit SATA </td> <td align="center"> <a class="auto-href" href="http://dbpedia.org/resource/Intel_Corporation" id="link-id0x1ab6edd8">Intel</a> ICH on Supermicro motherboard </td> <td align="center"> 16 MB </td> <td align="center"> None </td> </tr> </table> <p>We make sure that the files are not in OS cache by filling it with other big files, reading a total of 120 GB off SSDs with <code>`cat file > /dev/null`</code>. </p> <p>The configuration files are as in the report on the 1000 Mt run. We note as significant that we have a few file descriptors for each stripe, and that read-ahead for each is handled by its own thread.</p> <p>Two different read-ahead schemes are used: </p> <ul> <li> <p>With 6 Single, if a 2MB extent gets a second read within a given time after the first, the whole extent is scheduled for background read.</p> </li> <li> <p>With 7 Single, as an index search is vectored, we know a large number of values to fetch at one time and these values are sorted into an ascending sequence. Therefore, by looking at a node in an index tree, we can determine which sub-trees will be accessed and schedule these for read-ahead, skipping any that will not be accessed.</p> </li> </ul> <p>In either model, a sequential scan touching more than a couple of consecutive index leaf pages triggers a read-ahead, to the end of the scanned range or to the next 3000 index leaves, whichever comes first. However, there are no sequential scans of significant size in BSBM.</p> <p>There are a few different possibilities for the physical I/O: </p> <ol> <li> <p>Using a separate read system call for each page. There may be several open file descriptors on a file so that many such calls can proceed concurrently on different threads; the OS will order the operations.</p> </li> <li> <p>A thread finds it needs a page and reads it.</p> </li> <li> <p>Using Unix asynchronous I/O, <code>aio.h</code>, with the <code>aio_*</code> and <code>lio_listio</code> functions.</p> </li> <li> <p>Using single-read system calls for adjacent pages. In this way, the drive sees longer requests and should give better throughput. If there are short gaps in the sequence, the gaps are also read, wasting bandwidth but saving on latency.</p> </li> </ol> <p>The two latter apply only to bulk I/O that are scheduled on background threads, one per independently-addressable device (HDD, SSD, or RAID-set). These bulk-reads operate on an elevator model, keeping a sorted queue of things to read or write and moving through this queue from start to end. At any time, the queue may get more work from other threads.</p> <p>There is a further choice when seeing single-page random requests. They can either go to the elevator or they can be done in place. Taking the elevator is presumably good for throughput but bad for latency. In general, the elevator should have a notion of fairness; these matters are discussed in the <a href="http://www.cwi.nl/" id="link-id0x1f62abb8">CWI collaborative scan paper</a>. Here we do not have long queries, so we do not have to talk about elevator policies or scan sharing; there are no scans. We may touch on these questions later with the column store, the BSBM BI mix, and <a class="auto-href" href="http://www.tpc.org/" id="link-id0x1bfb17c0">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/TPC-H" id="link-id0x1e76bfc8">H</a>.</p> <p>While we may know principles, I/O has always given us surprises; the only way to optimize this is to measure.</p> <p>The metric we try to optimize here is the time it takes for a multiuser BSBM run starting from cold cache to get to 1200% <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x1d7b1d10">CPU</a>. When running from memory, the CPU is around 1350% for the system in question. </p> <p>This depends on getting I/O throughput, which in turn depends on having a lot of speculative reading since the workload itself does not give any long stretches to read. </p> <p>The test driver is set at 16 clients, and the run continues for 2000 query mixes or until target throughput is reached. Target throughput is deemed reached after the first 20 second stretch with CPU at 1200% or higher.</p> <p>The meter is a stored procedure that records the CPU time, count of reads, cumulative elapsed time spent waiting for I/O, and other metrics. The code for this procedure (for 7 Single; this file will not work on Virtuoso 6 or earlier) is <a href="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/BenchmarksReduxSupportingFiles/ldmeter.sql" id="link-id0x1b5adb08">available here</a>. </p> <p>The database space allocation gives each index a number of 2MB segments, each with 256 8K pages. When a page splits, the new page is allocated from the same extent if possible, or from a specific second extent which is designated as the overflow extent of this extent. This scheme provides for a sort of pseudo-locality within extents over random insert order. Thus there is a chance that pre-reading an extent will get key values in the same range a the ones on the page being requested in the first place. At least the pre-read pages will be from the same index tree. There are insertion orders that do not create good locality with this allocation scheme, though. In order to generally improve locality, one could shuffle pages of an all-dirty subtree before writing this out so as to have physical order match key order. We will look at some tricks in this vein with the column store.</p> <p>For the sake of simplicity we only run 7 Single with the 1000 Mt scale.</p> <p>The first experiment was with SSDs and the vectored read-ahead. The target throughput was reached after 280 seconds. </p> <p>The next test was with HDDs and extent read-ahead. One hour into the experiment, the CPU was about 70% after processing around 1000 query mixes. It might have been hours before HDD reads became rare enough for hitting 1200% CPU. The test was not worth continuing.</p> <p>The result with HDDs and vectored read-ahead would be worse since vectored read-ahead leads to smaller read-ahead batches and to less contiguous read patterns. The individual read times here, are over twice the individual read times with per-extent read-ahead. The fact that vectored read-ahead does not read potentially unneeded pages makes no difference. Hence this test is also not worth running to completion.</p> <p>There are other possibilities for improving HDD I/O. If only 2MB read requests are made, a transfer will be about 20 ms at a sequential transfer speed of 50 MB/s. Then seeking to the next 2MB extent will be a few ms, most often less than 20, so the HDD should give at least half the nominal throughput.</p> <p>We note that, when reading sequential 8K pages inside a single 2MB (256 page) extent, the seek latency is not 0 as one would expect but an extreme 5 ms. One would think that the drive would buffer a whole track, and a track would hold a large number of 2MB sections, but apparently this is not so. </p> <p>Therefore, now if we have a sequential read pattern that is more dense than 1 page out of 10, we read all the pages and just keep the ones we want.</p> <p>So now we set the read-ahead to merge reads that fall within 10 pages. This wastes bandwidth, but supposedly saves on latency. We will see. </p> <p>So we try, and we find that read-ahead does not account for most pages since it does not get triggered. Thus, we change the triggering condition to be the 2nd read to fall in the extent within 20 seconds of the first.</p> <p>The HDDs were in all cases 700% busy for 4 HDDs. But with the new setting we get longer requests, most often full extents, which gets a per-HDD transfer rate of about 5 MB/s. With the looser condition for starting read-ahead, 89% of all pages were read in a read-ahead batch. We see the I/O throughput decrease during the run because there are more single-page reads that do not trigger extent read-ahead. So HDDs have 1.7 concurrent operations pending, but the batch size drops, dropping the throughput.</p> <p> </p> <p>Thus with the best settings, the test with 2000 query mixes finishes in 46 minutes, and the CPU utilization is steadily increasing, hitting 392% for the last minute. In comparison, with SSDs and our worst read-ahead setting we got 1200% CPU in under 5 minutes from cold start. The I/O system can be further tuned; for example, by only reading full extents as long as the buffer pool is not full. In the next post we will measure some more. </p> <p> </p> <h3>BSBM Note </h3> <p>We look at query times with semi-warm cache, with CPU around 400%. We note that Q8-Q12 are especially bad. Q5 runs at about half speed. Q12 runs at under 1/10th speed. The relatively slowest queries appear to be single-instance lookups. Nothing short of the most aggressive speculative reading can help there. Neither query nor workload has any exploitable pattern. Therefore if an I/O component is to be included in a BSBM metric, the only way to score in this is to use speculative read to the maximum.</p> <p>Some of the queries take consecutive property values of a single instance. One could parallelize this pipeline, but this would be a one-off and would make sense only when reading from storage (whether HDD, SSD, or otherwise). Multithreading for single rows is not worth the overhead.</p> <p>A metric for BSBM warm-up is not interesting for database science, but may still be of practical interest in the specific case of <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1d6371d8">RDF</a> stores. Specially reading large chunks at startup time is good, so putting a section in BSBM that would force one to implement this would be a service to most end users. Measuring and reporting such I/O performance would favor space efficiency in general. Space efficiency is generally a good thing, especially at larger scales, so we can put an optional section in the report for warm-up. This is also good for comparing HDDs and SSDs, and for testing read-ahead, which is still something a database is expected to do. Implementors have it easy; just speculatively read everything.</p> <p>Looking at the BSBM fictional use case, anybody running such a portal would do this from RAM only, so it makes sense to define the primary metric as running from warm cache, in practice 100% from memory.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1ecb2af0">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x19d05678">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1d542328">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x13947e08">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1a7f6b30">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1d67dd40">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1ebcee68">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1a855ba0">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1b081e70">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1d7a7940">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d7e2cd0">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e375338">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d199728">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e808818">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-04#1666
2011-03-04T20:28:28Z
2011-03-14T17:56:38.000002-04:00
<p>Below is a questionnaire I sent to the <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x17f62428">BSBM</a> participants in order to get tuning instructions for the runs we were planning. I have filled in the answers for <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1d48ed28">Virtuoso</a>, here. This can be a checklist for pretty much any <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1e11b228">RDF</a> database tuning.</p> <ol> <li> <p> <b>Threading - What settings should be used (e.g., for query parallelization, I/O parallelization [e.g., prefetch, flush of dirty], thread pools [e,.g. web server], any other thread related)? We will run with 8 and 32 cores, so if there are settings controlling number of read/write (R/W) locks or mutexes or such for serializing diverse things, these should be set accordingly to minimize contention.</b> </p> <p>The following three settings are all <a href="http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_Parameters" id="link-id0x1ed4fe10">in the <code>[Parameters]</code> section of the <code>virtuoso.ini</code> file</a>. </p> <ul> <li> <p> <b><code>AsyncQueueMaxThreads</code> </b> controls the size of a pool of extra threads that can be used for query parallelization. This should be set to either <b>1.5 * the number of cores</b> or <b>1.5 * the number of core threads</b>; see which works better.</p> </li> <li> <p> <b><code>ThreadsPerQuery</code> </b> is the maximum number of threads a single query will take. This should be set to either <b>the number of cores</b> or <b>the number of core threads</b>; see which works better. </p> </li> <li> <p> <b><code>IndexTreeMaps</code> </b> is the number of mutexes over which control for buffering an index tree is split. This can generally be left at default (<b>256</b> in normal operation; valid settings are powers of 2 from 2 to 1024), but setting to <b>64, 128, or 512</b> may be beneficial.</p> <p>A low number will lead to frequent contention; upwards of 64 will have little contention. We have sometimes seen a multiuser workload go 10% faster when setting this to 64 (down from 256), which seems counter-intuitive. This may be a <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x1e12b618">cache</a> artifact.</p> </li> </ul> <p></p> <p> <a href="http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_HTTPServer" id="link-id0x1f8960a0">In the <code>[HTTPServer]</code> section of the <code>virtuoso.ini</code> file</a>, the <b><code>ServerThreads</code></b> setting is the number of web server threads, i.e., the maximum number of concurrent <a class="auto-href" href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id0x17e4d690">SPARQL protocol</a> requests. Having a value larger than the number of concurrent clients is OK; for large numbers of concurrent clients a lower value may be better, which will result in requests waiting for a thread to be available.</p> <p>Note â The <code>[HTTPServer] ServerThreads</code> are taken from the total pool made available by the <code>[Parameters] ServerThreads</code>. Thus, the <code>[Parameters] ServerThreads</code> should always be at least as large as (and is best set greater than) the <code>[HTTPServer] ServerThreads</code>, and if using the closed-source Commercial Version, <code>[Parameters] ServerThreads</code> cannot exceed the licensed thread count. </p> </li> <li> <p> <b>File layout - Are there settings for striping over multiple devices? Settings for other file access parallelism? Settings for SSDs (e.g., SSD based cache of hot set of larger db files on disk)? The target config is for 4 independent disks and 4 independent SSDs. If you depend on RAID, are there settings for this? If you need RAID to be set up, please provide the settings/script for doing this with 4 SSDs on Linux (RH and Debian). This will be software RAID, as we find the hardware RAID to be much worse than an independent disk setup on the system in question.</b> </p> <p>It is best to stripe database files over all available disks, and to not use RAID. If RAID is desired, then stripe database files across many RAID sets. Use the <code>segment</code> declaration in the <code>virtuoso.ini</code> file. It is very important to give each independently seekable device its own I/O queue thread. See the documentation on the <a class="auto-href" href="http://www.tpc.org/" id="link-id0x1e9a6bc0">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/C%2B%2B" id="link-id0x1ebdf210">C</a> sample for examples. </p> <p> <a href="http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_Parameters" id="link-id0x1f893f48">in the <code>[Parameters]</code> section of the <code>virtuoso.ini</code> file</a>, set <code>FDsPerFile</code> to be <code> (the number of concurrent threads * 1.5) ÷ the number of distinct database files</code>.</p> <p>There are no SSD specific settings.</p> </li> <li> <p> <b>Loading - How many parallel streams work best? We are looking for non-transactional bulk load, with no inference materialization. For partitioned cluster settings, do we divide the load streams over server processes? </b> </p> <p>Use one stream per core (not per core thread). In the case of a cluster, divide load streams evenly across all processes. The total number of streams on a cluster can equal the total number of cores; adjust up or down depending on what is observed.</p> <p>Use the built-in bulk load facility, i.e., </p> <blockquote> <code>ld_dir ('<source-filename-or-directory>', '<file name pattern>', '<destination graph iri>');</code> </blockquote> <p>For example,</p> <blockquote> <code><a class="auto-href" href="http://dbpedia.org/resource/SQL" id="link-id0x1dc52c58">SQL</a>> ld_dir ('/path/to/files', '*.n3', '<a class="auto-href" href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x1e76bfc8">http</a>://<a class="auto-href" href="http://dbpedia.org/resource/DBpedia" id="link-id0x1e9a6ad8">dbpedia</a>.org');</code> </blockquote> <p>Then do a <code>rdf_loader_run ()</code> on enough connections. For example, you can use the shell command </p> <blockquote> <code>isql rdf_loader_run () &</code> </blockquote> <p>to start one in a background isql process. When starting background load commands from the shell, you can use the shell <code>wait</code> command to wait for completion. If starting from isql, use the <code>wait_for_children;</code> command (see <a href="http://docs.openlinksw.com/virtuoso/isql.html" id="link-id0x1ae0f230">isql documentation</a> for details). </p> <p>See the <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d635820">BSBM disclosure report</a> for an example load script.</p> </li> <li> <p> <b>What command should be used after non-transactional bulk load, to ensure a consistent persistent state on disk, like a log checkpoint or similar? Load and checkpoint will be timed separately, load being <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x1e6f1000">CPU</a>-bound and checkpoint being I/O-bound. No roll-forward log or similar is required; the load does not have to recover if it fails before the checkpoint.</b> </p> <p>Execute </p> <blockquote> <code> CHECKPOINT;</code> </blockquote> <p>through a SQL client, e.g., <code>isql</code>. This is not a <a class="auto-href" href="http://dbpedia.org/resource/SPARQL" id="link-id0x1c2401d8">SPARQL</a> statement and cannot be executed over the SPARQL protocol.</p> </li> <li> <p> <b>What settings should be used for trickle load of small triple sets into a pre-existing graph? This should be as transactional as supported; at least there should be a roll forward log, unlike the case for the bulk load.</b> </p> <p>No special settings are needed for load testing; defaults will produce transactional behavior with a roll forward log. Default transaction isolation is <b><code>REPEATABLE READ</code></b>, but this may be altered via SQL session settings or at Virtuoso server start-up through <a href="http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_Parameters" id="link-id0x1a791b80">the <code>[Parameters]</code> section of the <code>virtuoso.ini</code> file</a>, with</p> <blockquote> <b><code><a href="http://wikis.openlinksw.com/dataspace/owiki/wiki/VirtuosoWikiWeb/ChangeVirtuosoSDefaultTransactionIsolationLevel" id="link-id0x1e5536b8">DefaultIsolation</a> = 4</code> </b> </blockquote> <p> Transaction isolation cannot be set over the SPARQL protocol.</p> <p> NOTE: When testing full CRUD operations, other isolation settings may be preferable, due to <a class="auto-href" href="http://dbpedia.org/resource/ACID" id="link-id0x1ce6a310">ACID</a> considerations. See answer #12, below, and detailed discussion in part 8 of this series, <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1b7eb5f0">BSBM <i>Explore and Update</i></a>.</p> </li> <li> <p> <b>What settings control allocation of memory for database caching? We will be running mostly from memory, so we need to make sure that there is enough memory configured. </b> </p> <p> <a href="http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_Parameters" id="link-id0x1acd8fe8">In the <code>[Parameters]</code> section of the <code>virtuoso.ini</code> file</a>, <b><code>NumberOfBuffers</code></b> controls the amount of RAM used by Virtuoso to cache database files. One buffer caches an 8KB database page. In practice, count 10KB of memory per page. If "swappiness" on Linux is low (e.g., 2), two-thirds or more of physical memory can be used for database buffers. If swapping occurs, decrease the setting.</p> </li> <li> <p> <b>What command gives status on memory allocation (e.g., number of buffers, number of dirty buffers, etc.) so that we can verify that things are indeed in server memory and not, for example, being served from OS disk cache. If the cached format is different from the disk layout (e.g., decompression after disk read), is there a command for space statistics for database cache? </b> </p> <p>In an <code>isql</code> session, execute </p> <blockquote> <code>STATUS ( ? ? );</code> </blockquote> <p>The second result paragraph gives counts of total, used, and dirty buffers. If used buffers is steady and less than total, and if the disk read count on the line below does not increase, the system is running from memory. The cached format is the same as the disk based format.</p> </li> <li> <p> <b>What command gives <a class="auto-href" href="http://dbpedia.org/resource/Information" id="link-id0x1c185f28">information</a> on disk allocation for different things? We are looking for the total size of allocated database pages for quads (including table, indices, anything else associated with quads) and dictionaries for literals, IRI names, etc. If there is a text index on literals, what command gives space stats for this? We count used pages, excluding any preallocated unused pages or other gaps. There is one number for quads and another for the dictionaries or other such structures, optionally a third for text index.</b> </p> <p>Execute on an <code>isql</code> session: </p> <blockquote> <code><pre> CHECKPOINT; SELECT TOP 20 * FROM sys_index_space_stats ORDER BY iss_pages DESC; </pre> </code> </blockquote> <p>The <code>iss_pages</code> column is the total pages for each index, including blob pages. Pages are 8KB. Only used pages are reported, gaps and unused pages are not counted. The rows pertaining to <code>RDF_QUAD</code> are for quads; <code>RDF_IRI</code>, <code>RDF_PREFIX</code>, <code>RO_START</code>, <code>RDF_OBJ</code> are for dictionaries; <code>RDF_OBJ_RO_FLAGS_WORDS</code> and <code>VTLOG_DB_DBA_RDF_OBJ</code> are for text index. </p> </li> <li> <p> <b>If there is a choice between triples and quads, we will run with quads. How do we ascertain that the run is with quads? How do we find out the index scheme? Should be use an alternate index scheme? Most of the <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x1c573780">data</a> will be in a single big graph.</b> </p> <p>The default scheme uses quads. The default index layout is <code>PSOG</code>, <code>POGS</code>, <code>GS</code>, <code>SP</code>, <code>OP</code>. To see the current index scheme, use an <code>isql</code> session to execute</p> <blockquote> <code>STATISTICS DB.DBA.RDF_QUAD;</code> </blockquote> </li> <li> <p> <b>For partitioned cluster settings, are there partitioning-related settings to control even distribution of data between partitions? For example, is there a way to set partitioning by <code>S</code> or <code>O</code> depending on which is first in key order for each index? </b> </p> <p>The default partitioning settings are good, i.e., partitioning is on <code>O</code> or <code>S</code>, whichever is first in key order.</p> </li> <li> <p> <b>For partitioned clusters, are there settings to control message batching or similar? What are the statistics available for checking interconnect operation, e.g. message counts, latencies, total aggregate throughput of interconnect?</b> </p> <p> <a href="http://docs.openlinksw.com/virtuoso/clusteroperation.html#clusteroperationgeneralclusterinifields" id="link-id0x1ec6dff0">In the <code>[Cluster]</code> section of the <code>cluster.ini</code> file</a>, <b><code>ReqBatchSize</code></b> is the number of query states dispatched between cluster nodes per message round trip. This may be incremented from the default of <code>10000</code> to <code>50000</code> or so if this is seen to be useful. </p> <p>To change this on the fly, the following can be issued through an <code>isql</code> session:</p> <blockquote> <code>cl_exec ( ' __dbf_set (''cl_request_batch_size'', 50000) ' ); </code> </blockquote> <p>The commands below may be executed through an <code>isql</code> session to get a summary of CPU and message traffic for the whole cluster or process-by-process, respectively. The documentation <a href="http://docs.openlinksw.com/virtuoso/clusteroperation.html#clusteroperationadminstdispl" id="link-id0x1dfccec0">details the fields</a>. </p> <blockquote> <pre> <code>STATUS ('cluster') ;; whole cluster</code> <br /> <code>STATUS ('cluster_d') ;; process-by-process</code> </pre></blockquote> </li> <li> <p> <b>Other settings - Are there settings for limiting query planning, when appropriate? For example, the BSBM <i>Explore</i> mix has a large component of unnecessary query optimizer time, since the queries themselves access almost no data. Any other relevant settings?</b> </p> <ul> <li> <p>For BSBM, needless query <a class="auto-href" href="http://dbpedia.org/resource/Program_optimization" id="link-id0x1f0ffab8">optimization</a> should be capped at Virtuoso server start-up through the <code>[Parameters]</code> section of the <code>virtuoso.ini</code>, with</p> <blockquote> <b><code>StopCompilerWhenXOverRun = 1</code> </b> </blockquote> </li> <li> <p>When testing full CRUD operations (not simply CREATE, i.e., load, as discussed in #5, above), it is essential to make queries run with transaction isolation of <code>READ COMMITTED</code>, to remove most lock contention. Transaction isolation cannot be adjusted via SPARQL. This can be changed through SQL session settings, or at Virtuoso server start-up <a href="http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_Parameters" id="link-id0x1f3a43c8">through the <code>[Parameters]</code> section of the <code>virtuoso.ini</code> file</a>, with</p> <blockquote> <b><code><a href="http://wikis.openlinksw.com/dataspace/owiki/wiki/VirtuosoWikiWeb/ChangeVirtuosoSDefaultTransactionIsolationLevel" id="link-id0x1a5a51e0">DefaultIsolation</a> = 2</code> </b> </blockquote> </li> </ul> </li> </ol> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1d6e5428">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1c3ea770">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1efeca30">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1bda5158">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1ec74808">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1ea253a0">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1b02d528">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1ae81fc0">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x197515c0">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1a78db90">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d32ae10">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1e8fcc18">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ae95050">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1dbf3158">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore
http://www.openlinksw.com/blog/vdb/blog/?date=2011-03-02#1664
2011-03-02T23:23:16Z
2011-03-14T17:16:56-04:00
<p>In this post I will summarize the figures for <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1dcf58f8">BSBM</a> Load and <i>Explore</i> mixes at 100 Mt, 200 Mt, and 1000 Mt. (1 Mt = 1 Megatriple, or one million triples.) The measurements were made on a 72GB 2xXeon 5520 with 4 SSDs. The exact specifications and configurations are in the raw reports to follow.</p> <p>The load time in <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/V6/index.html" id="link-id0x1f3716d8">the recent Berlin report</a> was measured with <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/V6/index.html#resultsExplore" id="link-id0x1dd37f80">the wrong function</a>, and so far as we can tell, without multiple threads. The intermediate cut of <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1ddb0c90">Virtuoso</a> they tested also <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/V6/index.html#resultsExploreAndUpdate" id="link-id0x1e5fcf40"> had broken</a> <a class="auto-href" href="http://dbpedia.org/resource/SPARQL" id="link-id0x1e1d2b70">SPARQL</a>/<a class="auto-href" href="http://dbpedia.org/page/SPARUL" id="link-id0x1bfb00c0">Update</a> (also known as <a class="auto-href" href="http://dbpedia.org/page/SPARUL" id="link-id0x1e0d5fd8">SPARUL</a>) features. We have fixed this since, and give <a href="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/BenchmarksReduxSupportingFiles/results.zip" id="link-id0x1edf36b0">here the right numbers</a>.</p> <p>In the course of the discussion to follow, we talk about 3 different kinds of Virtuoso:</p> <ul> <li> <p> <i>6 Single</i> is the generally available single server configuration of Virtuoso. Whether this is open source or not does not make a difference.</p> </li> <li> <p> <i>6 Cluster</i> is the generally available commercial only cluster-capable Virtuoso.</p> </li> <li> <p> <i>7 Single</i> is the next generation single server Virtuoso, about to be released as a preview.</p> </li> </ul> <p>To understand the numbers, we must explain how these differ from each other in execution:</p> <ul> <li> <p> <i>6 Single</i> has one thread-per-query, and operates on one state of the query at a time.</p> </li> <li> <p> <i>6 Cluster</i> has one thread-per-query-per-process, and between processes it operates on batches of some tens-of-thousands of simultaneous query states. Within each node, these batches run through the execution pipeline one state at a time. Aggregation is distributed, and the query optimizer is generally smart about shipping colocated functions together.</p> </li> <li> <p> <i>7 Single</i> has multiple threads-per-query and in all situations operates on batches of 10,000 or more simultaneous query states. This means, for example, that index lookups get large numbers of parameters which then are sorted to get an ascending search pattern which benefits from locality, so the <code>n * log(n)</code> index access for the batch becomes more like linear if the <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x1ceca188">data</a> accessed has any locality. Furthermore, if there are many operands to an operator, these can be split on multiple threads. Also, scans of consecutive rows can be split before the scan on multiple threads, each doing a range of the scan. These features are called <i>vectored execution</i> and <i>query parallelization</i>. These techniques will also be applied to the cluster variant in due time.</p> </li> </ul> <p>The version 6 and 7 variants discussed here use the same physical storage layout with row-wise <a class="auto-href" href="http://dbpedia.org/resource/Data_compression" id="link-id0x1e521fa0">key compression</a>. Additionally, there exists a column-wise storage option in 7 that can fit 4x the number of quads in the same space. This column store option is not used here because it still has some problems with random order inserts.</p> <p> We will first consider loading. Below are the load times and rates for 7 at each scale.</p> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="4" align="center">7 Single</th> </tr> <tr> <th align="center">Scale</th> <th align="center">Rate <br /> (quads per second)</th> <th align="center">Load time <br /> (seconds)</th> <th align="center">Checkpoint time <br /> (seconds)</th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 261,366 </td> <td align="center"> 301 </td> <td align="center"> 82 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 216,000 </td> <td align="center"> 802 </td> <td align="center"> 123 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 130,378 </td> <td align="center"> 6641 </td> <td align="center"> 1012 </td> </tr> </table> <p>In each case the load was made on 8 concurrent streams, each reading a file from a pool of 80 files for the two smaller scales and 360 files for the larger scale.</p> <p>We also loaded the smallest data set with 6 Single using the same load script. </p> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="4" align="center">6 Single</th> </tr> <tr> <th align="center">Scale</th> <th align="center">Rate <br /> (quads per second)</th> <th align="center">Load time <br /> (seconds)</th> <th align="center">Checkpoint time <br /> (seconds)</th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 74,713 </td> <td align="center"> 1192 </td> <td align="center"> 145 </td> </tr> </table> <p> <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x1132ad18">CPU</a> time with 6 Single was 8047 seconds. We compare this to 4453 seconds of CPU for the same load on 7 Single. The CPU% during the run was on either side of 700% for 6 Single and 1300% for 7 Single. Note that high percentages involve core threads, not real cores. </p> <p>The difference is mostly attributable to vectoring and the introduction of a non-transactional insert. The 6 Single inserts transactionally but makes very frequent commits and writes no log, resulting in <i>de facto</i> non-transactional behavior but still there is a lock and commit cycle. Inserts in <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1c750368">RDF</a> load usually exhibit locality on all SPOG. Sorting by value gives ascending insert order and eliminates much of the lookup time for deciding where the next row will go. Contention on page read-write locks is less because the engine stays longer on a page, inserting multiple values in one go, instead of re-acquiring the read-write lock and possible transaction locks for each row.</p> <p>Furthermore, for single stream loading the non-transactional mode can serve one thread doing the parsing with many threads doing the inserting; hence, in practice the speed is bounded by the parsing speed. In multi-stream load this parallelization also happens but is less significant, as adding threads past the count of core threads is not useful. Writes are all in-place, and no delta-merge mechanism is involved. For transactional inserts, the uncommitted rows are not visible to read-committed readers, which do not block. Repeatable and serializable readers would block before an uncommitted insert.</p> <p>Now for the run (larger numbers indicate more queries executed, and are therefore better):</p> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="3" align="center"> 6 Single Throughput <br /> (QMpH, query mixes per hour) </th> </tr> <tr> <th align="center">Scale</th> <th align="center"> Single User </th> <th align="center"> 16 User </th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 7641 </td> <td align="center"> 29433 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 6017 </td> <td align="center"> 13335 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 1770 </td> <td align="center"> 2487 </td> </tr> </table> <br /> <table border="1" cellspacing="2" cellpadding="2" align="center" width="90%"> <tr> <th colspan="3" align="center"> 7 Single Throughput <br /> (QMpH, query mixes per hour) </th> </tr> <tr> <th align="center">Scale</th> <th align="center"> Single User </th> <th align="center"> 16 User </th> </tr> <tr> <th align="center">100 Mt</th> <td align="center"> 11742 </td> <td align="center"> 72278 </td> </tr> <tr> <th align="center">200 Mt</th> <td align="center"> 10225 </td> <td align="center"> 60951 </td> </tr> <tr> <th align="center">1000 Mt</th> <td align="center"> 6262 </td> <td align="center"> 24672 </td> </tr> </table> <p>The 100 Mt and 200 Mt runs are entirely in memory; the 1000 Mt run is mostly in memory, with about a 1.6 MB/s trickle from SSD in steady state. Accordingly, the 1000 Mt run is longer, with 2000 query mixes in the timed period, preceded by a warm-up of 2000 mixes with a different seed. For the memory-only scales, we run 500 mixes twice, and take the timing of the second run.</p> <p>Looking at single user speeds, 6 Single and 7 Single are closest at the small end and drift farther apart at the larger scales. This comes from the increased opportunity to parallelize Q5, since this works on more data and is relatively more important as the scale gets larger. The 100 Mt run of 7 Single has about 130% CPU, and the 1000 Mt run has about 270%. This also explains why adding clients gives a larger boost at the smaller scale. </p> <p>Now let us look at the relative effects of parallelizing and vectoring in 7 Single. We run 50 mixes of Single User <i>Explore</i>: 6132 QMpH with both parallelizing and vectoring on; 2805 QMpH with execution limited to a single thread. Then we set the vector size to 1, meaning that the query pipeline runs one row at a time. This gets us 1319 QMpH which is a bit worse than 6 Single. This is to be expected since there is some overhead to running vectored with single-element vectors. Q5 on 7 Single with vectoring and a single thread runs at 1.9 qps; with single-element vectors, at 0.8 qps. The 6 Single engine runs Q5 at 1.13 qps.</p> <p>The 100 Mt scale 7 Single gains the most from adding clients; the 1000 Mt 6 Single gains the least. The reason for the latter is covered in detail in <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1b9ed390">A Benchmarking Story</a>. We note that while vectoring is primarily geared to better single-thread speed and better <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x1ddc2f48">cache</a> hit rates, it delivers a huge multithreaded benefit by eliminating the mutex contention at the index tree top which stops 6 Single dead at 1000 Mt.</p> <p>In conclusion, we see that even with a workload of short queries and little opportunity for parallelism, we get substantial benefits from query parallelization and vectoring. When moving to more complex workloads, the benefits become more pronounced. For a single user complex query load, we can get 7x speed-up from parallelism (8 core), plus up to 3x from vectoring. These numbers do not take into account the benefits of the column store; those will be analyzed separately a bit later.</p> <p>The full run details will be supplied at the end of this <a class="auto-href" href="http://dbpedia.org/resource/Blog" id="link-id0x1e9f6960">blog</a> series.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1d0bb988">Benchmarks, Redux (part 1): On RDF Benchmarks </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x155fc700">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li>Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1d96e218">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1d7a5170">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1def9ca0">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1a7a7800">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1e9c6c68">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1e80c208">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1dafd290">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1f34f7f8">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1df24f50">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1f4b19c8">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1de90cf8">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ebefbe8">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 2): A Benchmarking Story
http://www.openlinksw.com/blog/vdb/blog/?date=2011-02-28#1661
2011-02-28T21:12:28Z
2011-03-14T17:16:40-04:00
<blockquote> <i>Caeterum censeo, benchmarks are for vendors...</i> </blockquote> <p>This is an edifying story about benchmarks and how databases work. I will show how one detail makes a 5+x difference, and how one really must understand how things work in order to make sense of benchmarks.</p> <p>We begin right after the publication of the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/V6/index.html" id="link-id0x1df843f8">recent Berlin report</a>. This report gives us OK performance for queries and very bad performance for loading. Trickle updates were not measurable. This comes as a consequence of testing intermediate software cuts and having incomplete instructions for operating them. I will cover the whole <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1d0b6ea0">BSBM</a> matter and the general benchmarking question in forthcoming posts; for now, let's talk about specifics.</p> <p>In the course of the discussion to follow, we talk about 3 different kinds of <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1e09ee88">Virtuoso</a>:</p> <ul> <li> <p> <i>6 Single</i> is the generally available single-instance-server configuration of Virtuoso. Whether this is open source or not does not make a difference.</p> </li> <li> <p> <i>6 Cluster</i> is the generally available, commercial-only, cluster-capable Virtuoso.</p> </li> <li> <p> <i>7 Single</i> is the next-generation single-instance-server Virtuoso, about to be released as a preview.</p> </li> </ul> <p>We began by running the various parts of BSBM at different scales with different Virtuoso variants. In so doing, we noticed that the BSBM <i>Explore</i> mix at one scale got better throughput as we added more clients, approximately as one would expect based on <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x1c1b4860">CPU</a> usage and number of cores, while at another scale this was not so.</p> <p>At the 1-billion-triple scale (1000 Mt; 1 Mt = 1 Megatriple, or one million triples) we saw CPU going from 200% with 1 client to 1400% with 16 clients but throughput increased by less than 20%. </p> <p>When we ran the same scale with our shared-nothing 6 Cluster, running 8 processes on the same box, throughput increased normally with the client count. We have not previously tried BSBM with 6 Cluster simply because there is little to gain and a lot to lose by distributing this workload. But here we got a multiuser throughput with 6 Cluster that is easily 3 times that of the single server, even with a cluster-unfriendly workload. </p> <p> See, sometimes scaling out even within a shared memory multiprocessor pays! Still, what we saw was rather anomalous.</p> <p>Over the years we have looked at performance any number of times and have a lot of built-in meters. For cases of high CPU with no throughput, the prime suspect is contention on critical sections. Quite right, when building with the mutex meter enabled, counting how many times each mutex is acquired and how many times this results in a wait, we found a mutex which gets acquired 600M times in the run, of which an insane 450M result in a wait. One can count a microsecond of real time each time a mutex wait results in the kernel switching tasks. The run took 500 s or so, of which 450 s of real time were attributable to the overhead of waiting for this one mutex.</p> <p>Waiting for a mutex is a real train wreck. We have tried spinning a few times before it, which the OS does anyhow, but this does not help. Using spin locks is good only if waits are extremely rare; with any frequency of waiting, even for very short waits, a mutex is still a lot better.</p> <p>Now, the mutex in question happens to serialize the buffer <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x1e542088">cache</a> for one specific page of <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x17c853d8">data</a>, one level down from the root of the index for <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1d64a658">RDF</a> PSOG. By the luck of the draw, the Ps falling on that page are commonly accessed Ps pertaining to product features. In order to get any product feature value, one must pass via this page. At the smaller scale, the different properties web their different ways based on the index root.</p> <p>One might here ask why the problem is one level down from the root and not in the root. The index root is already handled specially, so the read-write locks for buffers usually apply only for the first level down. One might also ask why have a mutex in the first place. Well, unless one is read-only and all in memory, there simply must be a way to say that a buffer must not get written to by one thread while another is reading it. Same for cache replacement. Some in-memory people fork a whole copy of the database process to do a large query and so can forget about serialization. But one must have long queries for this and have all in memory. One can make writes less frequent by keeping deltas, but this does not remove the need to merge the deltas at some point, which cannot happen without serializing this with the readers.</p> <p>Most of the time the offending mutex is acquired for getting a property of a product in Q5, the one that looks for products with similar values of a numeric property. We retrieve this property for a number of products in one go, due to vectoring. Vectoring is supposed to save us from constantly hitting the index tree top when getting the next match. So how come there is contention in the index tree top? As it happens, the vectored index lookup checks for locality only when all search conditions on key parts are equalities. Here however there is equality on P and S and a range on O; hence, the lookup starts from the index root every time.</p> <p>So I changed this. The effect was Q5 getting over twice as fast, with the single user throughput at 1000 Mt going from 2000 to 5200 QMpH (Query Mixes per Hour) and the 16-user throughput going from 3800 to over 21000 QMpH. The previously "good" throughput of 40K QMpH at 100 Mt went to 66K QMpH. </p> <p>Vectoring can make a real difference. The throughputs for the same workload on 6 Single, without vectoring, thus unavoidably hitting the page with the crazy contention, are 1770 QMpH single user and 2487 QMpH with 16 users. The 6 Cluster throughput, avoiding the contention but without the increased locality from vectoring and with the increased latency of going out-of-process for most of the data, was about 11.5K QMpH with 16 users. Each partition had a page getting the hits but since the partitioning was on S and S was about-evenly distributed, each partition got 1/8 of the load; thus waiting on the mutex did not become a killer issue. </p> <p>We see how detailed analysis of benchmarks can lead to almost an order of magnitude improvements in a short time. This analysis is however both difficult and tedious. It is not readily delegable; one needs real <a class="auto-href" href="http://dbpedia.org/resource/Knowledge" id="link-id0x1e7249d0">knowledge</a> of how things work and of how they ought to work in order to get anywhere with this. Experience tends to show that a competitive situation is needed in order to motivate one to go to the trouble. Unless something really sticks out in an obvious manner, one is most likely not going to look deep enough. Of course, this is seen in applications too but application <a class="auto-href" href="http://dbpedia.org/resource/Program_optimization" id="link-id0x1d429e80">optimization</a> tends to stop at a point where the application is usable. Also stored procedures and specially-tweaked queries will usually help. In most application scenarios, we are not simultaneously looking at multiple different implementations, except maybe at the start of development but then this falls under benchmarking and evaluation.</p> <p>So, the usefulness of benchmarks is again confirmed. There is likely great unexplored space for improvement as we move to more interesting and diverse scenarios.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1658" id="link-id0x1f619550">Benchmarks, Redux (part 1): On RDF Benchmarks</a> </li> <li>Benchmarks, Redux (part 2): A Benchmarking Story <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1caa7cd8">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1d8b7648">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1f2a6ba8">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x17b425f0">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x1a7f6b30">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1ee5ec98">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1b7c5af8">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1dad7588">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1c5520a0">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1eb19bf8">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1eb2c398">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1fb6a118">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1f160580">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Benchmarks, Redux (part 1): On RDF Benchmarks
http://www.openlinksw.com/blog/vdb/blog/?date=2011-02-28#1659
2011-02-28T20:20:22Z
2011-03-14T17:16:34.000002-04:00
<p>This post introduces a series on <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1e724ae0">RDF</a> benchmarking. In these posts I will cover the following:</p> <ul> <li> <p>Correct misleading <a class="auto-href" href="http://dbpedia.org/resource/Information" id="link-id0x1e325480">information</a> about us in the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/V6/index.html" id="link-id0x1ded41d0">recent Berlin report</a>: The load rate is off-the wall and the update mix is missing. We supply the right numbers and explain how to load things so that one gets decent performance.</p> </li> <li> <p>Discuss configuration options for <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1e0a2548">Virtuoso</a>.</p> </li> <li> <p>Tell a story about multithreading and its perils and how vectoring and scale-out can save us.</p> </li> <li> <p>Analyze the run time behavior of Virtuoso 6 Single, 6 Cluster, and 7 Single.</p> </li> <li> <p>Look at the benefits of SSDs (solid-state storage devices) over HDDs (hard disk devices; spinning platters), and I/O matters in general.</p> </li> <li> <p>Talk in general about modalities of benchmark running, and how to reconcile vendors doing what they know best with the air of legitimacy of a third party. Whether to do things a la <a class="auto-href" href="http://www.tpc.org/" id="link-id0x1e0ef4f0">TPC</a> or a la TREC? We will hopefully try a bit of both, at least so I have proposed to our partners in <a class="auto-href" href="http://lod2.eu/" id="link-id0x1e54d3d8">LOD2</a>, the EU FP7 that also funded the recent Berlin report.</p> </li> <li> <p>Outline the desiderata for an RDF benchmark that is not just an RDF-ized relational workload, the Social Intelligence Benchmark.</p> </li> <li> <p>Talk about <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1e730bc8">BSBM</a> in specific. What does it measure?</p> </li> <li> <p>Discuss some experiments with the BI use case of BSBM.</p> </li> <li> <p>Document how the results mentioned here were obtained and suggest practices for benchmark running and disclosure.</p> </li> </ul> <p>The background is that the LOD2 FP7 project is supposed to deliver a report about the state of the art and benchmark laboratory by March 1. The Berlin report is a part thereof. In the project proposal we talk about an ongoing benchmarking activity and about having up-to-date installations of the relevant RDF stores and <a class="auto-href" href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x1c1551e0">RDBMS</a>.</p> <p>Since this is taxpayer money for supposedly the common good, I see no reason why such a useful thing should be restricted to the project participants. On the other hand, running a display window of stuff for benchmarking, when in at least in some cases licenses prohibit unauthorized publishing of benchmark results might be seen to conflict with the spirit of the license if not its letter. We will see.</p> <p>For now, my take is that we want to run benchmarks of all interesting software, inviting the vendors to tell us how to do that if they will, and maybe even letting them perform those runs themselves. Then we promise not to disclose results without the vendor's permission. Access to the installations is limited to whoever operates the equipment. Configuration files and detailed hardware specs and such on the other hand will be made public. If a run is published, it will be with permission and in a format that includes full information for replicating the experiment.</p> <p>In the LOD2 proposal we also in so many words say that we will stretch the limits of the state of the art. This stretching is surely not limited to the project's own products but should also include the general benchmarking aspect. I will say with confidence that running single server benchmarks at a max 200 Mtriples of <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x11327f10">data</a> is not stretching anything.</p> <p>So to ameliorate this situation, I thought to run the same at 10x the scale on a couple of large boxes we have access to. 1 and 2 billion triples are still comfortably single server scales. Then we could go for example to Giovanni's cluster at <a class="auto-href" href="http://dbpedia.org/resource/Digital_Enterprise_Research_Institute" id="link-id0x1bfaffa0">DERI</a> and do 10 and 20 billion triples, this should fly reasonably on 8 or 16 nodes of the DERI gear. Or we might talk to SEALS who by now should have their own lab. Even Amazon <a class="auto-href" href="http://aws.amazon.com/ec2/" id="link-id0x1bfafef8">EC2</a> might be an option, although not the preferred one.</p> <p>So I asked everybody about config instructions, which produced a certain amount of dismay as I might be said to be biased and to be skirting the edges of conflict of interest. The inquiry was not altogether negative though since <a class="auto-href" href="http://dbpedia.org/resource/Ontotext" id="link-id0x1eccc1e0">Ontotext</a> and <a class="auto-href" href="http://freebase.com/guid/9202a8c04000641f8000000005c908d6" id="link-id0x1eccc208">Garlik</a> provided some information. We will look into these this and next week. We will not publish any information without asking first.</p> <p>In this series of posts I will only talk about <a class="auto-href" href="http://www.openlinksw.com/dataspace/organization/openlink#this" id="link-id0x1bfa4030">OpenLink Software</a>.</p> <h3> <i>Benchmarks, Redux</i> Series</h3> <ul> <li>Benchmarks, Redux (part 1): On RDF Benchmarks <i>(this post)</i> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1660" id="link-id0x1b668d10">Benchmarks, Redux (part 2): A Benchmarking Story</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1663" id="link-id0x1b3a0c08">Benchmarks, Redux (part 3): Virtuoso 7 vs 6 on BSBM Load and Explore</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1665" id="link-id0x1f9f1740">Benchmarks, Redux (part 4): Benchmark Tuning Questionnaire</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1667" id="link-id0x1ad929f8">Benchmarks, Redux (part 5): BSBM and I/O; HDDs and SSDs </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1669" id="link-id0x1db437c0">Benchmarks, Redux (part 6): BSBM and I/O, continued</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1671" id="link-id0x17138c38">Benchmarks, Redux (part 7): What Does BSBM Explore Measure?</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1673" id="link-id0x1c0e74f8">Benchmarks, Redux (part 8): BSBM Explore and Update </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1675" id="link-id0x1f297d10">Benchmarks, Redux (part 9): BSBM With Cluster</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1677" id="link-id0x1e4994b8">Benchmarks, Redux (part 10): LOD2 and the Benchmark Process</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1678" id="link-id0x1ebea6d0">Benchmarks, Redux (part 11): On the Substance of RDF Benchmarks</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1d5c86c0">Benchmarks, Redux (part 12): Our Own BSBM Results Report</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1efec0e0">Benchmarks, Redux (part 13): BSBM BI Modifications </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1a9941f8">Benchmarks, Redux (part 14): BSBM BI Mix </a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=" id="link-id0x1ea26de8">Benchmarks, Redux (part 15): BSBM Test Driver Enhancements </a> </li> </ul>
Virtuoso Directions for 2011
http://www.openlinksw.com/blog/vdb/blog/?date=2011-01-19#1650
2011-01-19T16:29:37Z
2011-01-20T12:54:42.000002-05:00
<p> <a href="http://www.openlinksw.com/weblog/oerling/?id=1603" id="link-id0x1d584720">At the start of 2010, I wrote</a> that 2010 would be the year when <a class="auto-href" href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x2007b778">RDF</a> became performance- and cost-competitive with relational technology for <a class="auto-href" href="http://dbpedia.org/resource/Data" id="link-id0x7f5bf68">data</a> warehousing and analytics. More specifically, RDF would shine where data was heterogenous and/or where there was a high frequency of <a class="auto-href" href="http://dbpedia.org/resource/Database_schema" id="link-id0x1ffa18b0">schema</a> change.</p> <p>I will now discuss what we have done towards this end in 2010 and how you will gain by this in 2011.</p> <p>At the start of 2010, we had internally demonstrated 4x space efficiency gains from column-wise compression and 3x loop join speed gains from vectored execution. To recap, <i>column-wise compression</i> means a column-wise storage layout where values of consecutive rows of a single column are consecutive in memory/disk and are compressed in a manner that benefits from the homogenous data type and possible sort order of the column. <i>Vectored execution</i> means passing large numbers of query variable bindings between query operators and possibly sorting inputs to joins for improving locality. Furthermore, always operating on large sets of values gives extra opportunities for parallelism, from instruction level to threads to scale out.</p> <p>So, during 2010, we integrated these technologies into <a class="auto-href" href="http://virtuoso.openlinksw.com" id="link-id0x1fdf3f90">Virtuoso</a>, for relational- and graph-based applications alike. Further, even if we say that RDF will be close to relational speed in Virtuoso, the point is moot if Virtuoso's relational speed is not up there with the best of analytics-oriented <a class="auto-href" href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x7bf0d40">RDBMS</a>. RDF performance does rest on the basis of general-purpose database performance; what is sauce for the goose is sauce for the gander. So we reimplemented <code><a class="auto-href" href="http://dbpedia.org/resource/Hash_join" id="link-id0x7815c60">HASH JOIN</a></code> and <code>GROUP BY</code>, and fine-tuned many of the tricks required by <a class="auto-href" href="http://www.tpc.org/" id="link-id0x213d6de8">TPC</a>-<a class="auto-href" href="http://dbpedia.org/resource/TPC-H" id="link-id0x1fd92690">H. TPC-H</a> is not the sole final destination, but it is a step on the way and a valuable checklist for what a database ought to do.</p> <p>At the Semdata workshop of <a class="auto-href" href="http://www.vldb2010.org/" id="link-id0x21178a50">VLDB 2010</a> <a href="http://www.openlinksw.com/weblog/oerling/?id=1632" id="link-id0x1de8fee8">we presented some results</a> of our column store applied to RDF and relational tasks. As noted in the paper, the implementation did demonstrate significant gains over the previous row-wise architecture but was not yet well optimized, so not ready to be compared with the best of the relational analytics world. A good part of the fall of 2010 went into optimizing the column store and completing functionality such as transaction support with columns.</p> <p>A lot of this work is not specifically RDF oriented, but all of this work is constantly informed by the specific requirements of RDF. For example, the general idea of vectored execution is to eliminate overheads and optimize <a class="auto-href" href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x7ae0d58">CPU</a> <a class="auto-href" href="http://dbpedia.org/resource/Cache" id="link-id0x7bb7150">cache</a> and other locality by doing single query operations on arrays of operands so that the whole batch runs more or less in CPU cache. Are the gains not lost if data is typed at run time, as in RDF? In fact, the cost of run-time-typing turns out to be small, since data in practice tends to be of homogenous type and with locality of reference in values. Virtuoso's column store implementation resembles in broad outline other column stores like <a class="auto-href" href="http://www.vertica.com/" id="link-id0x7f61080">Vertica</a> or <a class="auto-href" href="http://www.ingres.com/vectorwise/" id="link-id0x2154ce38">VectorWise</a>, the main difference being the built-in support for run-time heterogenous types.</p> <p>The <a class="auto-href" href="http://lod2.eu/" id="link-id0x755e668">LOD2</a> EU FP 7 project <a href="http://www.openlinksw.com/weblog/oerling/?id=1630" id="link-id0x1d8eaf28">started in September 2010</a>. In this project OpenLink and the celebrated heroes of the column store, <a class="auto-href" href="http://dbpedia.org/resource/National_Research_Institute_for_Mathematics_and_Computer_Science" id="link-id0x1feba470">CWI</a> of <a class="auto-href" href="http://dbpedia.org/resource/MonetDB" id="link-id0x223bbe70">MonetDB</a> and VectorWise fame, represent the database side.</p> <p>The first database task of LOD2 is making a survey of the state of the art and a round of benchmarking of RDF stores. The <a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x20f50c20">Berlin SPARQL Benchmark</a> (<a class="auto-href" href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x780c430">BSBM</a>) has accordingly evolved to include a business intelligence section and an update stream. Initial results from running these will become available in February/March, 2011. The specifics of this process merit another post; let it for now be said that benchmarking is making progress. In the end, it is our conviction that we need a situation where vendors may publish results as and when they are available and where there exists a well defined process for documenting and checking results.</p> <p>LOD2 will continue by <i>linking the universe,</i> as I half-facetiously put it on a presentation slide. This means alignment of anything from schema to instance identifiers, with and without supervision, and always with provenance, summarization, visualization, and so forth. In fact, putting it this way, this gets to sound like the old chimera of generating applications from data or allowing users to derive actionable intelligence from data of which they do not even know the structure. No, we are not that unrealistic. But we are moving toward more ad-hoc discovery and faster time to answer. And since we provide an infrastructure element under all this, we want to do away with the "RDF tax," by which we mean any significant extra cost of RDF compared to an alternate technology. To put it another way, you ought to pay for unpredictable heterogeneity or complex inference only when you actually use them, not as a fixed up-front overhead.</p> <p>So much for promises. When will you see something? It is safe to say that we cannot very well publish benchmarks of systems that are not generally available in some form. This places an initial technology preview cut of Virtuoso 7 with vectored execution somewhere in January or early February. The column store feature will be built in, but more than likely the row-wise compressed RDF format of Virtuoso 6 will still be the default. Version 6 and 7 databases will be interchangeable unless column-store structures are used.</p> <p>For now, our priority is to release the substantial gains that have already been accomplished.</p> <p>After an initial preview cut, we will return to the agenda of making sure Virtuoso is up there with the best in relational analytics, and that the equivalent workload with an RDF data model runs as close as possible to relational performance. As a first step this means taking TPC-H as is, and then converting the data and queries to the trivially equivalent RDF and <a class="auto-href" href="http://dbpedia.org/resource/SPARQL" id="link-id0x25716618">SPARQL</a> and seeing how it goes. In <a href="http://www.openlinksw.com/weblog/oerling/?id=1627" id="link-id0x1af60d40">the September paper</a> we dabbled a little with the data at a small scale but now we must run the full set of queries at 100GB and 300GB scales, which come to about 14 billion and 42 billion triples, respectively. A well done analysis of the issues encountered, covering similarities and dissimilarities of the implementation of the workload as <a class="auto-href" href="http://dbpedia.org/resource/SQL" id="link-id0x223b0a88">SQL</a> and SPARQL, should make a good VLDB paper.</p> <p>Database performance is an entirely open-ended quest and the bag of potentially applicable tricks is as good as infinite. Having said this, it seems that the scales comfortably reached in the TPC benchmarks are more than adequate for pretty much anything one is likely to encounter in real world applications involving comparable workloads. Businesses getting over 6 million new order transactions per minute (the high score of TPC-<a class="auto-href" href="http://dbpedia.org/resource/C%2B%2B" id="link-id0x1f72a180">C</a>) or analyzing a warehouse of 60 billion orders shipped to 6 billion customers over 7 years (10000GB or 10TB TPC-H) are not very common if they exist at all.</p> <p>The real world frontier has moved on. Scaling up the TPC workloads remains a generally useful exercise that continues to contribute to the state of the art but the applications requiring this advance are changing.</p> <p>Someone once said that for a new technology to become mainstream, it needs to solve a new class of problem. Yes, while it is a preparatory step to run TPC-H translated to SPARQL without dying of overheads, there is little point in doing this in production since SQL is anyway likely better and already known, proven, and deployed.</p> <p>The new class of problem, as LOD2 sees it, is the matter of web-wide cross-organizational data integration. Web-wide does not necessarily mean crawling the whole web, but does tend to mean running into significant heterogeneity of sources, both in terms of modeling and in terms of usage of more-or-less standard data models. Around this topic we hear two messages. The database people say that inference beyond what you can express in SQL views is theoretically nice but practically not needed; on the other side, we hear that the inference now being standardized in efforts like <a class="auto-href" href="http://dbpedia.org/resource/Rule_Interchange_Format" id="link-id0x22b3ad68">RIF</a> and <a class="auto-href" href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x22b3ad90">OWL</a> is not expressive enough for the real world. As one expert put it, <i>if enterprise data integration in the 1980s was between a few databases, today it is more like between 1000 databases,</i> which makes this matter similar to searching the web. How can one know in such a situation that the data being aggregated is in fact meaningfully aggregate-able?</p> <p>Add to this the prevalence of unstructured data in the world and the need to mine it for actionable intelligence. Think of combining data from CRM, worldwide media coverage of own and competitive brands, and in-house emails for assessing organizational response to events on the market.</p> <p>These are the actual use cases for which we need RDF at relational DW performance and scale. This is not limited to RDF and OWL profiles, since we fully believe that inference needs are more diverse. The reason why this is RDF and not SQL plus some extension of <a class="auto-href" href="http://dbpedia.org/resource/Datalog" id="link-id0x7ee5130">Datalog</a>, is the widespread adoption of RDF and <a class="auto-href" href="http://dbpedia.org/resource/Linked_Data" id="link-id0x2111f968">linked data</a> as a data publishing format, with all the schema-last and <a class="auto-href" href="http://dbpedia.org/resource/Open_world_assumption" id="link-id0x2111f990">open world</a> aspects that have been there from the start.</p> <p>Stay tuned for more news later this month!</p> <h3>Related</h3> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1603" id="link-id0x1de6b370">Linked Data and Virtuoso in 2010</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1510" id="link-id0x1b031180">Linked Data & The Year 2009</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1286" id="link-id0x1a582d10">Retrospective and Outlook for 2008</a> </li> </ul>
VLDB Semdata Workshop
http://www.openlinksw.com/blog/vdb/blog/?date=2010-09-21#1635
2010-09-21T21:14:14Z
2010-09-21T16:22:18-04:00
<p>I will begin by extending my thanks to the organizers, in specific <a href="http://members.deri.at/~retok" id="link-id0x236ebfd0">Reto Krummenacher</a> of <a href="http://www.sti-innsbruck.at/" id="link-id0x2371aca8">STI</a> and Atanas Kiryakov of <a href="http://dbpedia.org/resource/Ontotext" id="link-id0x22e24190">Ontotext</a> for inviting me to give a position paper at the workshop. Indeed, it is the builders of bridges, the pontifs (pontifex) amongst us who shall be remembered by history. The idea of organizing a semantic <a href="http://dbpedia.org/resource/Data" id="link-id0x23781ba8">data</a> management workshop at VLDB is a laudable attempt at rapprochement between two communities to the advantage of all concerned.</p> <p> <a href="http://semanticweb.org/id/Franz_Inc" id="link-id0x22e09fa8">Franz</a>, Ontotext, and OpenLink were the vendors present at the workshop. To summarize very briefly, <a href="http://data.semanticweb.org/person/jans-aasman" id="link-id0x2380e7c8">Jans Aasman</a> of Franz talked about the telco call center automation solution by Amdocs, where the <a href="http://semanticweb.org/id/AllegroGraph" id="link-id0x237c9408">AllegroGraph</a> <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x236f96a8">RDF</a> store is integrated. On the technical side, AllegroGraph has Javascript as a stored procedure language, which is certainly a good idea. Naso of Ontotext talked about the BBC FIFA World Cup site. The technical proposition was that materialization is good and data partitioning is not needed; a set of replicated read-only copies is good enough.</p> <p>I talked about making RDF cost competitive with relational for data integration and BI. The crux is space efficiency and column store techniques.</p> <p>One question that came up was that maybe RDF could approach relational in some things, but what about string literals being stored in a separate table? Or <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x22ff2c78">URI</a> strings being stored in a separate table?</p> <p>The answer is that if one accesses a lot of these literals the access will be local and fairly efficient. If one accesses just a few, it does not matter. For user-facing reports, there is no point in returning a million strings that the user will not read anyhow. But then it turned out that there in fact exist reports in bioinformatics where there are 100,000 strings. Now taking the worst abuse of <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x236e43f8">SPARQL</a>, a regexp over all literals in a property of a given class. With a column store this is a scan of the column; with RDF, a three table join. The join is about 10x slower than the column scan. Quite OK, considering that a full text index is the likely solution for such workloads anyway. Besides, a sensible relational <a href="http://dbpedia.org/resource/Database_schema" id="link-id0x22e31050">schema</a> will also not use strings for foreign keys, and will therefore incur a similar burden from fetching the strings before returning the result.</p> <p>Another question was about whether the attitude was one of confrontation between RDF and relational and whether it would not be better to join forces. Well, as said in my talk, sauce for the goose is sauce for the gander and generally speaking relational techniques apply equally to RDF. There are a few RDB tricks that have no RDF equivalent, like clustering a fact table on dimension values, e.g., sales ordered by country, manufacturer, month. But by and large, column-store techniques apply. The execution engine can be essentially identical, just needing a couple of extra data types and some run-time typing and in some cases producing nulls instead of errors. Query <a href="http://dbpedia.org/resource/Program_optimization" id="link-id0x237d76e0">optimization</a> is much the same, except that RDB stats are not applicable as such; one needs to sample the data in the cost model. All in all, these adaptations to a RDB are not so large, even though they do require changes to source code.</p> <p>Another question was about combining data models, e.g., relational (rows and columns), RDF (graph), <a href="http://dbpedia.org/resource/XML" id="link-id0x23845418">XML</a> (tree), and full text. Here I would say that it is a fault of our messaging that we do not constantly repeat the necessity of this combining, as we take it for granted. Most RDF stores have a full text index on literal values. OWLIM and a <a href="http://dbpedia.org/resource/National_Research_Institute_for_Mathematics_and_Computer_Science" id="link-id0x22feefa0">CWI</a> prototype even have it for URIs. XML is a valid data type for an RDF literal, even though this does not get used very much. So doing SPARQL to select the values, and then doing <a href="http://dbpedia.org/resource/XPath" id="link-id0x235b5890">XPath</a> and XSLT on the values, is entirely possible, at least in <a href="http://virtuoso.openlinksw.com" id="link-id0x237f6428">Virtuoso</a> which has an XPath/XSLT engine built in. Same for invoking SPARQL from an XSLT sheet. Colocating a native <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x238265a8">RDBMS</a> with local and federated <a href="http://dbpedia.org/resource/SQL" id="link-id0x236f7bc8">SQL</a> is what Virtuoso has always done. One can, for example, map tables in heterogenous remote RDBs into tables in Virtuoso, then map these into RDF, and run SPARQL queries that get translated into SQL against the original tables, thereby getting SPARQL access without any materialization. Alongside this, one can ETL relational data into RDF via the same declarative mapping.</p> <p>Further, there are RDF extensions for geospatial queries in Virtuoso and AllegroGraph, and soon also in others.</p> <p>With all this cross-model operation, RDF is definitely not a closed island. We'll have to repeat this more.</p> <p>Of the academic papers, the SpiderStore (<a href="http://dbis-informatik.uibk.ac.at/5-1-Publications.html" id="link-id0x19ecd3f0">paper</a> is not yet available at time of writing, but should be soon) and <a href="http://www.few.vu.nl/~jui200/webpie.html" id="link-id0x1d60a498">Webpie</a> that should be specially noted.</p> <p>Let us talk about SpiderStore first.</p> <h2>SpiderStore</h2> <p>The SpiderStore from the University of Innsbruck is a main-memory-only system that has a record for each distinct IRI. The IRI record has one array of pointers to all IRI records that are objects where the referencing record is the subject, and a similar array of pointers to all records where the referencing record is the object. Both sets of pointers are clustered based on the predicate labeling the edge.</p> <p>According to the authors (Robert Binna, Wolfgang Gassler, Eva Zangerle, Dominic Pacher, and Günther Specht), a distinct IRI is 5 pointers and each triple is 3 pointers. This would make about 4 pointers per triple, i.e., 32 bytes with 64-bit pointers.</p> <p>This is not particularly memory efficient, since one must count unused space after growing the lists, fragmentation, etc., which will make the space consumption closer to 40 bytes per triple, plus should one add a graph to the mix one would need another pointer per distinct predicate, adding another 1-4 bytes per triple. Supporting non-IRI types in the object position is not a problem, as long as all distinct values have a chunk of memory to them with a type <a href="http://dbpedia.org/resource/Tag" id="link-id0x236fe4d0">tag</a>.</p> <p>We get a few times better memory efficiency with column compressed quads, plus we are not limited to main memory.</p> <p>But SpiderStore has a point. Making the traversal of an edge in the graph into a pointer dereference is not such a bad deal, especially if the data set is not that big. Furthermore, compiling the queries into <a href="http://dbpedia.org/resource/C%2B%2B" id="link-id0x235a2228">C</a> procedures playing with the pointers alone would give performance to match or exceed any hard coded graph traversal library and would not be very difficult. Supporting multithreaded updates would spoil much of the gain but allowing single threaded updates and forking read-only copies for reading would be fine.</p> <p>SpiderStore as such is not attractive for what we intend to do, this being aggregating RDF quads in volumes far exceeding main memory and scaling to clusters. We note that SpiderStore hits problems with distributed memory, since SpiderStore executes depth first, which is manifestly impossible if significant latencies are involved. In other words, if there can be latency, one must amortize by having a lot of other possible work available. Running with long vectors of values is one way, as in <a href="http://dbpedia.org/resource/MonetDB" id="link-id0x236e14a0">MonetDB</a> or Virtuoso Cluster. The other way is to have a massively multithreaded platform which favors code with few instructions but little memory locality. SpiderStore could be a good fit for massive multithreading, specially if queries were compiled to C, dramatically cutting down on the count of instructions to execute.</p> <p>We too could adopt some ideas from SpiderStore. Namely, if running vectored, one just in passing, without extra overhead, generates an array of links to the next IRI, a bit like the array that SpiderStore has for each predicate for the incoming and outgoing edges of a given IRI. Of course, here these would be persistent IDs and not pointers, but a hash from one to the other takes almost no time. So, while SpiderStore alone may not be what we are after for data warehousing, Spiderizing parts of the working set would not be so bad. This is especially so since the Spiderizable data structure almost gets made as a by-product of query evaluation.</p> <p>If an algorithm made several passes over a relatively small subgraph of the whole database, Spiderizing it would accelerate things. The memory overhead could have a fixed cap so as not to ruin the working set if locality happened not to hold.</p> <p>Running a SpiderStore-like execution model on vectors instead of single values would likely do no harm and might even result in better <a href="http://dbpedia.org/resource/Cache" id="link-id0x237eb508">cache</a> behavior. The exception is in the event of completely unpredictable patterns of connections which may only be amortized by massive multithreading.</p> <h2>Webpie</h2> <p>Webpie from <a href="http://www.vu.nl/" id="link-id0x23811bf8">VU Amsterdam</a> and the <a href="http://www.larkc.eu/" id="link-id0x22ff8fe8">LarKC</a> EU FP 7 project is, as it were, the opposite of SpiderStore. This is a map-reduce-based RDFS and <a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x238482a0">OWL</a> Horst inference engine which is all about breadth-first passes over the data in a map-reduce framework with intermediate disk-based storage.</p> <p>Webpie is not however a database. After the inference result has been materialized, it must be loaded into a SPARQL engine in order to evaluate a query against the result.</p> <p>The execution plan of Webpie is made from the ontology whose consequences must be materialized. The steps are sorted and run until a fixed point is reached for each. This is similar to running SPARQL <code>INSERT ⦠SELECT</code> statements until no new inserts are produced. The only requirement is that the <code>INSERT</code> statement should report whether new inserts were actually made. This is easy to do. In this way, a comparison between map-reduce plus memory-based joining and a parallel RDF database could be made.</p> <p>We have suggested such an experiment to the LarKC people. We will see.</p>
LOD2 Kick Off
http://www.openlinksw.com/blog/vdb/blog/?date=2010-09-21#1633
2010-09-21T21:13:03Z
2010-09-21T16:22:12-04:00
<p>The <a href="http://lod2.eu/" id="link-id0x22e06810">LOD2</a> <a href="http://lod2.eu/BlogPost/9-press-release-lod2-project-launch.html" id="link-id0x18c0c770">kick off meeting</a> was held in Leipzig on Sept 6-8. I will here talk about OpenLink plans as concerns LOD2; hence this is not to be taken as representative of the whole project. I will first discuss the immediate and conclude with the long term.</p> <p>As concerns OpenLink specifically, we have two short term activities, namely publishing the initial LOD2 repository in December and publishing a set of RDB and <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x22f9ba70">RDF</a> benchmarks in February.</p> <p>The LOD2 repository is a fusion of the OpenLink <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x2378d288">LOD</a> <a href="http://lod.openlinksw.com/" id="link-id0x23908828">Cloud</a> <a href="http://dbpedia.org/resource/Cache" id="link-id0x2378e6c8">Cache</a> (which includes <a href="http://dbpedia.org/resource/Data" id="link-id0x237d7d20">data</a> from <a href="http://uriburner.com/" id="link-id0x237c9408">URIBurner</a> and <a href="http://www.pingthesemanticweb.com/" id="link-id0x235b03b0">PingTheSemanticWeb</a>) and <a href="http://sindice.com/" id="link-id0x22e24190">Sindice</a>, both hosted at <a href="http://dbpedia.org/resource/Digital_Enterprise_Research_Institute" id="link-id0x237b80f8">DERI</a>. The value-add compared to Sindice or the <a href="http://virtuoso.openlinksw.com" id="link-id0x237b63c0">Virtuoso</a>-based LOD Cloud Cache alone is the merger of the timeliness and ping-ping crawling of Sindice with the <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x237f7568">SPARQL</a> of Virtuoso.</p> <p>Further down the road, after we migrate the system to the Virtuoso column store, we will also see gains in performance, primarily due to much better working set, as data is many times more compact than with the present row-wise <a href="http://dbpedia.org/resource/Data_compression" id="link-id0x235b0c38">key compression</a>.</p> <p>Still further, but before next September, we will have dynamic repartitioning; the time of availability is set as this is part of the LOD2 project roadmap. The operational need for this is pushed back somewhat by the compression gains from column-wise storage.</p> <p>As for benchmarks, I just compiled <a href="http://www.openlinksw.com/weblogs/oerling/" id="link-id0x1c29e720">a draft of suggested extensions to the BSBM</a> (<a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x22e31050">Berlin SPARQL Benchmark</a>). I talked about this with <a href="http://nl.linkedin.com/in/peterboncz" id="link-id0x237c90b0">Peter Boncz</a> and <a href="http://data.semanticweb.org/person/christian-bizer" id="link-id0x23813340">Chris Bizer</a>, to the effect that some extensions of BSBM could be done but that the time was a bit short for making a RDF-specific benchmark. We do recall that BSBM is fully feasible with a relational <a href="http://dbpedia.org/resource/Database_schema" id="link-id0x236f7ef8">schema</a> and that RDF offers no fundamental edge for the workload.</p> <p>There was a graph benchmark talk at the <a href="http://www.tpc.org/" id="link-id0x236f8170">TPC</a> workshop at <a href="http://www.vldb2010.org/" id="link-id0x235c6b90">VLDB 2010</a>. There too, the authors were suggesting a social network use case for benchmarking anything from RDF stores to graph libraries. The presentation did not include any specification of test data, so it may be that some cooperation is possible there. The need for such a benchmark is well acknowledged. The final form of this is not yet set but LOD2 will in time publish results from such.</p> <p>We did informally talk about a process for publishing with our colleagues from <a href="http://semanticweb.org/id/Franz_Inc" id="link-id0x23781d28">Franz</a> and <a href="http://dbpedia.org/resource/Ontotext" id="link-id0x23782740">Ontotext</a> at VLDB 2010. The idea is that vendors tune their own systems and do the runs and that the others check on this, preferably all using the same hardware.</p> <p>Now, the LOD2 benchmarks will also include relational-to-RDF comparisons, for example TPC-<a href="http://dbpedia.org/resource/TPC-H" id="link-id0x235a3568">H</a> in <a href="http://dbpedia.org/resource/SQL" id="link-id0x22e67370">SQL</a> and SPARQL. The SQL will be Virtuoso, <a href="http://dbpedia.org/resource/MonetDB" id="link-id0x22e70db0">MonetDB</a>, and possibly <a href="http://www.ingres.com/vectorwise/" id="link-id0x2378f750">VectorWise</a> and others, depending on what legal restrictions apply at the time. This will give an RDF-to-SQL comparison of TPC-H at least on Virtuoso, later also on MonetDB, depending on the schedule for a MonetDB SPARQL front-end.</p> <p>In the immediate term, this of course focuses our efforts on productizing the Virtuoso column store extension and the optimizations that go with it.</p> <p>LOD2 is however about much more than database benchmarks. Over the longer term, we plan to apply suitable parts of the ground-breaking database research done at <a href="http://dbpedia.org/resource/National_Research_Institute_for_Mathematics_and_Computer_Science" id="link-id0x23911830">CWI</a> to RDF use cases.</p> <p>This involves anything from adaptive indexing, to reuse and caching of intermediate results, to adaptive execution. This is however more than just mapping column store concepts to RDF. New challenges are posed by running on clusters and dealing with more expressive queries than just SQL, in specific queries with Datalog-like rules and recursion.</p> <p>LOD2 is principally about integration and alignment, from the schema to the instance level. This involves complex batch processing, close to the data, on large volumes of data. Map-reduce is not the be-all-end-all of this. Of course, a parallel database like Virtuoso, <a href="http://dbpedia.org/resource/Greenplum" id="link-id0x22feb520">Greenplum</a>, or <a href="http://www.vertica.com/" id="link-id0x237f7428">Vertica</a> can do map-reduce style operations under control of the SQL engine. After all, the SQL engine needs to do map-reduce and a lot more to provide good throughput for parallel, distributed SQL. Something like the <a href="http://www.eecs.berkeley.edu/Research/Projects/Data/105733.html" id="link-id0x235c2e28">Berkeley Orders Of Magnitude</a> (<a href="http://www.eecs.berkeley.edu/Research/Projects/Data/105733.html" id="link-id0x2380e7c8">BOOM</a>) distributed Datalog implementation (Overlog, Deadalus, BLOOM) could be a parallel computation framework that would subsume any map-reduce-style functionality under a more elegant declarative framework while still leaving control of execution to the developer for the cases where this is needed.</p> <p>From our viewpoint, the project's gains include:</p> <ul> <li> <p>Significant narrowing of the RDB to RDF performance gap. RDF will be an option for large scale warehousing, cutting down on time to integration by providing greater schema flexibility.</p> </li> <li> <p>Ready to use toolbox for data integration, including schema alignment and resolution of coreference.</p> </li> <li> <p>Data discovery, summarization and visualization</p> </li> </ul> <p>Integrating this into a relatively unified stack of tools is possible, since these all cluster around the task of linking the universe with RDF and <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x236e14a0">linked data</a>. In this respect the integration of results may be stronger than often seen in European large scale integrating projects.</p> <p>The use cases fit the development profile well: </p> <ul> <li> <p> <a href="http://dbpedia.org/resource/Wolters_Kluwer" id="link-id0x23820568">Wolters Kluwer</a> will develop an application for integrating resources around law, from the actual laws to court cases to media coverage. The content is modeled in a fine grained legal ontology.</p> </li> <li> <p> <a href="http://dbpedia.org/resource/Exalead" id="link-id0x22e50ba0">Exalead</a> will implement the linked data enterprise, addressing enterprise search and any typical enterprise data integration plus generating added value from open sources.</p> </li> <li> <p>The Open <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x236fb248">Knowledge</a> Foundation will create a portal of all government published data for easy access by citizens.</p> </li> </ul> <p>In all these cases, the integration requirements of schema alignment, resolution of identity, <a href="http://dbpedia.org/resource/Information" id="link-id0x2381ebb0">information</a> extraction, and efficient storage and retrieval play a significant role. The end user interfaces will be task-specific but developer interfaces around integration tools and query formulation may be quite generic and suited for generic RDF application development.</p>
Perseus, Andromeda, and RDF
http://www.openlinksw.com/blog/vdb/blog/?date=2010-09-13#1629
2010-09-13T22:10:12Z
2010-09-13T17:35:04-04:00
<p>It has been several months since <a href="http://www.openlinksw.com/weblog/oerling/?id=1622" id="link-id0x1c86c3a0">my last</a> <a href="http://dbpedia.org/resource/Blog" id="link-id0x15fa418">blog</a> post. In this day and age of the attention economy, what gives me the insolence so to neglect my duty to mindshare?</p> <p>Well, Perseus wasn't blogging or checking his email either, when he went to fetch the Gorgon's head. As Joseph Campbell puts it, the hero breaks into a world separate from the ordinary in order to bring back a blessing which will revitalize the community.</p> <p>Thus, I deliberately withdrew from the public conversation, in faith that it would take care of itself and that I would still not be altogether forgotten. As it happens, I was confirmed in this when recently invited to submit a talk for the <a href="http://semdata.org/events/2010/vldb" id="link-id0x1c319338">Semdata workshop</a> at <a href="http://www.vldb2010.org" id="link-id0x1b334640">VLDB 2010</a>.</p> <p>Great deeds are not only personal accomplishments but also play a role in a broader context. The quest may appear remote and difficult to execute but its outcome can be quite tangible: Andromeda needed no elaborate sales pitch to convince her of the advantages of not being eaten by the sea serpent.</p> <p>Thus right after <a href="http://www.openlinksw.com/weblog/oerling/?id=1614" id="link-id0x15fd6968">the meeting in Sofia last March</a>, I followed the vertical treasure map into the realm of first principles. As Perseus received advice from Athena, so was I informed by the Platonic ideas of locality and concurrency.</p> <p>The great quests have an outer and inner aspect. Likewise here, bringing the ideas to physical reality gave me a great deal of material on cognitive function itself. For human and computer alike, it appears that the main reason why anything at all works is <a href="http://dbpedia.org/resource/Cache" id="link-id0x140f1dc0">cache</a>. Locality and parallelism again. Maybe I will say something more about memory, attention, interface, and paradigm some other time. On the other hand, such material is bound to be unpopular even if valid.</p> <p>By now, you may ask yourself what I am talking about.</p> <p>We remember that Andromeda's fix was due to her mother, Cassiopeia, having claimed greater beauty than the daughters of the sea-god Poseidon. To transpose the archetype into the present, it is like Tim B-L saying that OWLs (by the way sacred to Athena) are more semantic than Codd's brainchild. Yet the relational community sees <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1d241648">RDF</a> as something not quite serious. A matter of scale(s) â just think of the sea serpent.</p> <p>So, I am talking about what I alluded to in the <a href="http://www.openlinksw.com/weblog/oerling/?id=1603" id="link-id0x19182b88">2010 New Year's statement on this blog</a>: RDF as a viable alternative to relational for big <a href="http://dbpedia.org/resource/Data" id="link-id0x1b896350">data</a>. This means that RDF is no longer a specialty niche where, due to the hopeless task of bringing everything into a relational model, the fact of everything taking several times both the time and space is tolerated because there is no real alternative.</p> <p>The value proposition is that for any current RDF user, the present assets will go four times farther than before with the next release of <a href="http://virtuoso.openlinksw.com" id="link-id0x6a223b0">Virtuoso</a>. For a prospective RDF user, the cost of keeping an ETLed RDF integration warehouse is now in the same ballpark as the relational cost, except that <a href="http://dbpedia.org/resource/Database_schema" id="link-id0x15f8ed8">schema</a> is now flexible, and the time to integrate and answer is accordingly shorter. For users of analytics-oriented <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x8bc44d8">RDBMS</a>, the next Virtuoso is a full cluster-capable <a href="http://dbpedia.org/resource/SQL" id="link-id0x127faf40">SQL</a> column store. Its merits compared to others in this space will be published later with benchmarks like <a href="http://www.tpc.org/" id="link-id0x6af7ae0">TPC</a>-<a href="http://dbpedia.org/resource/TPC-H" id="link-id0x1d46f230">H</a>. As an extra bonus for such users, Virtuoso brings SQL federation and a growth path to RDF, should this become interesting.</p> <p>This is accomplished by introducing a new column-wise compressed-storage engine with corresponding changes to query execution. The general principles are explained in <a href="http://cs-www.cs.yale.edu/homes/dna/papers/abadiphd.pdf" id="link-id0x1d259a88">Daniel Abadi's famous Ph.D. thesis</a>. The compression is tuned by the data itself, without user intervention. Further, our implementation remains capable of run-time-typing, thus the column-store advantages to RDF are obtained without going to a task-specific schema. But since data types, even if determined at run-time, are still in practice repetitive, the advantages of running on homogenous vectors are not lost.</p> <p>When storing an RDF extraction of TPC-H data, we get a storage usage of 6.3 bytes per quad. If you do not care about queries where the predicate is unspecified, the storage requirement drops to 4.7 bytes per quad. Whether storing the data as RDF quads or as Vertica-style multicolumn projections, the working set is about the same. Since having enough of the data in memory is the <i>sine qua non</i> prerequisite of flexible querying, the point is made. QED.</p> <p>In Virtuoso also, relational remains a bit faster but a penalty of 1.3x or so for RDF is quite tolerable, considering that <i>a priori</i> schema is no longer needed.</p> <p>This means that we are coming into an age where the warehouse becomes an <i>ad hoc</i> asset, to be filled with RDF, without the need to develop an <i>a priori</i> universal schema for all data one may ever wish to integrate, now or in the future. The data can be stored as RDF and projected from there into any form that may be needed at any time, whether the target format is more RDF or a task-specific relational schema.</p> <p>Availability is planned for late 2010, first as a Virtuoso Open Source preview.</p>
VLDB Semdata Workshop - The New Frontier of Semdata
http://www.openlinksw.com/blog/vdb/blog/?date=2010-09-13#1628
2010-09-13T22:09:24Z
2010-09-21T10:52:19.000004-04:00
<p>This is a revised version of the talk I will be giving at the <a href="http://semdata.org/events/2010/vldb" id="link-id0x1d137fe0">Semdata workshop</a> at <a href="http://www.vldb2010.org/" id="link-id0x2533b280">VLDB 2010</a>.</p> <p> <a href="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtDirectionsChallengesSemdata" id="link-id0x1cff6678">The paper</a> shows how we store <a href="http://www.tpc.org/" id="link-id0x244a65c0">TPC</a>-<a href="http://dbpedia.org/resource/TPC-H" id="link-id0x25136af8">H</a> <a href="http://dbpedia.org/resource/Data" id="link-id0x259a6460">data</a> as <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x268767b0">RDF</a> with relational-level efficiency and how we query both RDF and relational versions in comparable time. We also compare row-wise and column-wise storage formats as implemented in <a href="http://virtuoso.openlinksw.com" id="link-id0x2596dbc8">Virtuoso</a>.</p> <p>A question that has come up a few times during the Semdata initiative is how semantic data will avoid the fate of other would-be database revolutions like OODBMS and deductive databases.</p> <p>The need and opportunity are driven by the explosion of data in quantity and diversity of structure. The competition consists of analytics <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x2681bb10">RDBMS</a>, point solutions done with map-reduce or the like, and lastly in some cases from key-value stores with relaxed <a href="http://dbpedia.org/resource/Database_schema" id="link-id0x2493ca50">schema</a> but limited querying.</p> <p>The benefits of RDF are the ever expanding volume of data published in it, reuse of vocabulary, and well-defined semantics. The downside is efficiency. This is not so much a matter of absolute scalability â you can run an RDF database on a cluster â but a question of relative cost as opposed to alternatives.</p> <p>The baseline is that for relational-style queries, one should get relational performance or close enough. We outline in the paper how RDF reduces to a run-time-typed relational column-store, and gets all the compression and locality advantages traditionally associated with such. After memory is no longer the differentiator, the rest is engineering. So much for the scalability barrier to adoption.</p> <p>I do not need to talk here about the benefits of <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x245f72e8">linked data</a> and more or less <i>ad hoc</i> integration <i>per se</i>. But again, to make these practical, there are logistics to resolve: How to keep data up to date? How to distribute it incrementally? How to monetize freshness? We propose some solutions for these, looking at diverse-RDF replication and RDB-to-RDF replication in Virtuoso.</p> <p>But to realize the ultimate promise of RDF/Linked Data/Semdata, however we call it, we must look farther into the landscape of what is being done with big data. Here we are no longer so much running against the RDBMS, but against map-reduce and key-value stores.</p> <p>Given the psychology of geekdom, the charm of map-reduce is understandable: One controls what is going on, can work in the usual languages, can run on big iron without being picked to pieces by the endless concurrency and timing and order-of-events issues one gets when programming a cluster. Tough for the best, and unworkable for the rest.</p> <p>The key-value store has some of the same appeal, as it is the DBMS laid bare, so to say, made understandable, without the again intractably-complex questions of fancy query planning and distributed <a href="http://dbpedia.org/resource/ACID" id="link-id0x25c9a008">ACID</a> transactions. The psychological rewards of the sense of control are there, never mind the complex query; one can always hard code a point solution for the business question, if really must â maybe even in map-reduce.</p> <p>Besides, for some things that go beyond <a href="http://dbpedia.org/resource/SQL" id="link-id0x25149078">SQL</a> (for example, with graph structures), there really isn't a good solution.</p> <p>Now, enter <a href="http://www.vertica.com/" id="link-id0x268ecb90">Vertica</a>, <a href="http://dbpedia.org/resource/Greenplum" id="link-id0x25954eb8">Greenplum</a>, <a href="http://www.ingres.com/vectorwise/" id="link-id0x28cac500">VectorWise</a> (a <a href="http://dbpedia.org/resource/MonetDB" id="link-id0x28c239f8">MonetDB</a> project derivative from <a href="http://dbpedia.org/resource/Ingres" id="link-id0x24a2f498">Ingres</a>) and Virtuoso, maybe others, who all propose some combination of SQL- and explicit map-reduce-style control structures. This is nice but better is possible.</p> <p>Here we find the next frontier of Semdata. Take <a href="http://dbpedia.org/resource/Joseph_M._Hellerstein" id="link-id0x257db7c0">Joe Hellerstein</a> et al's work on <a href="http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-90.html" id="link-id0x1c64ba98">declarative logic for the data centric data center</a>.</p> <p>We have heard it many times â when the data is big, the logic must go to it. We can take declarative, location-conscious rules, <i>Ã la</i> <a href="http://www.eecs.berkeley.edu/Research/Projects/Data/105733.html" id="link-id0x29affc18">BOOM</a> and BLOOM, and combine these with the declarative query, well-defined semantics, parallel-database capability of the leading RDF stores. Merge this with locality compression and throughput from the best analytics DBMS.</p> <p>Here we have a data infrastructure that subsumes map-reduce as a special case of arbitrary distributed-parallel control flow, can send the processing to the data, and has flexible queries and schema-last capability.</p> <p>Further, since RDF more or less reduces to relational columns, the techniques of caching and reuse and materialized joins and demand-driven indexing, <i>Ã la</i> MonetDB, are applicable with minimal if any adaptation.</p> <p>Such a hybrid database-fusion frontier is relevant because it addresses heterogenous, large-scale data, with operations that are not easy to reduce to SQL, still without loss of the advantages of SQL. Apply this to anything from enhancing the business intelligence process by faster integration, including integration with <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x268168c8">linked open data</a> to the map-reduce bulk processing of today. Do it with strong semantics and inference close to the data.</p> <p>In short, RDF stays relevant by tackling real issues, with scale second to none, and decisive advantages in time-to-integrate and expressive power.</p> <p>Last week I was at the <a href="http://lod2.eu/" id="link-id0x29e23be8">LOD2</a> <a href="http://lod2.eu/BlogPost/9-press-release-lod2-project-launch.html" id="link-id0x1aec1c10">kick off</a> and a <a href="http://www.larkc.eu/" id="link-id0x245f1168">LarKC</a> meeting. The capabilities envisioned in this and the following post mirror our commitments to the EU co-funded LOD2 project. This week is VLDB and the Semdata workshop. I will talk more about how these trends are taking shape within the Virtuoso product development roadmap in future posts.</p>
Transactional High Availability in Virtuoso Cluster Edition
http://www.openlinksw.com/blog/vdb/blog/?date=2010-04-14#1623
2010-04-14T22:21:52Z
2010-04-14T19:13:00-04:00
<h2>Introduction</h2> <p>This post discusses the technical specifics of how we accomplish smooth transactional operation in a database server cluster under different failure conditions. (<a href="http://www.openlinksw.com/weblog/oerling/?id=1621" id="link-id0x198e8e68">A higher-level short version</a> was posted last week.) The reader is expected to be familiar with the basics of <a href="http://dbpedia.org/resource/Distributed_transaction" id="link-id0x25088028">distributed transactions</a>.</p> <p>Someone on a cloud computing discussion list called <a href="http://dbpedia.org/resource/Two-phase_commit_protocol" id="link-id0x21addd50">two-phase commit</a> (<a href="http://dbpedia.org/resource/Two-phase_commit_protocol" id="link-id0x1eb6bc90">2PC</a>) the "anti-availability protocol." There is indeed a certain anti-<a href="http://dbpedia.org/resource/SQL" id="link-id0x28e1dbd0">SQL</a> and anti-2PC sentiment out there, with key-value stores and "eventual consistency" being talked about a lot. Indeed, if we are talking about wide-area replication over high-latency connections, then 2PC with synchronously-sharp transaction boundaries over all copies is not really workable.</p> <p>For multi-site operations, a level of <i>eventual</i> consistency is indeed quite unavoidable. Exactly what the requirements are depends on the application, so I will focus here on operations inside one site.</p> <p>The key-value store culture seems to focus on workloads where a record is relatively self-contained. The record can be quite long, with repeating fields, different selections of fields in consecutive records, and so forth. Such a record would typically be split over many tables of a relational <a href="http://dbpedia.org/resource/Database_schema" id="link-id0x216479f8">schema</a>. In the <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x672d740">RDF</a> world, such a record would be split even wider, with the <a href="http://dbpedia.org/resource/Information" id="link-id0x72c8ec0">information</a> needed to reconstitute the full record almost invariably split over many servers. This comes from the mapping between the text of URIs and their internal IDs being partitioned in one way, and the many indices on the RDF quads each in yet another way.</p> <p>So it comes to pass that in the <a href="http://dbpedia.org/resource/Data" id="link-id0x216c6280">data</a> models we are most interested in, the application-level <a href="http://dbpedia.org/resource/Entity" id="link-id0x224444c0">entity</a> (<i>e.g.,</i> a user account in a social network) is not a contiguous unit with a single global identifier. The social network user account, that the key-value store would consider a unit of replication mastering and eventual consistency, will be in RDF or SQL a set of maybe hundreds of tuples, each with more than one index, nearly invariably spanning multiple nodes of the database cluster.</p> <p>So, before we can talk about wide-area replication and eventual consistency with application-level semantics, we need a database that can run on a fair-sized cluster and have cast-iron consistency within its bounds. If such a cluster is to be large and is to operate continuously, it must have some form of redundancy to cover for hardware failures, software upgrades, reboots, etc., without interruption of service.</p> <p>This is the point of the design space we are tackling here.</p> <h2>Non Fault-Tolerant Operation</h2> <p>There are two basic modes of operation we cover: bulk load, and online transactions.</p> <p>In the case of bulk load, we start with a consistent image of the database; load data; and finish by making another consistent image. If there is a failure during load, we lose the whole load, and restart from the initial consistent image. This is quite simple and is not properly transactional. It is quicker for filling a warehouse but is not to be used for anything else. In the remainder, we will only talk about online transactions.</p> <p>When all cluster nodes are online, operation is relatively simple. Each entry of each index belongs to a partition that is determined by the values of one or more partitioning columns of said index. There are no tables separate from indices; the relational row is on the index leaf of its primary key. Secondary indices reference the row by including the primary key. Blobs are in the same partition as the row which contains the blob. Each partition is then stored on a "cluster node." In non fault-tolerant operations, each such cluster node is a single process with exclusive access to its own permanent storage, consisting of database files and logs; <i>i.e.,</i> each node is a single server instance. It does not matter if the storage is local or on a SAN, the cluster node is still the only one accessing it.</p> <p>When things are not fault tolerant, transactions work as follows:</p> <p>When there are updates, two-phase commit is used to guarantee a consistent result. Each transaction is coordinated by one cluster node, which issues the updates in parallel to all cluster nodes concerned. Sending two update messages instead of one does not significantly impact latency. The coordinator of each transaction is the primary authority for the transaction's outcome. If the coordinator of the transaction dies between the phases of the commit, the transaction branches stay in the prepared state until the coordinator is recovered and can be asked again about the outcome of the transaction. Likewise, if a non-coordinating cluster node with a transaction branch dies between the phases, it will do a roll-forward and ask the coordinator for the outcome of the transaction.</p> <p>If cluster nodes occasionally crash and then recover relatively quickly, without ever losing transaction logs or database files, this is resilient enough. Everything is symmetrical; there are no cluster nodes with special functions, except for one master node that has the added task of resolving distributed deadlocks.</p> <p>I suppose our anti-SQL person called 2PC "anti-availability" because in the above situation we have the following problems: if any one cluster node is offline, it is quite likely that no transaction can be committed. This is so unless the data is partitioned on a key with application semantics, and all data touched by a transaction usually stays within a single partition. Then operations could proceed on most of the data while one cluster node was recovering. But, especially with RDF, this is never the case, since keys are partitioned in ways that have nothing to do with application semantics. Further, if one uses XA or <a href="http://dbpedia.org/resource/Microsoft" id="link-id0x785bc50">Microsoft</a> DTC with the monitor on a single box, this box can become a bottleneck and/or a single point of failure. (Among other considerations, this is why <a href="http://virtuoso.openlinksw.com" id="link-id0x72a1ea8">Virtuoso</a> does not rely on any such monitor.) Further, if a cluster node dies never to be heard of again, leaving prepared but uncommitted transaction branches, the rest of the system has no way of telling what to do with them, again unless relying on a monitor that is itself liable to fail.</p> <p>If transactions have a real world counterpart, it is possible, at least in theory, to check the outcome against the real world state: One can ask a customer if an order was actually placed or a shipment delivered. But when a transaction has to do with internal identifiers of things, for example whether <b><code>mailto://plaidskirt@hotdate.com</code></b> has internal ID <b><code>0xacebabe</code></b>, such a check against external reality is not possible.</p> <h2>Fault-Tolerant Operation</h2> <p>In a fault tolerant setting, we introduce the following extra elements: Cluster nodes are comprised of "quorums" of mutually-mirroring server instances. Each such quorum holds a partition of the data. Such a quorum typically consists of two server instances, but may have three for extra safety. If all server instances in the quorum are offline, then the cluster node is offline, and the cluster is not fully operational. If at least one server instance in a quorum is online, then the cluster node is online, and the cluster is operational and can process new transactions.</p> <p>We designate one cluster node (<i>i.e.,</i> one quorum of 2 or 3 server instances) to act as a master node, and we set an order of precedence among its member instances. In addition to arbitrating distributed deadlocks, the master instance on duty will handle reports of server instance failures, and answer questions about any transactions left hanging in prepared state by a dead transaction coordinator. If the master on duty fails, the next master in line will either notice this itself in the line of normal business or get a complaint from another server instance about not being able to contact the previous master.</p> <p>There is no global heartbeat messaging <i>per se,</i> but since connections between server instances are reused long-term, a dropped connection will be noticed and the master on duty will be notified. If all masters are unavailable, that entire quorum (<i>i.e.,</i> the master node) is offline and thus (as with any entire node going offline) most operations will fail anyway, unless by chance they do not hit any data managed by that failed quorum.</p> <p>When it receives a notice of unavailability, the master instance on duty tries to contact the unavailable server instance and if it fails, it will notify all remaining instances that that server instance is removed from the cluster. The effect is that the remaining server instances will stop attempting to access the failed instance. Updates to the partitions managed by the failed server instance are no longer sent to it, which results in updates to this data succeeding, as they are made against the other server instances in that quorum. Updates to the data of the failed server instance <i>will</i> fail in the window of time between the actual failure and the removal, which is typically well under a second. The removal of a failed server instance is delegated to a central authority in order not to have everybody get in each other's way when trying to effect the removal.</p> <p>If the failed server instance left prepared uncommitted transactions behind, the server instances having such branches will in due order contact the transaction coordinator to ask what should be done. This is a normal procedure for dealing with possibly dropped commit or rollback messages. When they discover that the coordinator has been removed, the master on duty will be contacted instead. Each prepare message of a transaction lists all the server instances participating in the transaction; thus the master can check whether each has received the prepare. If all have the prepare and none has an abort, the transaction is committed. The dead coordinator may not know this or may indeed not have the transaction logged, since it sends the prepares before logging its own prepare. The recovery will handle this though. We note that of the remaining branches, there is at least one copy of the branch with the failed server instance, or else we would have a whole quorum failed. In cases where there are branches participating in an unresolved transaction where all the quorum members have failed, the system cannot decide the outcome, and will periodically retry until at least one member of the failed quorum becomes available.</p> <p>The most complex part of the protocol is the recovery of a failed server instance. The recovery starts with a normal roll forward from the local transaction log. After this, the server instance will contact the master on duty to ask for its state. Typically, the master will reply that the recovering server instance had been removed and is out of date. When this is established, the recovering server instance will contact a live member of its quorum and ask for sync. The failed server instance has an approximate timestamp of its last received transaction. It knows this from the roll forward, where time markers are interspersed now and then between transaction records. The live partner then sends its transaction log(s) covering the time from a few seconds before the last transaction of the failed partner up to the present. A few transactions may get rolled forward twice but this does no harm, since these records have absolute values and no deltas and the second insert of a key is simply ignored. When the sender of the log reaches its last committed log entry, it asks the recovering server instance to confirm successful replay of the log so far. Having the confirmation, the sender will abort all unprepared transactions affecting it and will not accept any new ones until the sync is completed. If new transactions were committed between sending the last of the log and killing the uncommitted new transactions, these too are shipped to the recovering server instance in their committed or prepared state. When these are also confirmed replayed, the recovering server instance is in exact sync up to the transaction. The sender then notifies the rest of the cluster that the sync is complete and that the recovered server instance will be included in any updates of its slice of the data. The time between freeze and re-enable of transactions is the time to replay what came in between the first sync and finishing the freeze. Typically nothing came in, so the time is in milliseconds. If an application got its transaction killed in this maneuver, it will be seen as a deadlock.</p> <p>If the recovering server instance received transactions in prepared state, it will ask about their outcome as a part of the periodic sweep through pending transactions. One of these transactions could have been one originally prepared by itself, where the prepares had gone out before it had time to log the transaction. Thus, this eventuality too is covered and has a consistent outcome. Failures can interrupt the recovery process. The recovering server instance will have logged as far as it got, and will pick up from this point onward. Real time clocks on the host nodes of the cluster will have to be in approximate sync, within a margin of a minute or so. This is not a problem in a closely connected network.</p> <p>For simultaneous failure of a entire quorum of server instances (<i>i.e.,</i> a set of mutually-mirroring partners; a cluster node), the rule is that the last one to fail must be the first to come back up. In order to have uninterrupted service across arbitrary double failures, one must store things in triplicate; statistically, however, most double failures will not hit cluster nodes of the same group.</p> <p>The protocol for recovery of failed server instances of the master quorum (<i>i.e.,</i> the master cluster node) is identical, except that a recovering master will have to ask the other master(s) which one is more up to date. If the recovering master has a log entry of having excluded all other masters in its quorum from the cluster, it can come back online without asking anybody. If there is no such entry, it must ask the other master(s). If all had failed at the exact same instant, none has an entry of the other(s) being excluded and all will know that they are in the same state since any update to one would also have been sent to the other(s).</p> <h2>Failure of Storage Media</h2> <p>When a server instance fails, its permanent storage may or may not survive. Especially with mirrored disks, storage most often survives a failure. However, the survival of the database does not depend on any single server instance retaining any permanent storage over failure. If storage is left in place, as in the case of an OS reboot or replacing a faulty memory chip, rejoining the cluster is done based on the existing copy of the database on the server instance. if there is no existing copy, a copy can be taken from any surviving member of the same quorum. This consists of the following steps: First, a log checkpoint is forced on the surviving instance. Normally log checkpoints are done at regular intervals, independently on each server instance. The log checkpoint writes a consistent state of the database to permanent storage. The disk pages forming this consistent image will not be written to until the next log checkpoint. Therefore copying the database file is safe and consistent as long as a log checkpoint does not take place between the start and end of copy. Thus checkpoints are disabled right after the initial checkpoint. The copy can take a relatively long time; consider 20s per gigabyte on a 1GbE network a good day. At the end of copy, checkpoints are re-enabled on the surviving cluster node. The recovering database starts without a log, sees the timestamp of the checkpoint in the database, and asks for transactions from just before this time up to present. The recovery then proceeds as outlined above.</p> <h2>Network Failures</h2> <p>The CAP theorem states that Consistency, Availability, and Partition-tolerance do not mix. "Partition" here means the split of a network.</p> <p>It is trivially true that if the network splits so that on both sides there is a copy of each partition of the data, both sides will think themselves the live copy left online after the other died, and each will thus continue to accumulate updates. Such an event is not very probable within one site where all machines are redundantly connected to two independent switches. Most servers have dual 1GbE on the motherboard, and both ports should be used for cluster interconnect for best performance, with each attached to an independent switch. Both switches would have to fail in such a way as to split their respective network for a single-site network split to happen. Of course, the likelihood of a network split in multi-site situations is higher.</p> <p>One way of guarding against network splits is to require that at least one partition of the data have all copies online. Additionally, the master on duty can request each cluster node or server instance it expects to be online to connect to every other node or instance, and to report which they could reach. If the reports differ, there is a network problem. This procedure can be performed using both interfaces or only the first or second interface of each server to determine if one of the switches selectively blocks some paths. These simple sanity checks protect against arbitrary network errors. Using TCP for inter-cluster-node communication in principle protects against random message loss, but the Virtuoso cluster protocols do not rely on this. Instead, there are protocols for retry of any transaction messages and for using keep-alive messages on any long-running functions sent across the cluster. Failure to get a keep-alive message within a certain period will abort a query even if the network connections look OK. </p> <h2>Backups, and Recovery from Loss of Entire Site</h2> <p>For a constantly-operating distributed system, it is hard to define what exactly constitutes a consistent snapshot. The checkpointed state on each cluster node is consistent as far as this cluster node is concerned (<i>i.e.,</i> it contains no uncommitted data), but the checkpointed states on all the cluster nodes are not from exactly the same moment in time. The complete state of a cluster is the checkpoint state of each cluster node plus the current transaction log of each. If the logs were shipped in real time to off-site storage, a consistent image could be reconstructed from them. Since such shipping cannot be synchronous due to latency considerations, some transactions could be received only in part in the event of a failure of the off-site link. Such partial transactions can however be detected at reconstruction time because each record contains the list of all participants of the transaction. If some piece is found missing, the whole can be discarded. In this way integrity is guaranteed but it is possible that a few milliseconds worth of transactions get lost. In these cases, the online client will almost certainly fail to get the final success message and will recheck the status after recovery.</p> <p>For business continuity purposes, a live feed of transactions can be constantly streamed off-site, for example to a cloud infrastructure provider. One low-cost virtual machine on the cloud will typically be enough for receiving the feed. In the event of long-term loss of the whole site, replacement servers can be procured on the cloud; thus, capital is not tied up in an aging inventory of spare servers. The cloud-based substitute can be maintained for the time it takes to rebuild an owned infrastructure, which is still at present more economical than a cloud-only solution.</p> <p>Switching a cluster from an owned site to the cloud could be accomplished in a few hours. The prerequisite of this is that there are reasonably recent snapshots of the database files, so that replay of logs does not take too long. The bulk of the time taken by such a switch would be in transferring the database snapshots from S3 or similar to the newly provisioned machines, formatting the newly provisioned virtual disks, etc.</p> <p>Rehearsing such a maneuver beforehand is quite necessary for predictable execution. We do not presently have a productized set of tools for such a switch, but can advise any interested parties on implementing and testing such a disaster recovery scheme.</p> <h2>Conclusions</h2> <p>In conclusion, we have shown how we can have strong transactional guarantees in a database cluster without single points of failure or performance penalties when compared with a non fault-tolerant cluster. Operator intervention is not required for anything short of hardware failure. Recovery procedures are simple, at most consisting of installing software and copying database files from a surviving cluster node. Unless permanent storage is lost in the failure, not even this is required. Real-time off-site log shipment can easily be added to these procedures to protect against site-wide failures.</p> <p>Future work may be directed toward concurrent operation of geographically-distributed data centers with eventual consistency. Such a setting would allow for migration between sites in the event of whole-site failures, and for reconciliation between inconsistent histories of different halves of a temporarily split network. Such schemes are likely to require application-level logic for reconciliation and cannot consist of an out-of-the-box DBMS alone. All techniques discussed here are application-agnostic and will work equally well for Graph Model (<i>e.g.,</i> RDF) and Relational Model (<i>e.g.,</i> SQL) workloads.</p> <h3> <a href="http://dbpedia.org/resource/Glossary" id="link-id0x24f4e378">Glossary</a> </h3> <ul> <li> <b>Virtuoso Cluster (VC)</b> -- a collection of Virtuoso Cluster Nodes on one or more machines, working in parallel as part of a Virtuoso Cluster.</li> <li> <b>Virtuoso Cluster Node (VCN)</b> -- a Virtuoso Server Instance (Non Fault-Tolerant Operations), or a Quorum of Server Instances (Fault Tolerant Operations), which is a member of a collection of Virtuoso Cluster Nodes working in parallel as part of a Virtuoso Cluster.</li> <li> <b>Virtuoso Host Cluster (VHC)</b> -- a collection of machines, each hosting one or more Virtuoso Server Instances, making up a Virtuoso Cluster.</li> <li> <b>Virtuoso Host Cluster Node (VHCN)</b> -- a machine hosting one or more Virtuoso Server Instances that are members of a Virtuoso Cluster.</li> <li> <b>Virtuoso Server Instance (VSI)</b> -- a single Virtuoso process with exclusive access to its own permanent storage, consisting of database files and logs. May comprise an entire Virtuoso Cluster Node (Non Fault-Tolerant Operations), or be one member of a quorum which comprises a Virtuoso Cluster Node (Fault Tolerant Operations).</li> </ul> <h3>Also see</h3> <ul> <li> <a href="http://www.gbcacm.org/sites/www.gbcacm.org/files/slides/SpecialRelativity[1]_0.pdf" id="link-id0x16cb22d8">Special Relativity and the Problem of Database Scalability (PDF)</a>, by James Starkey of <a href="http://www.nimbusdb.com/" id="link-id0x18f30d58">NimbusDB, Inc.</a> </li> </ul>
Fault Tolerance in Virtuoso Cluster Edition (Short Version)
http://www.openlinksw.com/blog/vdb/blog/?date=2010-04-07#1621
2010-04-07T16:40:02Z
2010-04-14T19:12:47.000003-04:00
<p>We have for some time had the option of storing <a href="http://dbpedia.org/resource/Data" id="link-id0x28eb2178">data</a> in a cluster in multiple copies, in the Commercial Edition of <a href="http://virtuoso.openlinksw.com" id="link-id0x25178ed0">Virtuoso</a>. (This feature is not in and is not planned to be added to the Open Source Edition.)</p> <p>Based on some feedback from the field, we decided to make this feature more user friendly. The gist of the matter is that failure and recovery processes have been automated so that neither application developer nor operating personnel needs any <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x21fea428">knowledge</a> of how things actually work.</p> <p>So I will here make a few high level statements about what we offer for fault tolerance. I will follow up with technical specifics in another post.</p> <p>Three types of individuals need to know about fault tolerance:</p> <ul> <li>Executives: What does it cost? Will it really eliminate downtime?</li> <li>System Administrators: Is it hard to configure? What do I do when I get an alert?</li> <li>Application Developers/Programmers: Will I need to write extra code? Can old applications get fault tolerance with no changes?</li> </ul> <p>I will explain the matter to each of these three groups:</p> <h2>Executives</h2> <p>The value gained is elimination of downtime. The cost is in purchasing twice (or thrice) the hardware and software licenses. In reality, the cost is less since you get the whole money's worth of read throughput and half the money's worth of write throughput. Since most applications are about reading, this is a good deal. You do not end up paying for unused capacity.</p> <p>Server instances are grouped in "quorums" of two or, for extra safety, three; as long as one member of each quorum is available, the system keeps running and nobody sees a difference, except maybe for slower response. This does not protect against widespread power outage or the building burning down; the scope is limited to hardware and software failures at one site.</p> <p>The most basic site-wide disaster recovery plan consists of constantly streaming updates off-site. Using an off-site backup plus update stream, one can reconstitute the failed data center on a cloud provider in a few hours. Details will vary; please <a href="http://www.openlinksw.com/contact/" id="link-id0x2bdb0db8">contact us</a> for specifics.</p> <p>Running multiple sites in parallel is also possible but specifics will depend on the application. Again, please contact us if you have a specific case in mind.</p> <h2> System Administrators</h2> <p>To configure, divide your server instances into quorums of 2 or 3, according to which will be mirrors of each other, with each quorum member on a different host from the others in its quorum. These things are declared in a configuration file. Table definitions do not have to be altered for fault tolerance. It is enough for tables and indices to specify partitioning. Use two switches, and two NICs per machine, and connect one of each server's network cables to each switch, to cover switch failures.</p> <p>When things break, as long as there is at least one server instance up from each quorum, things will continue to work. Reboots and the like are handled without operator intervention; if there is a broken host, then remove it and put a spare in its place. If the disks are OK, put the old disks in the replacement host and start. If the disks are gone, then copy the database files from the live copy. Finally start the replacement database, and the system will do the rest. The system is online in read-write mode during all this time, including during copying.</p> <p>Having mirrored disks in individual hosts is optional since data will anyhow be in two copies. Mirrored disks will shorten the vulnerability window of running a partition on a single server instance since this will for the most part eliminate the need to copy many (hundreds) of GB of database files when recovering a failed instance.</p> <h2> Application Developers/Programmers</h2> <p>An application can connect to any server instance in the cluster and have access to the same data, with full <a href="http://dbpedia.org/resource/ACID" id="link-id0x6451870">ACID</a> properties.</p> <p>There are two types of errors that can occur in any database application: The database server instance may be offline or otherwise unreachable; and a transaction may be aborted due to a deadlock.</p> <p>For the missing server instance, the application should try to reconnect. An <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0x28e859b8">ODBC</a>/<a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0x28e11940">JDBC</a> connect string can specify a list of alternate server instances; thus as long as the application is written to try to reconnect as best practices dictate, there is no new code needed.</p> <p>For the deadlock, the application is supposed to retry the transaction. Sometimes when a server instance drops out or rejoins a running cluster, some transactions will have to be retried. To the application, these conditions look like a deadlock. If the application handles deadlocks (<a href="http://dbpedia.org/resource/SQL" id="link-id0x2bda4e40">SQL</a> State 40001) as best practices dictate, there is no change needed.</p> <h2>Conclusion</h2> <p>In summary...</p> <ul> <li>Limited extra cost for fault tolerance; no equipment sitting idle.</li> <li>Easy operation: Replace servers when they fail; the cluster does the rest.</li> <li>No changes needed to most applications.</li> <li>No proprietary SQL APIs or special fault tolerance logic needed in applications.</li> <li>Fully transactional programming model.</li> </ul> <p>All the above applies to both the Graph Model (<a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x22606f10">RDF</a>) and Relational (SQL) sides of Virtuoso. These features will be in the commercial release of Virtuoso to be publicly available in the next 2-3 weeks. Please <a href="http://www.openlinksw.com/contact/" id="link-id0x24f35648">contact OpenLink Software</a> Sales for details of availability or for getting advance evaluation copies.</p> <h3> <a href="http://dbpedia.org/resource/Glossary" id="link-id0x6648890">Glossary</a> </h3> <ul> <li> <b>Virtuoso Cluster (VC)</b> -- a collection of Virtuoso Cluster Nodes on one or more machines, working in parallel as part of a Virtuoso Cluster.</li> <li> <b>Virtuoso Cluster Node (VCN)</b> -- a Virtuoso Server Instance (Non Fault-Tolerant Operations), or a Quorum of Server Instances (Fault Tolerant Operations), which is a member of a collection of Virtuoso Cluster Nodes working in parallel as part of a Virtuoso Cluster.</li> <li> <b>Virtuoso Host Cluster (VHC)</b> -- a collection of machines, each hosting one or more Virtuoso Server Instances, making up a Virtuoso Cluster.</li> <li> <b>Virtuoso Host Cluster Node (VHCN)</b> -- a machine hosting one or more Virtuoso Server Instances that are members of a Virtuoso Cluster.</li> <li> <b>Virtuoso Server Instance (VSI)</b> -- a single Virtuoso process with exclusive access to its own permanent storage, consisting of database files and logs. May comprise an entire Virtuoso Cluster Node (Non Fault-Tolerant Operations), or be one member of a quorum which comprises a Virtuoso Cluster Node (Fault Tolerant Operations).</li> </ul> <h3>Also see</h3> <ul> <li> <a href="http://www.gbcacm.org/sites/www.gbcacm.org/files/slides/SpecialRelativity[1]_0.pdf" id="link-id0x1320f1e8">Special Relativity and the Problem of Database Scalability (PDF)</a>, by James Starkey of <a href="http://www.nimbusdb.com/" id="link-id0x1320f2b0">NimbusDB, Inc.</a> </li> </ul>
"The Acquired, The Innate, and the Semantic" or "Teaching Sem Tech"
http://www.openlinksw.com/blog/vdb/blog/?date=2010-04-05#1619
2010-04-05T15:21:19Z
2010-05-05T13:49:57-04:00
<p>I was recently asked to write a section for a policy document touching the intersection of database and semantics, as a follow up to the meeting in Sofia I <a href="http://www.openlinksw.com/weblog/oerling/?id=1614" id="link-id0x19c4f938">blogged about earlier</a>. I will write about technology, but this same document also touches the matter of education and computer science curricula. Since the matter came up, I will share a few thoughts on the latter topic.</p> <p>I have over the years trained a few truly excellent engineers and managed a heterogeneous lot of people. These days, since what we are doing is in fact quite difficult and the world is not totally without competition, I find that I must stick to core competence, which is hardcore tech and leave management to those who have time for it.</p> <p>When younger, I thought that I could, through sheer personal charisma, transfer either technical skills, sound judgment, or drive and ambition to people I was working with. Well, to the extent I believed this, my own judgment was not sound. Transferring anything at all is difficult and chancy. I must here think of a fantasy novel where a wizard said that, "working such magic that makes things do what they already want to do is easy." There is a grain of truth in that.</p> <p>In order to build or manage organizations, we must work, as the wizard put it, <i>with</i> nature, not against it. There are also counter-examples, for example my wife's grandmother had decided to transform a regular willow into a weeping one by tying down the branches. Such "magic," needless to say, takes constant maintenance; else the spell breaks.</p> <p>To operate efficiently, either in business or education, we need to steer away from such endeavors. This is a valuable lesson, but now consider teaching this to somebody. Those who would most benefit from this wisdom are the least receptive to it. So again, we are reminded to stay away from the fantasy of being able to transfer some understanding we think to have and to have this take root. It will if it will and if it does not, it will take constant follow up, like the would-be weeping willow.</p> <p>Now, in more specific terms, what can we realistically expect to teach about computer science?</p> <p>Complexity of algorithms would be the first thing. Understanding the relative throughputs and latencies of the memory hierarchy (i.e., <a href="http://dbpedia.org/resource/Cache" id="link-id0x13fcc8b8">cache</a>, memory, local network, disk, wide area network) is the second. Understanding the difference of synchronous and asynchronous and the cost of synchronization (i.e., anything from waiting for a mutex to waiting for a network message) is the third.</p> <p>Understanding how a database works would be immensely helpful for almost any application development task but this is probably asking too much.</p> <p>Then there is the question of engineering. Where do we put interfaces and what should these interfaces expose? Well, they certainly should expose multiple instances of whatever it is they expose, since passing through an interface takes time.</p> <p>I tried once to tell the <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x72d7490">SPARQL</a> committee that parameterized queries and array parameters are a self-evident truism on the database side. This is an example of an interface that exposes multiple instances of what it exposes. But the committee decided not to standardize these. There is something in the "semanticist" mind that is irrationally antagonistic to what is self-evident for databasers. This is further an example of ignoring precept 2 above, the point about the throughputs and latencies in the memory hierarchy. Nature is a better and more patient teacher than I; the point will become clear of itself in due time, no worry.</p> <p>Interfaces seem to be overvalued in education. This is tricky because we should not teach that interfaces are bad either. Nature has islands of tightly intertwined processes, separated by fairly narrow interfaces. People are taught to think in block diagrams, so they probably project this also where it does not apply, thereby missing some connections and porosity of interfaces.</p> <p> <a href="http://www.larkc.eu/" id="link-id0x1c5591f0">LarKC</a> (EU FP7 Large <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x15fae798">Knowledge</a> Collider project) is an exercise in interfaces. The lessons so far are that coupling needs to be tight, and that the roles of the components are not always as neatly separable as the block diagram suggests.</p> <p>Recognizing the points where interfaces are naturally narrow is very difficult. Teaching this in a curriculum is likely impossible. This is not to say that the matter should not be mentioned and examples of over-"paradigmatism" given. The geek mind likes to latch on to a paradigm (e.g., object orientation), and then they try to put it everywhere. It is safe to say that taking block diagrams too naively or too seriously makes for poor performance and needless code. In some cases, block diagrams can serve as tactical disinformation; i.e., you give lip service to the values of structure, <a href="http://dbpedia.org/resource/Information" id="link-id0x6f03e90">information</a> hiding, and reuse, which one is not allowed to challenge, ever, and at the same time you do not disclose the competitive edge, which is pretty much always a breach of these same principles.</p> <p>I was once at a <a href="http://dbpedia.org/resource/Data" id="link-id0x1d524ce0">data</a> integration workshop in the US where some very qualified people talked about the process of science. They had this delightfully American metaphor for it:</p> <blockquote> <i>The edge is created in the "Wild West" â there are no standards or hard-and-fast rules, and paradigmatism for paradigmatism's sake is a laughing matter with the cowboys in the fringe where new ground is broken. Then there is the OK Corral, where the cowboys shoot it out to see who prevails. Then there is Dodge City, where the lawman already reigns, and compliance, standards, and paradigms are not to be trifled with, lest one get the tar-and-feather treatment and be "driven out o'Dodge."</i> </blockquote> <p>So, if reality is like this, what attitude should the curriculum have towards it? Do we make innovators or followers? Well, as said before, they are not made. Or if they are made, they are not at least made in the university but much before that. I never made any of either, in spite of trying, but did meet many of both kinds. The education system needs to recognize individual differences, even though this is against the trend of turning out a standardized product. Enforced mediocrity makes mediocrity. The world has an amazing tolerance for mediocrity, it is true. But the edge is not created with this, if edge is what we are after.</p> <p>But let us move to specifics of semantic technology. What are the core precepts, the equivalent of the complexity/memory/synchronization triangle of general purpose CS basics? Let us not forget that, especially in semantic technology, when we have complex operations, lots of data, and almost always multiple distributed data sources, forgetting the laws of physics carries an especially high penalty.</p> <ul> <li> <p> <b>Know when to ontologize, when to folksonomize.</b> The history of standards has examples of "stacks of Babel," sky-high and all-encompassing, which just result in non-communication and non-adoption. Lighter weight, community driven, <a href="http://dbpedia.org/resource/Tag" id="link-id0x1dbd9018">tag</a> folksonomy, VoCamp-style approaches can be better. But this is a judgment call, entirely contextual, having to do with the maturity of the domain of discourse, etc.</p> </li> <li> <p> <b>Answer only questions that are actually asked.</b> This precept is two-pronged. The literal interpretation is not to do inferential closure for its own sake, materializing all implied facts of the knowledge base.</p> <p>The broader interpretation is to take real-world problems. Expanding RDFS semantics with map-reduce and proving how many iterations this will take is a thing one can do but real-world problems will be more complex and less neat.</p> </li> <li> <p> <b>Deal with ambiguity.</b> Data on which semantic technologies will be applied will be dirty, with errors from machine processing of natural language to erroneous human annotations. The knowledge bases will not be contradiction free. Michael Witbrock of CYC said many good things about this in Sofia; he would have something to say about a curriculum, no doubt.</p> </li> </ul> <p>Here we see that semantic technology is a younger discipline than computer science. We can outline some desirable skills and directions to follow but the idea of core precepts is not as well formed.</p> <p>So we can approach the question from the angle of needed skills more than of precepts of science. What should the certified semantician be able to do?</p> <ul> <li> <p> <b>Data integration.</b> Given heterogenous relational schemas talking about the same entities, the semantician should find existing ontologies for the domain, possibly extend these, and then map the relational data to them. After the mapping is conceptually done, the semantician must know what combination of ETL and on-the-fly mapping fits the situation. This does mean that the semantician indeed must understand databases, which I above classified as an almost unreachable ideal. But there is no getting around this. Data is increasingly what makes the world go round. From this it follows that everybody must increasingly publish, consume, and refine, i.e., integrate. The anti-database attitude of the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x2038d520">semantic web</a> community simply has to go.</p> </li> <li> <p> <b>Design and implement workflows for content extraction,</b> e.g., <a href="http://dbpedia.org/resource/Natural_language_processing" id="link-id0x713cdc0">NLP</a> or information extraction from images. This also means familiarity with NLP, desirably to the point of being able to tune the extraction rule sets of various NLP frameworks.</p> </li> <li> <p> <b>Design SOA workflows.</b> The semantician should be able to extract and represent the semantics of business transactions and the data involved therein.</p> </li> <li> <p> <b>Lightweight knowledge engineering.</b> The experience of building expert systems from the early days of AI is not the best possible, but with semantics attached to data, some sort of rules seem about inevitable. The rule systems will merge into the DBMS in time. Some ability to work with these, short of making expert systems, will be desirable.</p> </li> <li> <p> <b>Understand information quality</b> in the sense of trust, provenance, errors in the information, etc. If the world is run based on data analytics, then one must know what the data in the warehouse means, what accidental and deliberate errors it contains, etc.</p> </li> </ul> <p>Of course, most of these tasks take place at some sort of organizational crossroads or interface. This means that the semantician must have some project management skills; must be capable of effectively communicating with different publics and simply getting the job done, always in the face of organizational inertia and often in the face of active resistance from people who view the semantician as some kind of intruder on their turf.</p> <p>Now, this is a tall order. The semantician will have to be reasonably versatile technically, reasonably clever, and a self-starter on top. The self-starter aspect is the hardest.</p> <p>The semanticists I have met are more of the scholar than the IT consultant profile. I say <i>semanticist</i> for the semantic web research people and <i>semantician</i> for the practitioner we are trying to define.</p> <p>We could start by taking people who already do data integration projects and educating them in some semantic technology. We are here talking about a different breed than the one that by nature gravitates to description logics and AI. Projecting semanticist interests or attributes on this public is a source of bias and error.</p> <p>If we talk about a university curriculum, the part that cannot be taught is the leadership and self-starter aspect, or whatever makes a good IT consultant. Thus the semantic technology studies must be profiled so as to attract people with this profile. As quoted before, the dream job for each era is a scarce skill that makes value from something that is plentiful in the environment. At this moment and for a few moments to come, this is the data geek, or maybe even semantician profile, if we take data geek past statistics and traditional business intelligence skills.</p> <p>The semantic tech community, especially the academic branch of it, needs to reinvent itself in order to rise to this occasion. The flavor of the dream job curriculum will be away from the theoretical computer science towards the hands-on of database, large systems performance, and the practicalities of getting data intensive projects delivered.</p> <p> <b>Related</b> </p> <ul> <li> <a href="http://virtuoso.openlinksw.com/presentations/Linked_Data_Virtualization/Linked_Data_Virtualization.html" id="link-id0x199aca78">Linked Data Driven Data Virtualization for Web-scale Integration (presentation)</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1603" id="link-id0x13297a70">Linked Data and Virtuoso in 2010</a> </li> <li> <a href="http://www.openlinksw.com/blog/~kidehen/?id=1595" id="link-id0x1a3d0bd0">Getting The Linked Data Value Pyramid Layers Right</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1572" id="link-id0x1802b170">Provenance and Reification in Virtuoso</a> </li> <li> <a href="http://www.openlinksw.com/blog/~kidehen/?id=1519" id="link-id0x19af4220">The Time for RDBMS Primacy Downgrade is Nigh!</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/?id=1375" id="link-id0x1a07a378">Aspects of RDF to RDF Mapping</a> </li> </ul>
Upcoming RDF Loader in Unclustered Virtuoso loads Uniprot at 279 Ktriples/s!
http://www.openlinksw.com/blog/vdb/blog/?date=2010-04-02#1617
2010-04-02T14:15:01Z
2010-04-02T12:59:15-04:00
<p>We recently heard that <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x21414c58">Oracle</a> 11G loaded <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x28281e50">RDF</a> faster than we did. Now, we never thought the speed of loading a database was as important as the speed of query results, but since this is the <b><i>sole</i></b> area where they have reportedly been tested as faster, we decided it was time loading was addressed. Indeed, without Oracle to challenge us on query performance, we would not be half as good as we are. So, spurred on by the Oracular influence, we did something about our RDF loading.</p> <p>Performance, I have said before, is a matter of locality and parallelism. So we applied both to the otherwise quite boring exercise of loading RDF. The recipe is this: Take a large set of triples; resolve the IRIs and literals into their IDs; then insert each index of the triple table on its own thread. All the lookups and inserts are first sorted in key order to get the locality. Running the indices in parallel gets the parallelism. Then run the parser on its own thread, fetching chunks of consecutive triples and queueing them for a pool of loader threads. Then run several parsers concurrently on different files so as to make sure there is work enough at all times. Do not make many more process threads than available <a href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x30f3b20">CPU</a> threads, since they would just get in each other's way.</p> <p>The whole process is non-transactional, starting from a checkpoint and ending with a checkpoint.</p> <p>The test system was a dual-Xeon 5520 with 72G RAM. The <a href="http://virtuoso.openlinksw.com" id="link-id0x3256138">Virtuoso</a> was a single server; no cluster capability was used.</p> <p>We loaded English <a href="http://dbpedia.org/resource/DBpedia" id="link-id0x33b3e58">Dbpedia</a>, 179M triples, in 15 minutes, for a rate of 198 Kt/s. Uniprot with 1.33 G triples loaded in 79 minutes, for 279 Kt/s.</p> <p>The source files were the Dbpedia 3.4 English files and the <a href="http://www.bio2rdf.org/" id="link-id0x28266c20">Bio2RDF</a> copy of Uniprot, both in Turtle syntax. The uniref, uniparc and uniprot files from the Bio2RDF set were sliced into smaller chunks so as to have more files to load in parallel; the taxonomy file was as such; and no other Bio2RDF files were loaded. Both experiments ran with 8 load streams, 1 per core. The CPU utilization was mostly between 1400% and 1500%, 14-15 of 16 CPU threads busy. Top load speed for a measurement window of 2 minutes was 383 Kt/s.</p> <p>The index scheme for RDF quads was the default Virtuoso 6 configuration of 5 indices â GS, SP, OP, PSOG, and POGS. (We call this "3+2" indexing, because there are 3 partial and 2 full indices, delivering massive performance benefits over most other index schemes.) IRIs and literals reside in their own tables, each indexed from string to ID and vice versa. A full-text index on literals was not used.</p> <p>Compared to previous performance, we have more than tripled our best single server multi-stream load speed, and multiplied our single stream load speed by a factor of 8. Some further gains may be reached by adjusting thread counts and matching vector sizes to CPU <a href="http://dbpedia.org/resource/Cache" id="link-id0x20403130">cache</a>.</p> <p>This will be available in a forthcoming release; this is not for download yet. Now that you know this, you may guess what we are doing with queries. More on this another time.</p>
SemData@Sofia Roundtable write-up
http://www.openlinksw.com/blog/vdb/blog/?date=2010-03-15#1615
2010-03-15T14:46:57Z
2010-03-22T12:34:40.000010-04:00
<p>There was last week an <a href="http://www.semdata.org/" id="link-id11a83cf98">invitation-based roundtable</a> about semantic <a href="http://dbpedia.org/resource/Data" id="link-id0x1d37f598">data</a> management in <a href="http://www.dbpedia.org/resource/Sofia" id="link-id0x1ba4a208">Sofia, Bulgaria</a>.</p> <p>Lots of smart people together. The meeting was hosted by <a href="http://dbpedia.org/resource/Ontotext" id="link-id0x1cfc83f8">Ontotext</a> and chaired by <a href="http://www.dbpedia.org/resource/Dieter_Fensel" id="link-id0x1dc6e0d0">Dieter Fensel</a>. On the database side we had Ontotext, <a href="http://www.systap.com/" id="link-id0x1cda77f0">SYSTAP</a> (<a href="http://www.systap.com/bigdata.htm" id="link-id0x1dba6a30">Bigdata</a>), <a href="http://dbpedia.org/resource/National_Research_Institute_for_Mathematics_and_Computer_Science" id="link-id0x1d8e1d88">CWI</a> (<a href="http://dbpedia.org/resource/MonetDB" id="link-id0x1d8cbcf0">MonetDB</a>), <a href="http://www.dbpedia.org/resource/Karlsruhe_Institute_of_Technology" id="link-id0x1e204cb0">Karlsruhe Institute of Technology</a> (YARS2/<a href="http://swse.deri.ie/" id="link-id0x1e653bf0">SWSE</a>). <a href="http://www.larkc.eu/" id="link-id0x1e6a4408">LarKC</a> was well represented, being our hosts, with STI, Ontotext, CYC, and <a href="http://www.vu.nl/" id="link-id0x1c8a6090">VU Amsterdam</a>. Notable absences were <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x1e5ab690">Oracle</a>, <a href="http://freebase.com/guid/9202a8c04000641f8000000005c908d6" id="link-id0x1f5e5ff0">Garlik</a>, <a href="http://semanticweb.org/id/Franz_Inc" id="link-id0x1d9c08f0">Franz</a>, and <a href="http://www.talis.com/" id="link-id0x1d338b30">Talis</a>.</p> <p>Now of semantic data management... What is the difference between a relational database and a semantic repository, a triple/quad store, a whatever-you-call-them?</p> <p>I had last fall a meeting at CWI with Martin Kersten, Peter Boncz and Lefteris Sidirourgos from CWI, and Frank van Harmelen and Spiros Kotoulas of VU Amsterdam, to start a dialogue between semanticists and databasers. Here we were with many more people trying to discover what the case might be. What are the differences?</p> <p>Michael <a href="http://dbpedia.org/resource/Michael_Stonebraker" id="link-id0x1da55730">Stonebraker</a> and Martin Kersten have basically said that what is sauce for the goose is sauce for the gander, and that there is no real difference between relational DB and <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1d828310">RDF</a> storage, except maybe for a little tuning in some data structures or parameters. Semantic repository implementors on the other hand say that when they tried putting triples inside an RDB it worked so poorly that they did everything from scratch. (It is a geekly penchant to do things from scratch, but then this is not always unjustified.)</p> <p> <a href="http://www.openlinksw.com/dataspace/organization/openlink#this" id="link-id0x1cf1e620">OpenLink Software</a> and <a href="http://virtuoso.openlinksw.com" id="link-id0x1cfbc1d8">Virtuoso</a> are in agreement with both sides, contradictory as this might sound. We took our <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x1e1f6a20">RDBMS</a> and added data types and structures and cost model alterations to an existing platform. Oracle did the same. MonetDB considers doing this and time will tell the extent of their RDF-oriented alterations. Right now the estimate is that this will be small and not in the kernel.</p> <p>I would say with confidence that without source code access to the RDB, RDF will not be particularly convenient or efficient to accommodate. With source access, we found that what serves RDB also serves RDF. For example, execution engine and data compression considerations are the same, with minimal tweaks for RDF's run time typing needs.</p> <p>So now we are founding a platform for continuing this discussion. There will be workshops and calls for papers and the beginnings of a research community.</p> <p>After the initial meeting at CWI, I tried to figure what the difference was between the databaser and semanticist minds. Really, the things are close but there is still a disconnect. Database is about big sets and semantics is about individuals, maybe. The databaser discovers that the operation on each member of the set is not always the same, and the semanticist discovers that the operation on each member of the set is often the same.</p> <p>So the semanticist says that big joins take time. The databaser tells the semanticist not to repeat what's been obvious for 40 years and for which there is anything from partitioned hashes to merges to various vectored execution models. Not to mention columns.</p> <p>Spiros of VU Amsterdam/LarKC says that map-reduce materializes inferential closure really fast. Lefteris of CWI says that while he is not a semantic person, he does not understand what the point of all this materializing is, nobody is asking the question, right? So why answer? I say that computing inferential closure is a semanticist tradition; this is just what they do. Atanas Kiryakov of Ontotext says that this is not just a tradition whose start and justification is in the forgotten mists of history, but actually a clear and present need; just look at all the joining you would need.</p> <p>Michael Witbrock of CYC says that it is not about forward or backward inference on toy rule sets, but that both will be needed and on massively bigger rule sets at that. Further, there can be machine learning to direct the inference, doing the meta-reasoning merged with the reasoning itself.</p> <p>I say that there is nothing wrong with materialization if it is guided by need, in the vein of memo-ization or cracking or recycling as is done in MonetDB. Do the work when it is needed, and do not do it again.</p> <p>Brian Thompson of Systap/Bigdata asks whether it is not a contradiction in terms to both want pluggability and merging inference into the data, like LarKC would be doing. I say that this is difficult but not impossible and that when you run joins in a cluster database, as you decide based on the data where the next join step will be, so it will be with inference. Right there, between join steps, integrated with whatever data partitioning logic you have, for partitioning you <i>will</i> have, data being bigger and bigger. And if you have reuse of intermediates and demand driven indexing <i>Ã la</i> MonetDB, this too integrates and applies to inference results.</p> <p>So then, LarKC and CYC, can you picture a pluggable inference interface at this level of granularity? So far, I have received some more detail as to the needs of inference and database integration, essentially validating our previous intuitions and plans.</p> <p>Aside talking of inference, we have the more immediate issue of creating an industry out of the semantic data management offerings of today.</p> <p>What do we need for this? We need close-to-parity with relational â doing your warehouse in RDF with the attendant agility thereof can't cost 10x more to deploy than the equivalent relational solution.</p> <p>We also want to tell the key-value, anti-<a href="http://dbpedia.org/resource/SQL" id="link-id0x172e8c80">SQL</a> people, who throw away transactions and queries, that there is a better way. And for this, we need to improve our gig just a little bit. Then you have the union of some level of <a href="http://dbpedia.org/resource/ACID" id="link-id0x1e0de2e8">ACID</a>, at least consistent read, availability, complex query, large scale.</p> <p>And to do this, we need a benchmark. It needs a differentiation of online queries and browsing and analytics, graph algorithms and such. We are getting there. We will soon propose a social web benchmark for RDF which has both online and analytical aspects, a data generator, a test driver, and so on, with a <a href="http://www.tpc.org/" id="link-id0x1e3cb130">TPC</a>-style set of rules. If there is agreement on this, we will all get a few times faster. At this point, RDF will be a lot more competitive with mainstream and we will cross another qualitative threshold. </p>
RDF Geography With Virtuoso
http://www.openlinksw.com/blog/vdb/blog/?date=2009-11-11#1588
2009-11-11T17:17:27Z
2010-02-01T09:14:29.000012-05:00
<p>We have just added a geometry <a href="http://dbpedia.org/resource/Data" id="link-id0x1c0e02b0">data</a> type and corresponding <a href="http://dbpedia.org/resource/R-tree" id="link-id0x1e093220">R</a>-tree index to <a href="http://virtuoso.openlinksw.com" id="link-id0x1ddccfe8">Virtuoso</a>. This follows the general scheme of <a href="http://dbpedia.org/resource/SQL" id="link-id0x1b88a580">SQL</a>/MM, as is implemented by <a href="http://dbpedia.org/resource/PostGIS" id="link-id0x1d271a90">PostGIS</a> and many others. We have all the engine-side stuff, including optimizer support for geometry cardinality sampling and good execution plans for combinations of spatial and other joins. We have however not yet implemented all the different geometry types and library function support for them, like shortest distance between two arbitrary shapes.</p> <p>The geometry support is for both SQL and <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1b8d4ca8">SPARQL</a>. On the SQL side, it works with the ISO/IEC 13249 SQL/MM API; with <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1ed69318">RDF</a>, a geometry can occur as the object of a quad. If the object is a typed-literal of the <code>virtrdf:Geometry</code> type, it gets indexed in a geometry index over all geometries in quads; no special declarations are needed. After this, SQL MM predicates and functions can be used with SPARQL, like this:</p> <blockquote> <pre><code> PREFIX geo: <<a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x1d2d0ae0">http</a>://www.w3.org/2003/01/geo/wgs84_pos#> SELECT ?class COUNT (*) WHERE { ?m geo:geometry ?geo . ?m a ?class . FILTER ( <bif:st_intersects> ( ?geo, <bif:st_point> (0, 52), 100 ) ) } GROUP BY ?class ORDER BY DESC 2 </code> </pre></blockquote> <p>This returns the counts of objects of each class occurring within 100 km of (0, 52), a point near London.</p> <p>For any data set with <a href="http://dbpedia.org/resource/World_Geodetic_System" id="link-id0x1ec00578">WGS 84</a> <code>geo:long</code> and <code>geo:lat</code> values, a simple SQL function makes a point geometry for each such coordinate pair and adds it as the <code>geo:geometry</code> property of the subject with the long/lat. This then enables fast spatial access to arbitrary location data in RDF.</p> <p>Right now, we hardly see any geometries other than points in RDF data, even though there are some efforts for vocabularies for more complex entities. As these get adopted we will support them.</p> <p>For scalability, we tried the implementation with <a href="http://www.openstreetmap.org/" id="link-id0x1c781e68">OpenStreetMap</a>'s 350 million or so points. The geometry implementation partitions well over a cluster, similarly to a full text index, i.e., every server has its slice of the geometries, partitioned by the geometry object's key, thus not by range of coordinates or such. Like this, the items are evenly spread even though the coordinate distribution is highly uneven.</p> <p>We can do spatial joins like â</p> <blockquote> <pre><code> SELECT ?s ( <sql:num_or_null> (?p) ) COUNT (*) WHERE { ?s <http://<a href="http://dbpedia.org/resource/DBpedia" id="link-id0x1f885868">dbpedia</a>.org/ontology/populationTotal> ?p . FILTER ( <sql:num_or_null> (?p) > 1000000 ) . ?s geo:geometry ?geo . FILTER ( <bif:st_intersects> ( ?pt, ?geo, 5 ) ) . ?xx geo:geometry ?pt } GROUP BY ?s ( <sql:num_or_null> (?p) ) ORDER BY DESC 3 LIMIT 20 </code> </pre></blockquote> <p>This takes the DBpedia subjects that have a population over 1 million and a geometry. We then count all the geometries within 5 km of the point location of the first geometry. With DBpedia (about 5 million points), <a href="http://www.geonames.org/" id="link-id0x1d4279b0">GeoNames</a> (7 million points), and OpenStreetMap (350 million points), we get the result:</p> <blockquote> <pre><code>http://dbpedia.org/resource/Munich 1356594 117280 http://dbpedia.org/resource/London 7355400 81486 http://dbpedia.org/resource/Davao_City 1363337 58640 http://dbpedia.org/resource/Belo_Horizonte 2412937 58640 http://dbpedia.org/resource/Chengde 3610000 58640 http://dbpedia.org/resource/Hamburg 1769117 51664 http://dbpedia.org/resource/San_Diego%2C_California 1266731 47685 http://dbpedia.org/resource/Bursa 1562828 47685 http://dbpedia.org/resource/Port-au-Prince 1082800 47685 http://dbpedia.org/resource/Oakland_County%2C_Michigan 1194156 45636 http://dbpedia.org/resource/Sana%27a 1747627 40923 http://dbpedia.org/resource/Milan 1303437 40923 http://dbpedia.org/resource/Campinas 1059420 40923 http://dbpedia.org/resource/Hohhot 2580000 40923 http://dbpedia.org/resource/Brussels 1031215 40923 http://dbpedia.org/resource/Bogra_District 2988567 40923 http://dbpedia.org/resource/Cort%C3%A9s_Department 1202510 40923 http://dbpedia.org/resource/Berlin 3416300 35668 http://dbpedia.org/resource/New_York_City 8274527 30810 http://dbpedia.org/resource/Los_Angeles%2C_California 3849378 25614<br /> 20 Rows. -- 1733 msec.<br /> Cluster 8 nodes, 1 s. 358 m/s 1596 KB/s 664% <a href="http://dbpedia.org/resource/Central_processing_unit" id="link-id0x1e6403b0">cpu</a> 2% read 16% clw threads 1r 0w 0i buffers 1124351 0 d 0 w 0 pfs </code></pre></blockquote> <p>This takes 1.7 seconds on a Virtuoso Cluster configured with 8 processes on a single dual-Xeon 5520 box, running at about 664% CPU with warm <a href="http://dbpedia.org/resource/Cache" id="link-id0x1e81f610">cache</a>. Fair enough for a first crack, this can obviously be optimized further. Still, the geo part of the processing is already as good as instantaneous.</p> <p>We will shortly have the geography features installed on DBpedia and the other data sets we host. As these come online we will show more demo queries.</p> <p>For more about SQL/MM, you can look to a couple of PDFs:</p> <ul> <li> <a href="http://www.fer.hr/_download/repository/SQLMM_Spatial-_The_Standard_to_Manage_Spatial_Data_in_Relational_Database_Systems.pdf" id="link-id133775f0">SQL/MM Spatial: The Standard to Manage Spatial Data in Relational Database Systems</a> by Knut Stolze</li> <li> <a href="http://www.sigmod.org/record/issues/0112/standards.pdf" id="link-id1433c5e0">SQL Multimedia and Application Packages (SQL/MM)</a> by Jim Melton and Andrew Eisenberg</li> </ul>
European Commission and the Data Overflow
http://www.openlinksw.com/blog/vdb/blog/?date=2009-10-27#1586
2009-10-27T18:29:51Z
2009-10-27T14:57:31-04:00
<p>The European Commission recently circulated a questionnaire to selected experts on what could be done for the future of big <a href="http://dbpedia.org/resource/Data" id="link-id0x43bae00">data</a>.</p> <p>Since the <a href="http://cordis.europa.eu/fp7/ict/content-knowledge/consultation_en.html" id="link-id1191c0f8">questionnaire is public</a>, I am publishing my answers below.</p> <ol type="1" start="1"> <li> <p> <b>Data and data types</b> </p> <ol type="a" start="1"> <li> <p> <b>What volumes of data are we dealing with today? What is the growth rate? Where can we expect to be in 2015? </b> </p> <p>Private data warehouses of corporations have more than doubled yearly for the past years; hundreds of TB is not exceptional. This will continue. The real shift is in structured data being published in increasing quantities with a minimum level of integrate-ability through use of <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x5c7add0">RDF</a> and <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x5c7adb8">linked data</a> principles. There are rewards for use of standard vocabularies and identifiers through search engines recognizing such data. There is convergence around <a href="http://dbpedia.org/resource/DBpedia" id="link-id0x5c7ada0">DBpedia</a> identifiers for real-world entities, e.g., most things that would be in the news.</p> <p>This also means that internal data processes and silos may be enriched with this content. There is consequent pressure for accommodating more diversity of data, with more flexible <a href="http://dbpedia.org/resource/Database_schema" id="link-id0x7d87a88">schema</a>.</p> <p>Ultimately, all content presently stored in RDBs and presented in public accessible dynamic web pages will end up on the web of linked data. Examples are product catalogs, price lists, event schedules and the like.</p> <p>The volume of the well known linked data sets is around 10 billion statements. With the above mentioned trends, growth by two or three orders of magnitude by 2015 seems reasonable, This is so especially if explicit semantics are extracted from the document web and if there is some further progress in the precision/recall of such extraction.</p> <p>Relevant sections of this mass of data are a potential addition to any present or future analytics application.</p> <p>Since arbitrary analytics over the database which is the web cannot be economically provided by a centralized search engine, a cloud model may be used for on-demand selection of relevant data and mixing it with private data. This will drive database innovation for the next years even more than the continued classical warehouse growth.</p> <p>Science data is another driver of the data overflow. For example, faster gene sequencing, more accurate measurements in high energy physics, better imaging, and remote sensing will produce large volumes of data. This data has highly regular structure but labeling this data with source and lineage calls for a flexible, schema-last, self-describing model, such as RDF and linked data. Data and <a href="http://dbpedia.org/resource/Metadata" id="link-id0x7a3fb40">metadata</a> should travel together but may have different data models.</p> <p>By and large, the metadata of science data will be another stream to the web of linked data, at least to the degree it is publicly accessible. Restricted circles can and likely will implement similar ideas.</p> </li> <li> <p> <b>What types of data can we deal with intelligently due to their inherent structure (geospatial, temporal, social or <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x5a48058">knowledge</a> graphs, 3D, sensor streams...)?</b> </p> <p>All the above types should be supported inside one DBMS so as to allow efficient querying combining conditions on all these types of data, e.g., <i>photos of sunsets taken last summer in Ibiza, with over 20 megapixels, by people I know.</i> </p> <p>Note that the test for being a sunset is an operation on the image blob that should be taken to the data; the images cannot be economically transferred.</p> <p>Interleaving of all database functions and types becomes increasingly important.</p> </li> </ol> </li> <li> <p> <b>Industries, communities</b> </p> <ol type="a" start="1"> <li> <p> <b>Who is producing these data and why? Could they do it better? How?</b> </p> <p>Right now, projects such as <a href="http://www.bio2rdf.org/" id="link-id0x2a29de8">Bio2RDF</a>, <a href="http://neurocommons.org/page/Main_Page" id="link-id0x7ddaed0">Neurocommons</a>, and DBPedia produce this data. The processes are in place and are reasonable. Incremental improvement is to be expected. These processes, along with the <a href="http://www.w3.org/DesignIssues/LinkedData.html" id="link-id0xbab4dfd0">linked data meme</a> generally taking off, drive demand for better <a href="http://dbpedia.org/resource/Natural_language_processing" id="link-id0x51f4e0">NLP</a> (<a href="http://dbpedia.org/resource/Natural_language_processing" id="link-id0x51a1b48">Natural Language Processing</a>), e.g., <a href="http://dbpedia.org/resource/Entity" id="link-id0x956680">entity</a> and relationship extraction, especially extraction that can produce instance data in given ontologies (e.g., events) using common identifiers (e.g., DBPedia URIs).</p> <p>Mapping of RDBs to RDF is possible, and a W3C working group is developing standards for this. The required baseline level has been reached; the rest is a matter of automating deployment. Within the enterprise, there are advantages to be gained for <a href="http://dbpedia.org/resource/Information" id="link-id0x7da9e80">information</a> integration; e.g., all entities in the CRM space can be integrated with all email and support tickets through giving everything a <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x71673f8">URI</a>. Some of this information may even be published on an <a href="http://dbpedia.org/resource/Extranet" id="link-id0x9aa6e0">extranet</a> for self-service and web-service interfaces. This has been done at small scales and the rest is a matter of spreading adoption and lowering the entry barrier. Incremental progress will take place, eventually resulting in qualitatively better integration along the value chain when adoption is sufficiently widespread.</p> </li> <li> <p> <b>Who is consuming these data and why? Could they do it better? How?</b> </p> <p>Consumers are various. The greatest need is for tools that summarize complex data and allow getting a bird's eye view of what data is in the first instance available. Consuming the data is hindered by the user not even necessarily knowing what data there is. This is somewhat new, as traditionally the business analyst did know the schema of the warehouse and was proficient with <a href="http://dbpedia.org/resource/SQL" id="link-id0x7f7b148">SQL</a> report generators and statistics packages.</p> <p>Where Web 2.0 made the <i>citizen journalist</i>, the web of linked data will make the <i>citizen analyst</i>. For this to happen, with benefits for individuals, enterprises, and governments alike, more work in user interfaces, knowledge discovery, and query composition will be useful. We may envision a "meshup economy" where data is plentiful, but the unit of value and exchange is the smart report that crystallizes actionable value from this ocean.</p> </li> <li> <p> <b>What industrial sectors in Europe could become more competitive if they became much better at managing data?</b> </p> <p>Any sector could benefit. Early adopters are seen in the biomedical field and to an extent in media. </p> </li> <li> <p> <b>Is the regulation landscape imposing constraints (privacy, compliance ...) that don't have today good tool support?</b> </p> <p>The regulation landscape drives database demand through data retention requirements and the like.</p> <p>With data integration, especially with privacy-sensitive data (as in medicine), there are issues of whether one dares put otherwise-shareable information online. Regulation is needed to protect individuals, but integration should still be possible for science.</p> <p>For this, we see a need for progress in applying policy-based approaches (e.g., row level security) to relatively schema-last data such as RDF. This is possible but needs some more work. Also, creating on-the-fly-anonymizing views on data might help.</p> <p>More research is needed for reconciling the need for security with the advantages of broad-based <i>ad hoc</i> integration. Ideally, data should be intelligent, aware of its origins and classification and cautious of whom it interacts with, all of this supported under the covers so that the user could ask anything but the data might refuse to answer or might restrict answers according to the user's profile. This is a tall order and implementing something of the sort is an open question.</p> </li> <li> <p> <b>What are the main practical problem identified for individuals and organizations? Please give examples and tell us about the main obstacles and barriers.</b> </p> <p>We have come across the following:</p> <ul> <li>Knowing that the data exists in the first place.</li> <li>If the data is found, figuring out the provenance, units and precision of measurement, identifiers, and the like.</li> <li>Compatible subject matter but incompatible representation: For example, one has numbers on a map with different maps for different points in time; another has time series of instrument data with geo-location for the instrument. It is only to be expected that the time interval between measurements is not the same. So there is need for a lot of one-off programming to align data.</li> </ul> <p>Other problems have to do with sheer volume, i.e., transfer of data even in a local area network is too slow, let alone over a wide area network. Computation needs to go to the data, and databases need to support this.</p> </li> </ol> </li> <li> <p> <b>Services, software stacks, protocols, standards, benchmarks</b> </p> <ol type="a" start="1"> <li> <p> <b>What combinations of components are needed to deal with these problems?</b> </p> <p>Recent times have seen a proliferation of special purpose databases. Since the data needs of the future are about combining data with maximum agility and minimum performance hit, there is need to gather the currently-separate functionality into an integrated system with sufficient flexibility. We see some of this in integration of map-reduce and scale-out databases. The former antagonists have become partners. Vertica, <a href="http://dbpedia.org/resource/Greenplum" id="link-id0x7a94e70">Greenplum</a>, and OpenLink <a href="http://virtuoso.openlinksw.com" id="link-id0x2ab2868">Virtuoso</a> are example of DBMS featuring work in this direction.</p> <p>Interoperability and at least <i>de facto</i> standards in ways of doing this will emerge.</p> </li> <li> <p> <b>What data exchange and processing mechanisms will be needed to work across platforms and programming languages?</b> </p> <p> <a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x78a0458">HTTP</a>, <a href="http://dbpedia.org/resource/XML" id="link-id0x7ff2360">XML</a>, and RDF are in fact very verbose, yet these are the formats and models that have uptake. Thus, these will continue to be used even though one might think binary formats to be more efficient.</p> <p>There are of course science data set standards that are more compressed and these will continue, hopefully adding a practice of rich metadata in RDF.</p> <p>For internals of systems, MPI and TCP/IP with proprietary optimized wire formats will continue. Inter-system communication will likely continue to be HTTP, XML, and RDF as appropriate.</p> </li> <li> <p> <b>What data environments are today so wastefully messy that they would benefit from the development of standards?</b> </p> <p>RDF and <a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x5643d70">OWL</a> are not messy but they could use some more performance; we are working on this. <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x152ab18">SPARQL</a> is finally acquiring the capabilities of a serious query language, so things are slowly coming together.</p> <p>Community process for developing application domain specific vocabularies works quite well, even though one could argue it is <i>ad hoc</i> and not up to what a modeling purist might wish.</p> <p>Top-down imposition of standards has a mixed history, with long and expensive development and sometimes no or little uptake, consider some WS* standards for example.</p> </li> <li> <p> <b>What kind of performance is expected or required of these systems? Who will measure it reliably? How?</b> </p> <p>Relational databases have a history of substantial investment in <a href="http://dbpedia.org/resource/Program_optimization" id="link-id0xecc100">optimization</a> and some of them are very good for what they do, e.g., the newer generation of analytics databases.</p> <p>The very large schema-last, no-SQL, sometimes eventually consistent key-value stores have a somewhat shorter history but do fill a real need.</p> <p>These trends will merge: Extreme scale, schema-last, complex queries, even more complex inference, custom code for in-database machine learning and other bulk processing.</p> <p>We find RDF augmented with some binary types at this crossroads. This point of the design space will have to provide performance roughly on the level of today's best relational solution for workloads that fit the relational model. The added cost of schema-last and inference must come down. We are working on this. Research work such as carried out with <a href="http://dbpedia.org/resource/MonetDB" id="link-id0x7ae2890">MonetDB</a> gives clues as to how these aims can be reached.</p> <p>The separation of query language and inference is artificial. After the concepts are mature, these functions will merge and execute close to the data; there are clear evolutionary pressures in this direction.</p> <p>Benchmarks are key. Some gain can be had even from repurposing standard relational benchmarks like <a href="http://www.tpc.org/" id="link-id0x71eb528">TPC</a>-<a href="http://dbpedia.org/resource/TPC-H" id="link-id0x5e16a40">H</a>. But the TPC-H rules do not allow official reporting of such.</p> <p>Development of benchmarks for RDF, complex queries, and inference is needed. A bold challenge to the community, it should be rooted in real-life integration needs and involve high heterogeneity. A key-value store benchmark might also be conceived. A transaction benchmark like TPC-<a href="http://dbpedia.org/resource/C%2B%2B" id="link-id0x78562d0">C</a> might be the basis, maybe augmented with massive user-generated content like reviews and blogs.</p> <p>If benchmarks exist and are not too easy nor inaccessibly difficult nor too expensive to run â think of the high end TPC-C results â then TPC-style rules and processes would be quite adequate. The threshold to publish should be lowered: Everybody runs the TPC workloads internally but few publish.</p> <p>Some EC initiative for benchmarking could make sense, similar to the TREC initiative of the US government. Industry should be consulted for the specific content; possibly the answers to the present questionnaire can provide an approximate direction.</p> <p>Benchmarks should be run by software vendors on their own systems, tuned by themselves. But there should be a process of disclosure and auditing; the TPC rules give an example. Compliance should not be too expensive or time consuming. Some community development for automating these things would be a worthwhile target for EC funding.</p> </li> </ol> </li> <li> <p> <b>Usability and training</b> </p> <ol type="a" start="1"> <li> <p> <b>How difficult will it be for a developer of average competence to deploy components whose core is based on rather deep computer science? Do we all need to understand Monads and Continuations? What can be done to make it ever easier?</b> </p> <p>In the database world, huge advances in technology have taken place behind a relatively simple and stable interface: SQL. For the linked data <a href="http://dbpedia.org/resource/Giant_Global_Graph" id="link-id0x7761e50">web</a>, the same will take place behind SPARQL.</p> <p>Beyond these, for example, programming with MPI with good utilization of a cluster platform for an arbitrary algorithm, is quite difficult. The casual amateur is hereby warned.</p> <p>There is no single solution. For automatic parallelization, since explicit, programmatic parallelization of things with MPI for example is very unscalable in terms of required skill, we should favor declarative and/or functional approaches.</p> <p>Developing a debugger and explanation engine for rule-based and description-logics-based inference would be an idea.</p> <p>For procedural workloads, things like Erlang may be good in cases and are not overly difficult in principle, especially if there are good debugging facilities.</p> <p>For shipping functions in a cluster or cloud, the <a href="http://www.eecs.berkeley.edu/Research/Projects/Data/105733.html" id="link-id0x5494b0">BOOM</a> (<a href="http://www.eecs.berkeley.edu/Research/Projects/Data/105733.html" id="link-id0x7f1f148">Berkeley Orders Of Magnitude</a>) approach or logic programming with explicit specification of compute location seem promising, surely more flexible than map-reduce. The question is whether a <a href="http://dbpedia.org/resource/PHP" id="link-id0x5c758c8">PHP</a> developer can be made to do logic programming.</p> <p>This bridge will be crossed only with actual need and even then reluctantly. We may look at the Web 2.0 practice of sharding <a href="http://dbpedia.org/resource/MySQL" id="link-id0x432f868">MySQL</a>, inconvenient as this may be, for an example. There is inertia and thus re-architecting is a constant process that is generally in reaction to facts, <i>post hoc</i>, often a point solution. One could argue that planning ahead would be smarter but by and large the world does not work so.</p> <p>One part of the answer is an infinitely-scalable SQL database that expands and shrinks in the clouds, with the usual semantics, maybe optional eventual consistency and built-in map reduce. If such a thing is inexpensive enough and syntax-level-compatible with present installed base, many developers do not have to learn very much more.</p> <p>This is maybe good for the bread-and-butter IT, but European competitiveness should not rest on this. Therefore we wish to go for bold new application types for which the client-server database application is not the model. Data-centric languages like BOOM, if they can be made very efficient and have good debugging support, are attractive there. These do require more intellectual investment but that is not a problem since the less-inquisitive part of the developer community is served by the first part of the answer.</p> </li> <li> <p> <b>How is a developer of average skills going to learn about these new advanced tools? How can we plan for excellent documentation and training, community mentoring, exchange of good practices, etc... across all EU countries?</b> </p> <p>For the most part, developers do not learn things for the sake of learning. When they have learned something and it is adequate, they stay with it for the most part and are even reluctant to engage in cross-camps interaction. The research world is often similarly insular. A new inflection in the application landscape is needed to drive learning. This inflection is provided by the <a href="https://wiki.mozilla.org/Labs/Ubiquity" id="link-id0x7f051c8">ubiquity</a> of mobile devices, sensor data, explicit semantics, NLP concept extraction, web of linked data, and such factors.</p> <p>RDFa is a good example of a new technique piggybacking on something everybody uses, namely HTML. These new things should, within possibility, be deployed in the usual technology stack, <a href="http://en.wikipedia.org/wiki/LAMP_%28software_bundle%29" id="link-id0x77151e0">LAMP</a> or Java. Of course these do not have to be LAMP or Java or HTML or HTTP themselves but they must manifest through these.</p> <p>A lot of the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x7940cd0">semantic web</a> potential can be realized within the client-server database application model, thus no fundamental re-architecting, just some new data types and queries.</p> <p>For data- or processing-intensive tasks, an on-demand hookup to cloud-based servers with Erlang and/or BOOM for programming model would be easy enough to learn and utilize.</p> <p>The question is one of providing challenges. Addressing actual challenges with these techniques will lead to maturity, documentation, examples, and training. With virtual, Europe-wide distributed teams a reality in many places, Europe-wide dissemination is no longer insurmountable.</p> <p>As the data overflow proceeds, its victims will multiply and create demand for solutions. The EC could here encourage research project use cases gaining an extended life past the end of research projects, possibly being maintained and multiplied and spun off.</p> <p>If such things could be mutated into self-sustaining service businesses with pay-per-use revenue, say through a cloud SaaS business model, still primarily leveraging an open source technology stack, we could have self-propagating and self-supporting models for exploiting advanced IT. This would create interest, and interest would drive training and dissemination.</p> <p>The problem is creating the pull.</p> </li> </ol> </li> <li> <p> <b>Challenges</b> </p> <ol type="a" start="1"> <li> <p> <b>What should be, in this domain, the equivalent of the Netflix challenge, Ansari X Prize, <a href="http://dbpedia.org/resource/Google" id="link-id0x7e72f40">Google</a> Lunar X Prize, etc. ... ?</b> </p> <p>The EC itself no doubt suffers from data overflow in one function or another. Unless security/secrecy prohibits, simply publishing a large data set and a description of what operations should be done on it would be a start. The more real the data, the better â reality is consistently more complex and surprising than imagination. Since many interesting problems touch on fraud detection and law enforcement, there may be some security obstacles for using these application domains as subject matters of open challenges.</p> <p>Once there is a good benchmark, as discussed above, there can be some prize money allocated for the winners, specially if the race is tight.</p> <p>The Semantic Web Challenge and the Billion Triples Challenge exist and are useful as such, but do not seem to have any huge impact.</p> <p>The incentives should be sufficient and part of the expenses arising from running for such challenges could be funded. Otherwise investing in existing business development will be more interesting to industry. Some industry participation seems necessary; we would wish academia and industry to work closer. Also, having industry supply the baseline guarantees that academia actually does further the state of the art. This is not always certain.</p> <p>If challenges are based on actual problems, whether of the EC, its member governments, or private entities, and winning the challenge may lead to a contract for supplying an actual solution, these will naturally become more interesting for consortia involving integrators, specialist software vendors, and academia. Such a model would build actual capacity to deploy leading edge technologies in production, which is sorely needed.</p> </li> <li> <p> <b>What should one do to set up such a challenge, administer, and monitor it?</b> </p> <p>The EC should probably circulate a call for actual problem scenarios involving big data. If the matter of the overflow is as dire as represented, cases should be easy to find. A few should be selected and then anonymized if needed.</p> <p>The party with the use case would benefit by having hopefully the best work on it. The contestants would benefit from having real world needs guide R&D. The EC would not have to do very much, except possibly use some money for funding the best proposals. The winner would possibly get a large account and related sales and service income. The contestants would have to be teams possibly involving many organizations; for example, development and first-line services and support could come from different companies along a systems integrator model such as is widely used in the US.</p> <p>There may be a good benchmark at the time, possibly resulting from FP7 itself. In such a case, the EC could offer a prize for winners. Details would have to be worked out case by case. Such a challenge could be repeated a few times, as benchmark-driven progress in databases or TREC for example have taken some years to reach a point of slowdown in progress.</p> <p>Administrating such an activity should not be prohibitive, as most of the expertise can be found with the stakeholders.</p> </li> </ol> </li> </ol>
Provenance and Reification in Virtuoso
http://www.openlinksw.com/blog/vdb/blog/?date=2009-09-01#1573
2009-09-01T14:44:08Z
2009-09-01T11:20:46.000006-04:00
<p>These days, <a href="http://dbpedia.org/resource/Data" id="link-id0x37019c8">data</a> provenance is a big topic across the board, ranging from the <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x53c3620">linked data</a> <a href="http://dbpedia.org/resource/Giant_Global_Graph" id="link-id0x4aa3848">web</a>, to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x385aff0">RDF</a> in general, to any kind of data integration, with or without RDF. Especially with scientific data we encounter the need for metadata and provenance, repeatability of experiments, etc. Data without context is worthless, yet the producers of said data do not always have a model or budget for metadata. And if they do, the approach is often a proprietary relational schema with web services in front.</p> <p>RDF and linked data principles could evidently be a great help. This is a large topic that goes into the culture of doing science and will deserve a more extensive treatment down the road.</p> <p>For now, I will talk about possible ways of dealing with provenance annotations in <a href="http://virtuoso.openlinksw.com" id="link-id0x51c4da0">Virtuoso</a> at a fairly technical level.</p> <p>If data comes many-triples-at-a-time from some source (e.g., library catalogue, user of a social network), then it is often easiest to put the data from each source/user into its own graph. Annotations can then be made on the graph. The graph IRI will simply occur as the subject of a triple in the same or some other graph. For example, all such annotations could go into a special annotations graph.</p> <p>On the query side, having lots of distinct graphs does not have to be a problem if the index scheme is the right one, i.e., the 4 index scheme <a href="http://docs.openlinksw.com/virtuoso/rdfperformancetuning.html#rdfperfindexes" id="link-id142a0798">discussed in the Virtuoso documentation</a>. If the query does not specify a graph, then triples in any graph will be considered when evaluating the query.</p> <p>One could write queries like â</p> <blockquote> <code><pre>SELECT ?pub WHERE { GRAPH ?g { ?person foaf:knows ?contact } ?contact foaf:name "Alice" . ?g xx:has_publisher ?pub }</pre> </code> </blockquote> <p>This would return the publishers of graphs that assert that somebody knows Alice.</p> <p>Of course, the <a href="http://www.w3.org/TR/2004/REC-rdf-primer-20040210/#reification" id="link-id14fa9488">RDF reification vocabulary</a> can be used as-is to say things about single triples. It is however very inefficient and is not supported by any specific optimization. Further, reification does not seem to get used very much; thus there is no great pressure to specially optimize it.</p> <p>If we have to say things about specific triples and this occurs frequently (i.e., for more than 10% or so of the triples), then modifying the quad table becomes an option. For all its inefficiency, the RDF reification vocabulary is applicable if reification is a rarity.</p> <p>Virtuoso's <code>RDF_QUAD</code> table can be altered to have more columns. The problem with this is that space usage is increased and the RDF loading and query functions will not know about the columns. A <a href="http://dbpedia.org/resource/SQL" id="link-id0x4784bf0">SQL</a> update statement can be used to set values for these additional columns if one knows the <code>G,S,P,O</code>. </p> <p>Suppose we annotated each quad with the user who inserted it and a timestamp. These would be columns in the <code>RDF_QUAD</code> table. The next choice would be whether these were primary key parts or dependent parts. If primary key parts, these would be non-<code>NULL</code> and would occur on every index. The same quad would exist for each distinct user and time this quad had been inserted. For loading functions to work, these columns would need a default. In practice, we think that having such metadata as a dependent part is more likely, so that <code>G,S,P,O</code> are the unique identifier of the quad. Whether one would then include these columns on indices other than the primary key would depend on how frequently they were accessed.</p> <p>In <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x4a8a7c0">SPARQL</a>, one could use an extension syntax like â</p> <blockquote> <code><pre>SELECT * WHERE { ?person foaf:knows ?connection OPTION ( time ?ts ) . ?connection foaf:name "Alice" . FILTER ( ?ts > "2009-08-08"^^xsd:datetime ) }</pre> </code> </blockquote> <p>This would return everybody who knows Alice since a date more recent than 2009-08-08. This presupposes that the quad table has been extended with a datetime column.</p> <p>The <code>OPTION (time ?ts)</code> syntax is not presently supported but we can easily add something of the sort if there is user demand for it. In practice, this would be an extension mechanism enabling one to access extension columns of <code>RDF_QUAD</code> via a column <code>?variable</code> syntax in the <code>OPTION</code> clause.</p> <p>If quad metadata were not for every quad but still relatively frequent, another possibility would be making a separate table with a key of <code>GSPO</code> and a dependent part of <code>R</code>, where <code>R</code> would be the reification <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x49e6108">URI</a> of the quad. Reification statements would then be made with <code>R</code> as a subject. This would be more compact than the reification vocabulary and would not modify the <code>RDF_QUAD</code> table. The syntax for referring to this could be something like â</p> <blockquote> <code><pre>SELECT * WHERE { ?person foaf:knows ?contact OPTION ( reify ?r ) . ?r xx:assertion_time ?ts . ?contact foaf:name "Alice" . FILTER ( ?ts > "2008-8-8"^^xsd:datetime ) }</pre> </code> </blockquote> <p>We could even recognize the reification vocabulary and convert it into the reify option if this were really necessary. But since it is so unwieldy I don't think there would be huge demand. Who knows? You tell us.</p>
More On Parallel RDF/Text Query Evaluation
http://www.openlinksw.com/blog/vdb/blog/?date=2009-08-19#1571
2009-08-19T17:28:50Z
2009-08-19T14:00:32.000006-04:00
<p>We have received some more questions about <a href="http://virtuoso.openlinksw.com" id="link-id0x15ca9a30">Virtuoso</a>'s parallel query evaluation model.</p> <p>In answer, we will here explain how we do search engine style processing by writing <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1574c560">SPARQL</a>. There is no need for custom procedural code because the query optimizer does all the partitioning and the equivalent of map reduce.</p> <p>The point is that what used to require programming can often be done in a generic query language. The technical detail is that the implementation must be smart enough with respect to parallelizing queries for this to be of practical benefit. But by combining these two things, we are a step closer to the web being the database.</p> <p>I will here show how we do some joins combining full text, <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x15949970">RDF</a> conditions, and aggregates and <code>ORDER BY</code>. The sample task is finding the top 20 entities with New York in some attribute value. Then we specify the search further by only taking actors associated with New York. The results are returned in the order of a composite of <a href="http://dbpedia.org/resource/Entity" id="link-id0x213bf310">entity</a> rank and text match score.</p> <p>The basic query is:</p> <blockquote> <code><pre> SELECT ( <a href="http://dbpedia.org/resource/SQL" id="link-id0x23632230">sql</a>:s_sum_page ( <sql:vector_agg> ( <bif:vector> ( ?c1 , ?sm ) ), bif:vector ( 'new', 'york' ) ) ) AS ?res WHERE { { SELECT ( <SHORT_OR_LONG::>(?s1) ) AS ?c1 ( <sql:S_SUM> ( <SHORT_OR_LONG::IRI_RANK> ( ?s1 ) , <SHORT_OR_LONG::> ( ?s1textp ) , <SHORT_OR_LONG::> ( ?o1 ) , ?sc ) ) AS ?sm WHERE { ?s1 ?s1textp ?o1 . ?o1 bif:contains "new AND york" OPTION ( SCORE ?sc ) } ORDER BY DESC ( <sql:sum_rank> (( <sql:S_SUM> ( <SHORT_OR_LONG::IRI_RANK> ( ?s1 ) , <SHORT_OR_LONG::> ( ?s1textp ) , <SHORT_OR_LONG::> ( ?o1 ) , ?sc ) )) ) LIMIT 20 } } </pre> </code> </blockquote> <p>This takes some explaining. The basic part is</p> <blockquote> <code><pre>{ ?s1 ?s1textp ?o1 . ?o1 bif:contains "new AND york" OPTION ( SCORE ?sc ) }</pre> </code> </blockquote> <p>This just makes tuples where <code>?s1</code> is the object, <code>?s1textp</code> the property, and <code>?o1</code> the literal which contains "New York". For a single <code>?s1</code>, there can of course be many properties which all contain "New York".</p> <p>The rest of the query gathers all the "New York" containing properties of an entity into a single aggregate, and then gets the entity ranks of all such entities.</p> <p>After this, the aggregates are sorted by a sum of the entity rank and a combined text score calculated based on the individual text match scores between "New York" and the strings containing "New York". The text hit score is higher if the words repeat often and in close proximity.</p> <p>The <code>s_sum</code> function is a user-defined aggregate which takes 4 arguments: The rank of the subject of the triple; the predicate of the triple containing the text; the object of the triple containing the text; and the text match score.</p> <p>These are grouped by the subject of the triple. After this, these are sorted by <code>sum_score</code> of the aggregate constructed with <code>s_sum</code>. The <code>sum_score</code> is a SQL function combining the entity rank with the text scores of the different literals.</p> <p>This executes as one would expect: All partitions make a text index lookup, retrieving the object of the triple. The text index entries of an object are stored in the same partition as the object. But the entity rank is a property of the subject and is partitioned by the subject. Also the <code>GROUP BY</code> is by the subject. Thus the <a href="http://dbpedia.org/resource/Data" id="link-id0x15da01b8">data</a> is produced from all partitions, then streamed into the receiving partitions, determined by the subject. This partition can then get the score and group the matches by the subject. Since all these partial aggregates are partitioned by the subject, there is no need to merge them; thus, the top <code>k</code> sort can be done for each partition separately. Finally, the top 20 of each partition are merged into the global top 20. This is then passed to a final function <code>s_sum_page</code> that turns this all into an <a href="http://dbpedia.org/resource/XML" id="link-id0x15d59fc8">XML</a> fragment that can be processed with XSLT for inclusion on a web page.</p> <p>This differs from the text search engine in that the query pipeline can contain arbitrary cross-partition joins. Also, the string "New York" is a common label that occurs in many distinct entities. Thus one text match, to one document, in the case the containing only the string "New York" will get many entities, likely all from different partitions.</p> <p>So, if we only want actors with a mention of "New York", we need to get the inner part of the query as:</p> <blockquote> <code><pre>{ ?s1 ?s1textp ?o1 . ?o1 bif:contains "new AND york" OPTION ( SCORE ?sc ) . ?s1 a <<a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x15befb10">http</a>://<a href="http://umbel.org/about/" id="link-id0x15c92330">umbel</a>.org/umbel/sc/Actor> }</pre> </code> </blockquote> <p>Whether an entity is an actor can be checked in the same partition as the rank of the entity. Thus the query plan gets this check right before getting the rank. This is natural since there is no point in getting the rank of something that is not an actor.</p> <p>The <code><short_or_long::sql:func></code> notation means that we call <code>func</code>, which is a SQL stored procedure with the arguments in their internal form. Thus, if a variable bound to an IRI is passed, the <code>short_or_long</code> specifies that it is passed as its internal ID and is not converted into its text form. This is essential, since there is no point getting the text of half a million IRIs when only 20 at most will be shown in the end.</p> <p>Now, when we run this on a collection of 4.5 billion triples of <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x153772e8">linked data</a>, once we have the working set, we can get the top 20 "New York" occurrences, with text summaries and all, in just 1.1s, with 12 of 16 cores busy. (The hardware is two boxes with two quad-core Xeon 5345 each.)</p> <p>If we run this query in two parallel sessions, we get both results in 1.9s, with 14 of 16 cores busy. This gets about 200K "New York" strings, which becomes about 400K entities with New York somewhere, for which a rank then has to be retrieved. After this, all the possibly-many occurrences of New York in the title, text, and other properties of the entity are aggregated together, resolving into some 220K groups. These are then sorted. This is internally over 1.5 million random lookups and some 40MB of traffic between processes. Restricting the type of the entity to actor drops the execution time of one query to 0.8s because there are then fewer ranks to retrieve and less data to aggregate and sort.</p> <p>By adding partitions and cores, we scale horizontally, as evaluating the query involves almost no central control, even though data are swapped between partitions. There is some flow control to avoid constructing overly-large intermediate results but generally partitions run independently and asynchronously. In the above case, there is just one fence at the point where all aggregates are complete, so that they can be sorted; otherwise, all is asynchronous.</p> <p>Doing <code>JOINs</code> between partitions and partitioned <code>GROUP BY</code>/<code>ORDER BY</code> is pretty regular database stuff. Applying this to RDF is a most natural thing.</p> <p>If we do not parallelize the user-defined aggregate for grouping all the "New York" occurrences, the query takes 8s instead of 1.1s. If we could not put SQL procedures as user-defined aggregates to be parallelized with the query, we'd have to either bring all the data to a central point before the top k, which would destroy performance, or we would have to do procedures with explicit parallel procedure calls which is hard to write, surely too hard for <i>ad hoc</i> queries.</p> <a href="http://bit.ly/4jAVHC" id="link-id114d58f0">Results of live execution</a> may not be complete on initial load, as this link includes a "Virtuoso Anytime" timeout of 10 seconds. Running against a cold cache, these results may take much longer to return; a warm cache will deliver response times along the lines of those discussed above. <p>Engineering matters. If we wish to commoditize queries on a lot of data, such intelligence in the DBMS is necessary; it is very unscalable to require people to do procedural code or give query parallelization hints. If you need to optimize a workload of 10 different transactions, this is of course possible and even desirable, but for the infinity of all search or analysis, this will not happen.</p>
Updated hardware improves LUBM 8000 load rate in Virtuoso 6
http://www.openlinksw.com/blog/vdb/blog/?date=2009-08-14#1569
2009-08-14T19:01:30Z
2009-08-15T15:27:29-04:00
<p>We repeated the <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1562" id="link-id173d3068">earlier LUBM 8000 experiment</a> on a newer machine, with 2 x Xeon 5520 and 72G 1333MHz memory, and once again with the 2 machines as a networked cluster. Otherwise the settings were the same.</p> <p>The load rate is now 160,739 triples-per-second.</p> <table> <tr> <th></th> <td> </td> <th align="center"><a href="http://virtuoso.openlinksw.com" id="link-id0x240daf38">Virtuoso</a> 6 <br /> (previous run)</th> <td> </td> <th align="center">Virtuoso 6 <br /> (new run)</th> <td> </td> <th align="center">Virtuoso 6 <br /> (newest run)</th> </tr> <tr> <td align="left">blades</td> <td> </td> <td align="center">1 </td> <td> </td> <td align="center">1 </td> <td> </td> <td align="center">2</td> </tr> <tr> <td align="left">processors</td> <td> </td> <td align="center">2 x Xeon 5410</td> <td> </td> <td align="center">2 x Xeon 5520</td> <td> </td> <td align="center"> 2 x Xeon 5520 <br />+ <br />2 x Xeon 5410 <br />with 1x1GigE <br />interconnect </td> </tr> <tr> <td align="left">memory</td> <td> </td> <td align="center"> 16G 667 MHz</td> <td> </td> <td align="center">72G 1333 MHz</td> <td> </td> <td align="center">72G 1333 MHz <br />+ <br /> 16G 667 MHz <br /> respectively</td> </tr> <tr> <td align="left">reported load rate<br />triples-per-second</td> <td> </td> <td align="center"> 110,532 </td> <td> </td> <td align="center"> 160,739 </td> <td> </td> <td align="center"> 214,188 </td> </tr> </table> <p>Again, if others talk about loading LUBM, so must we. Otherwise, this metric is rather uninteresting.</p>
Single Virtuoso host loads 110,500 triples-per-second on LUBM 8000
http://www.openlinksw.com/blog/vdb/blog/?date=2009-06-29#1563
2009-06-29T16:12:34Z
2009-08-15T16:06:45.000001-04:00
<p>LUBM load speed still seems to be a metric that is quoted in comparisons of <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id142df6e8">RDF</a> stores. Consequently, we too measured the load time of LUBM 8000, 1,068-million triples, on the newest <a href="http://virtuoso.openlinksw.com" id="link-id1389dfa0">Virtuoso</a>.</p> <p>The real time for the load was 161m 3s. The rate was 110,532 triples-per-second. The hardware was one machine with 2 x Xeon 5410 (quad core, 2.33 GHz) and 16G 667 MHz RAM. The software was Virtuoso 6 Cluster, configured into 8 partitions (processes) — one partition per CPU core. Each partition had its database striped over 6 disks total; the 6 disks on the system were shared between the 8 database processes.</p> <p>The load was done on 8 streams, one per server process. At the beginning of the load, the CPU usage was 740% with no disk; at the end, it was around 700% with 25% disk wait. 100% counts here for one CPU core or one disk being constantly busy.</p> <p>The RDF store was configured with the default two indices over quads, these being GSPO and OGPS. Text indexing of literals was not enabled. No materialization of entailed triples was made.</p> <p>We think that LUBM loading is not a realistic benchmark for the world but since other people publish such numbers, so do we.</p>
Comparing Virtuoso Performance on Different Processors
http://www.openlinksw.com/blog/vdb/blog/?date=2009-05-28#1558
2009-05-28T14:54:59Z
2009-05-28T11:15:41.000006-04:00
<p>Over the years we have run <a href="http://virtuoso.openlinksw.com" id="link-id0x16735e20">Virtuoso</a> on different hardware. We will here give a few figures that help identify the best price point for machines running Virtuoso.</p> <p>Our test is very simple: <i>Load 20 warehouses of <a href="http://dbpedia.org/resource/TPC-C" id="link-id0x16e0dba8">TPC-C</a> <a href="http://dbpedia.org/resource/Data" id="link-id0x14ff4f80">data</a>, and then run one client per warehouse for 10,000 new orders</i>. The way this is set up, disk I/O does not play a role and lock contention between the clients is minimal.</p> <p>The test essentially has 20 server and 20 client threads running the same workload in parallel. The load time gives the single thread number; the 20 clients run gives the multi-threaded number. The test uses about 2-3 GB of data, so all is in RAM but is large enough not to be all in processor cache.</p> <p>All times reported are real times, starting from the start of the first client and ending with the completion of the last client.</p> <p>Do not confuse these results with official TPC-C. The measurement protocols are entirely incomparable.</p> <style type="text/css"> TABLE { background: none; border: none } TH { text-align: center; font-weight: bold } TR.top { background: } TD { text-align: center; border: none } </style> <table align="center" cellspacing="10"> <tr> <th>Test</th> <th>Platform</th> <th>Load<br />(seconds)</th> <th>Run<br />(seconds)</th> <th>GHz / cores / threads</th> </tr> <tr> <td>1</td> <td>Amazon <a href="http://aws.amazon.com/ec2/" id="link-id0x15d68e20">EC2</a> Extra Large<br />(4 virtual cores)</td> <td>340</td> <td>42</td> <td>1.2 GHz? / 4 / 1</td> </tr> <tr> <td>1</td> <td>Amazon EC2 Extra Large<br />(4 virtual cores)</td> <td>305</td> <td>43.3</td> <td>1.2 GHz? / 4 / 1</td> </tr> <tr> <td>2</td> <td>1 x dual-core AMD 5900</td> <td>263</td> <td>58.2</td> <td>2.9 GHz / 2 / 1</td> </tr> <tr> <td>3</td> <td>2 x dual-core Xeon 5130 ("Woodcrest")</td> <td>245</td> <td>35.7</td> <td>2.0 GHz / 4 / 1</td> </tr> <tr> <td>4</td> <td>2 x quad-core Xeon 5410 ("Harpertown")</td> <td>237</td> <td>18.0</td> <td>2.33 GHz / 8 / 1</td> </tr> <tr> <td>5</td> <td>2 x quad-core Xeon 5520 ("Nehalem")</td> <td>162</td> <td>18.3</td> <td>2.26 GHz / 8 / 2</td> </tr> </table> <p>We tried two different EC2 instances to see if there would be variation. The variation was quite small. The tested EC2 instances costs 20 US cents per hour. The AMD dual-core costs 550 US dollars with 8G. The 3 Xeon configurations are Supermicro boards with 667MHz memory for the Xeon 5130 ("Woodcrest") and Xeon 5410 ("Harpertown"), and 800MHz memory for the Nehalem. The Xeon systems cost between 4000 and 7000 US dollars, with 5000 for a configuration with 2 x Xeon 5520 ("Nehalem"), 72 GB RAM, and 8 x 500 GB SATA disks.</p> <p> <i>Caveat: Due to slow memory (we could not get faster within available time), the results for the Nehalem do not take full advantage of its principal edge over the previous generation, i.e., memory subsystem. We'll see another time with faster memories.</i> </p> <p>The operating systems were various 64 bit Linux distributions.</p> <p>We did some further measurements comparing Harpertown and Nehalem processors. The Nehalem chip was a bit faster for a slightly lower clock but we did not see any of the twofold and greater differences advertised by Intel.</p> <p>We tried some <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1460b688">RDF</a> operations on the two last systems:</p> <table align="center" cellspacing="10"> <tr> <th>operation</th> <th> Harpertown</th> <th>Nehalem</th> </tr> <tr> <th>Build text index for <a href="http://dbpedia.org/resource/DBpedia" id="link-id0x16a94590">DBpedia</a></th> <td>1080s</td> <td>770s</td> </tr> <tr> <th><a href="http://dbpedia.org/resource/Entity" id="link-id0xc37f380">Entity</a> Rank iteration</th> <td>263s</td> <td>251s</td> </tr> </table> <p>Then we tried to see if the core multithreading of Nehalem could be seen anywhere. To this effect, we ran the Fibonacci function in <a href="http://dbpedia.org/resource/SQL" id="link-id0x15842a20">SQL</a> to serve as an example of an all in-cache integer operation. 16 concurrent operations took exactly twice as long as 8 concurrent ones, as expected.</p> <p>For something that used memory, we took a count of RDF quads on two different indices, getting the same count. The database was a cluster setup with one process per core, so a count involved one thread per core. The counts in series took 5.02s and in parallel they took 4.27s.</p> <p>Then we took a more memory intensive piece that read the RDF quads table in the order of one index and for each row checked that there is the equal row on another, differently-partitioned index. This is a cross-partition join. One of the indices is read sequentially and the other at random. The throughput can be reported as random-lookups-per-second. The data was English DBpedia, about 140M triples. One such query takes a couple of minutes with a 650% CPU utilization. Running multiple such queries should show effects of core multithreading since we expect frequent cache misses.</p> <ol> <li>On the host OS of the Nehalem system â <table align="center" cellspacing="10"> <tr> <th>n</th> <th>cpu%</th> <th>rows per second</th> </tr> <tr> <th>1 query</th> <td>503</td> <td>906,413</td> </tr> <tr> <th>2 queries</th> <td>1263</td> <td>1,578,585</td> </tr> <tr> <th>3 queries</th> <td>1204</td> <td>1,566,849</td> </tr> </table> </li> <li>In a VM under Xen, on the Nehalem system â <table align="center" cellspacing="10"> <tr> <th>n</th> <th>cpu%</th> <th>rows per second</th> </tr> <tr> <th>1 query</th> <td>652</td> <td>799,293</td> </tr> <tr> <th>2 queries</th> <td>1266</td> <td>1,486,710</td> </tr> <tr> <th>3 queries</th> <td>1222</td> <td>1,484,093</td> </tr> </table> </li> <li> On the host OS of the Harpertown system â <table align="center" cellspacing="10"> <tr> <th>n</th> <th>cpu%</th> <th>rows per second</th> </tr> <tr> <th>1 query</th> <td> 648 </td> <td> 1,041,448 </td> </tr> <tr> <th>2 queries</th> <td> 708 </td> <td> 1,124,866 </td> </tr> </table> </li> </ol> <p>The CPU percentages are as reported by the OS: user + system CPU divided by real time.</p> <p>So, Nehalem is in general somewhat faster, around 20-30%, than Harpertown. The effect of core multithreading can be noticed but is not huge, another 20% or so for situations with more threads than cores. The join where Harpertown did better could be attributed to its larger cache â 12 MB vs 8 MB.</p> <p>We see that Xen has a measurable but not prohibitive overhead; count a little under 10% for everything, also tasks with no I/O. The VM was set up to have all CPU for the test and the queries did not do disk I/O.</p> <p>The executables were compiled with <code>gcc</code> with default settings. Specifying <code>-march=nocona</code> (Core 2 target) dropped the cross-partition join time mentioned above from 128s to 122s on Harpertown. We did not try this on Nehalem but presume the effect would be the same, since the out-of-order unit is not much different. We did not do anything about process-to-memory affinity on Nehalem, which is a non-uniform architecture. We would expect this to increase performance since we have many equal size processes with even load.</p> <p>The mainstay of the Nehalem value proposition is a better memory subsystem. Since the unit we got was at 800 MHz memory, we did not see any great improvement. So if you buy Nehalem, you should make sure it is with 1333 MHz memory, else the best case will not be over 50% over a 667 MHz Core 2-based Xeon.</p> <p>Nehalem remains a better deal for us because of more memory per board. One Nehalem box with 72 GB costs less than two Harpertown boxes with 32 GB and offers almost the same performance. Having a lot of memory in a small space is key. With faster memory, it might even outperform two Harpertown boxes, but this remains to be seen.</p> <p>If space were not a constraint, we could make a cluster of 12 small workstations for the price of our largest system and get still more memory and more processor power per unit of memory. The Nehalem box was almost 4x faster than the AMD box but then it has 9x the memory, so the CPU to memory ratio might be better with the smaller boxes.</p>
Short Recap of Virtuoso Basics (#3 of 5)
http://www.openlinksw.com/blog/vdb/blog/?date=2009-04-30#1552
2009-04-30T15:49:53Z
2009-04-30T12:11:45-04:00
<p>(Third of five posts related to the <a href="http://www2009.org/" id="link-id0x14b582b8">WWW 2009</a> conference, held the week of April 20, 2009.) </p> <p>There are some points that came up in conversation at WWW 2009 that I will reiterate here. We find there is still some lack of clarity in the product image, so I will here condense it.</p> <p> <a href="http://virtuoso.openlinksw.com" id="link-id0x14bf48b8">Virtuoso</a> is a DBMS. We pitch it primarily to the <a href="http://dbpedia.org/resource/Data" id="link-id0x16bc4490">data</a> web space because this is where we see the emerging frontier. Virtuoso does both <a href="http://dbpedia.org/resource/SQL" id="link-id0x1223dc30">SQL</a> and <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x170eec88">SPARQL</a> and can do both at large scale and high performance. The popular perception of <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x15a05fc0">RDF</a> and Relational models as mutually exclusive and antagonistic poles is based on the poor scalability of early RDF implementations. What we do is to have all the RDF specifics, like IRIs and typed literals as native SQL types, and to have a cost based optimizer that knows about this all.</p> <p>If you want application-specific data structures as opposed to a schema-agnostic quad-store model (triple + graph-name), then Virtuoso can give you this too. <a href="http://docs.openlinksw.com/virtuoso/rdfsparqlintegrationmiddleware.html#rdfviews" id="link-id14ddc7c8">Rendering application specific data structures as RDF</a> applies equally to relational data in non-Virtuoso databases because Virtuoso SQL can <a href="http://docs.openlinksw.com/virtuoso/qsvdbsrv.html" id="link-id14aaea70">federate tables from heterogenous DBMS</a>.</p> <p>On top of this, there is a <a href="http://docs.openlinksw.com/virtuoso/qswebserver.html" id="link-id16fcde60">web server built in</a>, so that no extra server is needed for web services, web pages, and the like.</p> <p>Installation is simple, just one exe and one config file. There is a huge amount of code in <a href="http://docs.openlinksw.com/virtuoso/installation.html" id="link-id16767b40">installers</a> â application code and test suites and such â but none of this is needed when you deploy. Scale goes from a 25MB memory footprint on the desktop to hundreds of gigabytes of RAM and endless terabytes of disk on shared-nothing clusters.</p> <p>Clusters (coming in Release 6) and SQL federation are <a href="http://download.openlinksw.com/download/product_matrix.vsp?p=l_os&c=39&df=16" id="link-id16722550">commercial only</a>; the rest can be had <a href="http://sourceforge.net/project/showfiles.php?group_id=161622" id="link-id131080a8">under GPL</a>.</p> <p>To condense further:</p> <ul> <li>Scalable Delivery of <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x1060ad98">Linked Data</a> </li> <li>SPARQL and SQL <ul> <li>Arbitrary RDF Data + Relational</li> <li>Also From 3rd Party <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x16bbce60">RDBMS</a> </li> </ul> </li> <li>Easy Deployment </li> <li>Standard Interfaces <ul> <li> <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0x12e284d8">ODBC</a>, <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0xb5e1400">JDBC</a>, OLE DB, <a href="http://dbpedia.org/resource/ADO.NET" id="link-id0x15a55db8">ADO</a>.<a href="http://dbpedia.org/resource/.NET_Framework" id="link-id0x16beb070">NET</a>, XMLA</li> <li> <a href="http://jena.sourceforge.net/" id="link-id0x122b5008">Jena</a>, <a href="http://sourceforge.net/projects/sesame/" id="link-id0x148d4078">Sesame</a>, etc.</li> <li>All Web Protocols </li> </ul> </li> </ul>
Linked Data at WWW 2009 (#1 of 5)
http://www.openlinksw.com/blog/vdb/blog/?date=2009-04-27#1545
2009-04-27T21:28:11Z
2009-04-28T11:27:57-04:00
<p>(First of five posts related to the <a href="http://www2009.org/" id="link-id0x12d8ed90">WWW 2009</a> conference, held the week of April 20, 2009.)</p> <p>We gave a talk at the <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x152bf430">Linked Open Data</a> workshop, <a href="http://events.linkeddata.org/ldow2009/" id="link-id0x191721c8">LDOW 2009</a>, at WWW 2009. I did not go very far into the technical points in the talk, as there was almost no time and the points are rather complex. Instead, I emphasized what new things had become possible with recent developments.</p> <p>The problem we do not cease hearing about is scale. We have solved most of it. There is scale in the schema: Put together, ontologies go over a million classes/properties. Which ones are relevant depends, and the user should have the choice. The instance <a href="http://dbpedia.org/resource/Data" id="link-id0x17c8f998">data</a> is in the tens of billions of triples, much derived from Web 2.0 sources but also much published as <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xd562090">RDF</a>.</p> <p>To make sense of this all, we need quick summaries and search. Without navigation via joins, the value will be limited. Fast joining, counting, grouping, and ranking are key.</p> <p>People will use different terms for the same thing. The issue of identity is philosophical. In order to do reasoning one needs strong identity; a statement like <i>x is a bit like y</i> is not very useful in a database context. Whether any x and y can be considered the same depends on the context. So leave this for query time. The conditions under which two people are considered the same will depend on whether you are doing marketing analysis or law enforcement. A general purpose data store cannot anticipate all the possibilities, so smush on demand, as you go, as has been said many times.</p> <p>Against this backdrop, we offer a solution with which anybody who so chooses can play with big data, whether a search or analytics player.</p> <p>We are going in the direction of more and more ad hoc processing at larger and larger scale. With good query parallelization, we can do big joins without complex programming. No explicit Map Reduce jobs or the like. What was done with special code with special parallel programming models, can now be done in <a href="http://dbpedia.org/resource/SQL" id="link-id0x60bd0c48">SQL</a> and <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x13db1ff0">SPARQL</a>.</p> <p>To showcase this, we do <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x10a5dde8">linked data</a> search, browsing, and so on, but are essentially a platform provider.</p> <p>Entry costs into relatively high end databases have dropped significantly. A cluster with 1 TB of RAM sells for $75K or so at today's retail prices and fits under a desk. For intermittent use, the rent for 1TB RAM is $1228 per day on <a href="http://aws.amazon.com/ec2/" id="link-id0xa59039d8">EC2</a>. With this on one side and <a href="http://virtuoso.openlinksw.com" id="link-id0x19f86c10">Virtuoso</a> on the other, a lot that was impractical in the past is now within reach. Like <a href="http://g1o.net/foaf.rdf#me" id="link-id0xa1853af8">Giovanni Tummarello</a> put it for airplanes, the physics are as they were for <a href="http://dbpedia.org/resource/Leonardo_da_Vinci" id="link-id0x12df02e0">da Vinci</a> but materials and engines had to develop a bit before there was commercial potential. So it is also with analytics for everyone.</p> <p>A remark from the audience was that all the stuff being shown, not limited to Virtuoso, was non-standard, having to do with text search, with ranking, with extensions, and was in fact not SPARQL and pure linked data principles. Further, by throwing this all together, one got something overcomplicated, too heavy.</p> <p>I answered as follows, which apparently cannot be repeated too much:</p> <p>First, everybody expects a text search box, and is conditioned to having one. No text search and no ranking is a non-starter. <i>Ceterum censeo</i>, for database, the next generation cannot be less expressive than the previous. All of SQL and then some is where SPARQL must be. The barest minimum is being able to say anything one can say in SQL, and then justify SPARQL by saying that it is better for heterogenous data, schema last, and so on. On top of this, transitivity and rules will not hurt. For now, the current SPARQL working group will at least reach basic SQL parity; the edge will still remain implementation dependent.</p> <p>Another remark was that joining is slow. Depends. Anything involving more complex disk access than linear reading of a blob is generally not good for interactive use. But with adequate memory, and with all hot spots in memory, we do some 3.2 million random-accesses-per-second on 12 cores, with easily 80% platform utilization for a single large query. The high utilization means that times drop as processing gets divided over more partitions.</p> <p>There was a talk about <a href="http://semanticweb.org/wiki/MashQL" id="link-id0x1642a780">MashQL</a> by <a href="http://data.semanticweb.org/person/mustafa-jarrar" id="link-id0x116e5af8">Mustafa Jarrar</a>, concerning an abstraction on top of SPARQL for easy composition of tree-structured queries. The idea was that such queries can be evaluated "on the fly" as they are being composed. As it happens, we already have an <a href="http://dbpedia.org/resource/XML" id="link-id0x11442520">XML</a>-based query abstraction layer incorporated into Virtuoso 6.0's built-in <a href="http://lod.openlinksw.com/fct/facet.vsp" id="link-id0x6a9ebfe0">Faceted Data Browser Service</a>, and the effects are probably quite similar. The most important point here is that by using XML, both of these approaches are interoperable against a Virtuoso back-end. Along similar lines, we did not get to talk to the G Facets people but our message to them is the same: <i>Use the <a href="http://lod.openlinksw.com/fct/facet.vsp" id="link-id0x1676e158">faceted browser service</a> to get vastly higher performance when querying against Linked Data, be it <a href="http://dbpedia.org/resource/DBpedia" id="link-id0x12653418">DBpedia</a> or the <a href="http://dbpedia.org/resource/Entity" id="link-id0x10a61e78">entity</a> <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x164150d8">LOD</a> <a href="http://lod.openlinksw.com/" id="link-id0xc5ec918">Cloud</a>. Virtuoso 6.0 (Open Source Edition) "<a href="http://sourceforge.net/project/showfiles.php?group_id=161622&package_id=319652&release_id=677866" id="link-id12159728">TP1</a>" is now publicly available as a Technology Preview (beta).</i> </p> <p>We heard that there is an effort for porting Freebase's Parallax to SPARQL. The same thing applies to this. With a number of different data viewers on top of SPARQL, we come closer to broad-audience linked-data applications. These viewers are still too generic for the end user, though. We fully believe that for both search and transactions, application-domain-specific workflows will stay relevant. But these can be made to a fair degree by specializing generic linked-data-bound controls and gluing them together with some scripting.</p> <p>As said before, the application will interface the user to the vocabulary. The vocabulary development takes the modeling burden from the application and makes for interchangeable experience on the same data. The data in turn is "virtualized" into the database cloud or the local secure server, as the use case may require. </p> <p>For ease of adoption, open competition, and safety from lock-in, the community needs a SPARQL whose usability is not totally dependent on vendor extensions. But we might <i>de facto</i> have that in just a bit, whenever there is a working draft from the SPARQL WG.</p> <p>Another topic that we encounter often is the question of integration (or lack thereof) between communities. For example, database conferences reject <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x12563ea0">semantic web</a> papers and vice versa. Such politics would seem to emerge naturally but are nonetheless detrimental. We really should partner with people who write papers as their principal occupation. We ourselves do software products and use very little time for papers, so some of the bad reviews we have received do make a legitimate point. By rights, we should go for database venues but we cannot have this take too much time. So we are open to partnering for splitting the opportunity cost of multiple submissions.</p> <p>For future work, there is nothing radically new. We continue testing and productization of cluster databases. Just deliver what is in the pipeline. The essential nature of this is adding more and more cases of better and better parallelization in different query situations. The present usage patterns work well for finding bugs and performance bottlenecks. For presentation, our goal is to have third party viewers operate with our platform. We cannot completely leave data browsing and UI to third parties since we must from time to time introduce various unique functionality. Most interaction should however go via third party applications.</p>
Web Scale and Fault Tolerance
http://www.openlinksw.com/blog/vdb/blog/?date=2009-04-01#1541
2009-04-01T15:18:06Z
2009-04-01T11:18:54.000012-04:00
<p>One concern about <a href="http://virtuoso.openlinksw.com" id="link-id0x719d2f8">Virtuoso</a> Cluster is fault tolerance. This post talks about the basics of fault tolerance and what we can do with this, from improving resilience and optimizing performance to accommodating bulk loads without impacting interactive response. We will see that this is yet another step towards a 24/7 web-scale <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0xa9a1d8d8">Linked Data</a> <a href="http://dbpedia.org/resource/Giant_Global_Graph" id="link-id0x25201030">Web</a>. We will see how large scale, continuous operation, and redundancy are related.</p> <p>It has been said many times â when things are large enough, failures become frequent. In view of this, basic storage of partitions in multiple copies is built into the Virtuoso cluster from the start. Until now, this feature has not been tested or used very extensively, aside from the trivial case of keeping all schema <a href="http://dbpedia.org/resource/Information" id="link-id0x4548898">information</a> in synchronous replicas on all servers.</p> <h2>Approaches to Fault Tolerance</h2> <p>Fault tolerance has many aspects but it starts with keeping <a href="http://dbpedia.org/resource/Data" id="link-id0x18757400">data</a> in at least two copies. There are shared-disk cluster databases like <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x711c900">Oracle</a> RAC that do not depend on partitioning. With these, as long as the disk image is intact, servers can come and go. The fault tolerance of the disk in turn comes from mirroring done by the disk controller. Raids other than mirrored disk are not really good for databases because of write speed.</p> <p>With shared-nothing setups like Virtuoso, fault tolerance is based on multiple servers keeping the same logical data. The copies are synchronized transaction-by-transaction but are not bit-for-bit identical nor write-by-write synchronous as is the case with mirrored disks.</p> <p>There are asynchronous replication schemes generally based on log shipping, where the replica replays the transaction log of the master copy. The master copy gets the updates, the replica replays them. Both can take queries. These do not guarantee an entirely ACID fail-over but for many applications they come close enough.</p> <p>In a tightly coupled cluster, it is possible to do synchronous, transactional updates on multiple copies without great added cost. Sending the message to two places instead of one does not make much difference since it is the latency that counts. But once we go to wide area networks, this becomes as good as unworkable for any sort of update volume. Thus, wide area replication must in practice be asynchronous.</p> <p>This is a subject for another discussion. For now, the short answer is that wide area log shipping must be adapted to the application's requirements for synchronicity and consistency. Also, exactly what content is shipped and to where depends on the application. Some application-specific logic will likely be involved; more than this one cannot say without a specific context.</p> <h2>Basics of Partition Fail-Over</h2> <p>For now, we will be concerned with redundancy protecting against broken hardware, software slowdown, or crashes inside a single site.</p> <p>The basic idea is simple: Writes go to all copies; reads that must be repeatable or serializable (i.e., locking) go to the first copy; reads that refer to committed state without guarantee of repeatability can be balanced among all copies. When a copy goes offline, nobody needs to know, as long as there is at least one copy online for each partition. The exception in practice is when there are open cursors or such stateful things as aggregations pending on a copy that goes down. Then the query or transaction will abort and the application can retry. This looks like a deadlock to the application.</p> <p>Coming back online is more complicated. This requires establishing that the recovering copy is actually in sync. In practice this requires a short window during which no transactions have uncommitted updates. Sometimes, forcing this can require aborting some transactions, which again looks like a deadlock to the application.</p> <p>When an error is seen, such as a process no longer accepting connections and dropping existing cluster connections, we in practice go via two stages. First, the operations that directly depended on this process are aborted, as well as any computation being done on behalf of the disconnected server. At this stage, attempting to read data from the partition of the failed server will go to another copy but writes will still try to update all copies and will fail if the failed copy continues to be offline. After it is established that the failed copy will stay off for some time, writes may be re-enabled â but now having the failed copy rejoin the cluster will be more complicated, requiring an atomic window to ensure sync, as mentioned earlier.</p> <p>For the DBA, there can be intermittent software crashes where a failed server automatically restarts itself, and there can be prolonged failures where this does not happen. Both are alerts but the first kind can wait. Since a system must essentially run itself, it will wait for some time for the failed server to restart itself. During this window, all reads of the failed partition go to the spare copy and writes give an error. If the spare does not come back up in time, the system will automatically re-enable writes on the spare but now the failed server may no longer rejoin the cluster without a complex sync cycle. This all can happen in well under a minute, faster than a human operator can react. The diagnostics can be done later.</p> <p>If the situation was a hardware failure, recovery consists of taking a spare server and copying the database from the surviving online copy. This done, the spare server can come on line. Copying the database can be done while online and accepting updates but this may take some time, maybe an hour for every 200G of data copied over a network. In principle this could be automated by scripting, but we would normally expect a human DBA to be involved.</p> <p>As a general rule, reacting to the failure goes automatically without disruption of service but bringing the failed copy online will usually require some operator action.</p> <h2>Levels of Tolerance and Performance</h2> <p>The only way to make failures totally invisible is to have all in duplicate and provisioned so that the system never runs at more than half the total capacity. This is often not economical or necessary. This is why we can do better, using the spare capacity for more than standby.</p> <p>Imagine keeping a repository of linked data. Most of the content will come in through periodic bulk replacement of data sets. Some data will come in through pings from applications publishing FOAF and similar. Some data will come through on-demand RDFization of resources.</p> <p>The performance of such a repository essentially depends on having enough memory. Having this memory in duplicate is just added cost. What we can do instead is have all copies store the whole partition but when routing queries, apply range partitioning on top of the basic hash partitioning. If one partition stores IDs 64K - 128K, the next partition 128K - 192K, and so forth, and all partitions are stored in two full copies, we can route reads to the first 32K IDs to the first copy and reads to the second 32K IDs to the second copy. In this way, the copies will keep different working sets. The RAM is used to full advantage.</p> <p>Of course, if there is a failure, then the working set will degrade, but if this is not often and not for long, this can be quite tolerable. The alternate expense is buying twice as much RAM, likely meaning twice as many servers. This workload is memory intensive, thus servers should have the maximum memory they can have without going to parts that are so expensive one gets a new server for the price of doubling memory.</p> <h2>Background Bulk Processing</h2> <p>When loading data, the system is online in principle, but query response can be quite bad. A large <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x19fd9c18">RDF</a> load will involve most memory and queries will miss the cache. The load will further keep most disks busy, so response is not good. This is the case as soon as a server's partition of the database is four times the size of RAM or greater. Whether the work is bulk-load or bulk-delete makes little difference.</p> <p>But if partitions are replicated, we can temporarily split the database so that the first copies serve queries and the second copies do the load. If the copies serving on line activities do some updates also, these updates will be committed on both copies. But the load will be committed on the second copy only. This is fully appropriate as long as the data are different. When the bulk load is done, the second copy of each partition will have the full up to date state, including changes that came in during the bulk load. The online activity can be now redirected to the second copies and the first copies can be overwritten in the background by the second copies, so as to again have all data in duplicate.</p> <p>Failures during such operations are not dangerous. If the copies doing the bulk load fail, the bulk load will have to be restarted. If the front end copies fail, the front end load goes to the copies doing the bulk load. Response times will be bad until the bulk load is stopped, but no data is lost.</p> <p>This technique applies to all data intensive background tasks â calculation of <a href="http://dbpedia.org/resource/Entity" id="link-id0x20b7a568">entity</a> search ranks, data cleansing, consistency checking, and so on. If two copies are needed to keep up with the online load, then data can be kept just as well in three copies instead of two. This method applies to any data-warehouse-style workload which must coexist with online access and occasional low volume updating.</p> <h2>Configurations of Redundancy</h2> <p>Right now, we can declare that two or more server processes in a cluster form a group. All data managed by one member of the group is stored by all others. The members of the group are interchangeable. Thus, if there is four-servers-worth of data, then there will be a minimum of eight servers. Each of these servers will have one server process per core. The first hardware failure will not affect operations. For the second failure, there is a 1/7 chance that it stops the whole system, if it falls on the server whose pair is down. If groups consist of three servers, for a total of 12, the two first failures are guaranteed not to interrupt operations; for the third, there is a 1/10 chance that it will.</p> <p>We note that for big databases, as said before, the RAM cache capacity is the sum of all the servers' RAM when in normal operation.</p> <p>There are other, more dynamic ways of splitting data among servers, so that partitions migrate between servers and spawn extra copies of themselves if not enough copies are online. The Google File System (GFS) does something of this sort at the file system level; Amazon's Dynamo does something similar at the database level. The analogies are not exact, though.</p> <p>If data is partitioned in this manner, for example into 1K slices, each in duplicate, with the rule that the two duplicates will not be on the same physical server, the first failure will not break operations but the second probably will. Without extra logic, there is a probability that the partitions formerly hosted by the failed server have their second copies randomly spread over the remaining servers. This scheme equalizes load better but is less resilient.</p> <h2>Maintenance and Continuity</h2> <p>Databases may benefit from defragmentation, rebalancing of indices, and so on. While these are possible online, by definition they affect the working set and make response times quite bad as soon as the database is significantly larger than RAM. With duplicate copies, the problem is largely solved. Also, software version changes need not involve downtime.</p> <h2>Present Status</h2> <p>The basics of replicated partitions are operational. The items to finalize are about system administration procedures and automatic synchronization of recovering copies. This must be automatic because if it is not, the operator will find a way to forget something or do some steps in the wrong order. This also requires a management view that shows what the different processes are doing and whether something is hung or failing repeatedly. All this is for the recovery part; taking failed partitions offline is easy.</p>
An Update on Virtuoso Development
http://www.openlinksw.com/blog/vdb/blog/?date=2009-03-05#1529
2009-03-05T10:23:49Z
2009-03-05T09:58:16-05:00
<p>It is time for an update on <a href="http://virtuoso.openlinksw.com" id="link-id0x151e89e8">Virtuoso</a> developments.</p> <p>We continue enhancing our hosting of the <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1464e168">Linked Open Data</a> (<a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x151e7f38">LOD</a>) cloud at <a href="http://lod.openlinksw.com" id="link-id11ac2448">http://lod.openlinksw.com</a>.</p> <p>We have now added result ranking for both text and URIs. Text hit scores are based on word frequency and proximity; <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x14f9fae0">URI</a> scores are based on link density.</p> <p>We calculate each URI's rank by adding up references and weighing these by the score of the referrer. This is like in web search. Each iteration of the ranking will join every referred to each of its referrers. We do about 1.2 million such joins per second, across partitions, over 2.2 billion triples and 400M distinct subjects without any great optimization, just using <a href="http://dbpedia.org/resource/SQL" id="link-id0xaa36a458">SQL</a> stored procedures and partitioned function calls. This is a sort of SQL map-reduce. We would do over twice as fast if it were all in <a href="http://dbpedia.org/resource/C%2B%2B" id="link-id0x1571e270">C</a> but this is adequate for now. The more interesting bit will be tuning the scoring based on what type of link we have. This is what the web search engines cannot do as well, since document links are untyped.</p> <p>We are moving toward a decent user interface for the LOD hosting, including offering ready-made domain-specific queries, e.g., biomedical.</p> <p>Things like "URI finding with autocomplete" are done and just have to be put online.</p> <p>With <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x14325e08">linked data</a>, there is the whole question of identifier choice. We will have a special page just for this. There we show reference statistics, synonyms declared by <code><a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x638b3900">owl</a>:sameAs</code>, synonyms determined by shared property values, etc. In this way we become a terminology lookup service.</p> <p>Copies of the LOD cluster system are available for evaluators, on a case by case basis. We will make this publicly available on EC2 also in not too long.</p> <p>Otherwise, we continue working on productization, primarily things like reliability and recovery. One exercise is running <a href="http://dbpedia.org/resource/TPC-C" id="link-id0x144d00f0">TPC-C</a> with intentionally stupid partitioning, so that almost all joins and deadlocks are distributed. Then we simulate a cluster interconnect that drops messages now and then, sometimes kill server processes, and still keep full ACID properties. Cloud capable, also in bad weather.</p> <p>The open source release of Virtuoso 6 (no cluster) is basically ready to go, mostly this is a question of logistics.</p> <p>I will talk about these things in greater individual detail next week.</p>
Facets and Large Ontologies of the LOD Cloud
http://www.openlinksw.com/blog/vdb/blog/?date=2009-02-16#1527
2009-02-16T11:21:05Z
2009-02-17T16:24:34.000004-05:00
<p>We have just submitted <a href="http://www.openlinksw.com/weblog/oerling/lodw.pdf" id="link-id13d9bc68">this paper</a> to the WWW09 <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x51c2fd00">Linked Open Data</a> Workshop.</p> <p>The thing is intermittently live with both <a href="http://dbpedia.org/resource/DBpedia" id="link-id0xa16f87a0">Dbpedia</a> on one instance and a <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1a8212d8">LOD</a> Cloud <a href="http://dbpedia.org/resource/Data" id="link-id0xc1097a8">data</a> collection of about 2 billion triples on another. We will give out the links once we have tested a bit more.</p> <p>The present activity is all about testing <a href="http://virtuoso.openlinksw.com" id="link-id0x17d64f90">Virtuoso</a> 6 for release, cluster and otherwise.</p>
Faceted Search: Unlimited Data in Interactive Time
http://www.openlinksw.com/blog/vdb/blog/?date=2009-01-09#1516
2009-01-09T22:03:11Z
2009-01-09T17:15:39-05:00
<p>Why not see the whole world of <a href="http://dbpedia.org/resource/Data" id="link-id0xc3f6b38">data</a> as facets? Well, we'd like to, but there is the feeling that this is not practical.</p> <p>The old problem has been that it is not really practical to pre-compute counts of everything for all possible combinations of search conditions and counting/grouping/sorting. The actual matches take time.</p> <p>Well, neither is in fact necessary. When there are large numbers of items matching the conditions, counting them can take time but then this is the beginning of the search, and the user is not even likely to look very closely at the counts. It is enough to see that there are many of one and few of another. If the user already knows the precise predicate or class to look for, then the top-level faceted view is not even needed. The faceted view for guiding search and precise analytics are two different problems.</p> <p>There are client-side faceted views like Exhibit or our own <a href="http://ode.openlinksw.com/" id="link-id0x1bc1cfe0">ODE</a>. The problem with these is that there are a few orders of magnitude difference between the actual database size and what fits on the user agent. This is compounded by the fact that one does not know what to cache on the user agent because of the open nature of the data web. If this were about a fixed workflow, then a good guess would be possible â but we are talking about the data web, the very soul of serendipity and unexpected discovery.</p> <p>So we made a web service that will do faceted search on arbitrary <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xbb62170">RDF</a>. If it does not get complete results within a timeout, it will return what it has counted so far, using <a href="http://virtuoso.openlinksw.com" id="link-id0xb122b00">Virtuoso</a>'s <a href="http://www.openlinksw.com/weblog/oerling/?id=1494" id="link-id117b0df0"><b>Anytime</b></a> feature. Looking for subjects with some specific combination of properties is however a bit limited, so this will also do <code>JOINs</code>. Many features are one or two <code>JOINs</code> away; take geographical locations or social networks, for example.</p> <p>Yet a faceted search should be point-and-click, and should not involve a full query construction. We put the compromise at starting with full text or property or class, then navigating down properties or classes, to arbitrary depth, tree-wise. At each step, one can see the matching instances or their classes or properties, all with counts, faceted-style.</p> <p>This is good enough for queries like 'what do Harry Potter fans also like' or 'who are the authors of articles tagged <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0xbee32d8">semantic web</a> and machine learning and published in 2008'. For complex grouping, sub-queries, arithmetic or such, one must write the actual query.</p> <p>But one can begin with facets, and then continue refining the query by hand since the service also returns <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xbcc9f38">SPARQL</a> text. We made a small web interface on top of the service with all logic server side. This proves that the web service is usable and that an interface with no AJAX, and no problems with browser interoperability or such, is possible and easy. Also, the problem of syncing between a user-agent-based store and a database is entirely gone.</p> <p>If we are working with a known data structure, the user interface should choose the display by the data type and offer links to related reports. This is all easy to build as web pages or AJAX. We show how the generic interface is done in Virtuoso PL, and you can adapt that or rewrite it in <a href="http://dbpedia.org/resource/PHP" id="link-id0xcdbe268">PHP</a>, Java, JavaScript, or anything else, to accommodate use-case specific navigation needs such as data format.</p> <p>The web service takes an <a href="http://dbpedia.org/resource/XML" id="link-id0xc019c08">XML</a> representation of the search, which is more restricted and easier to process by machine than the SPARQL syntax. The web service returns the results, the SPARQL query it generated, whether the results are complete or not, and some resource use statistics.</p> <p>The source of the PL functions, Web Service and Virtuoso Server Page (HTML UI) will be available as part of Virtuoso 6.0 and higher. A Programmer's Guide will be available as part of the standard Virtuoso Documentation collection, including the Virtuoso Open Source Edition Website.</p>
Linked Data & The Year 2009 (updated)
http://www.openlinksw.com/blog/vdb/blog/?date=2009-01-02#1511
2009-01-02T16:17:06Z
2009-01-02T13:26:42.000003-05:00
<p>As is fitting for the season, I will editorialize a bit about what has gone before and what is to come.</p> <p> <a href="http://www.w3.org/People/Berners-Lee/card#i" id="link-id1119f250">Sir Tim</a> said it at WWW08 in <a href="http://www2008.org/" id="link-id0x1dcb93a0">Beijing</a> â <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x13a3efb8">linked data</a> and the linked data <a href="http://dbpedia.org/resource/Giant_Global_Graph" id="link-id0x13a44cd0">web</a> is the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x10d25788">semantic web</a> and the Web done right.</p> <p>The grail of <i>ad hoc</i> analytics on infinite <a href="http://dbpedia.org/resource/Data" id="link-id0xa201d518">data</a> has lost none of its appeal. We have seen fresh evidence of this in the realm of data warehousing products, as well as storage in general.</p> <p>The benefits of a data model more abstract than the relational are being increasingly appreciated also outside the data web circles. Microsoft's <a href="http://dbpedia.org/resource/Entity" id="link-id0x12fa4e40">Entity</a> Frameworks technology is an example. Agility has been a buzzword for a long time. Everything should be offered in a service based business model and should interoperate and integrate with everything else â business needs first; schema last.</p> <p>Not to forget that when money is tight, reuse of existing assets and paying on a usage basis are naturally emphasized. <a href="http://dbpedia.org/resource/Information" id="link-id0x175b32e8">Information</a>, as the asset it is, is none the less important, on the contrary. But even with information, value should be realized economically, which, among other things, entails not reinventing the wheel.</p> <p>It is against this backdrop that this year will play out.</p> <p>As concerns research, I will <a href="http://www.openlinksw.com/weblog/oerling/?id=1374" id="link-id1151b128">again quote</a> <a href="http://www.ibiblio.org/hhalpin/#" id="link-id141cb740">Harry Halpin</a> at <a href="http://www.eswc2008.org/" id="link-id0x18a8a858">ESWC 2008</a>: "Men will fight in a war, and even lose a war, for what they believe just. And it may come to pass that later, even though the war were lost, the things then fought for will emerge under another name and establish themselves as the prevailing reality" [or words to this effect].</p> <p>Something like the data web, and even the semantic web, will happen. Harry's question was whether this would be the descendant of what is today called semantic web research.</p> <p>I heard in conversation about a project for making a very large metadata store. I also heard that the makers did not particularly insist on this being <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x3c39ed80">RDF</a>-based, though.</p> <p>Why should such a thing be RDF-based? If it is already accepted that there will be <i>ad hoc</i> schema and that queries ought to be able to view the data from all angles, not be limited by having indices one way and not another way, then why not RDF?</p> <p>The justification of RDF is in reusing and linking-to data and terminology out there. Another justification is that by using an RDF store, one is spared a lot of work and tons of compromises which attend making an <a href="http://dbpedia.org/resource/Entity-attribute-value_model" id="link-id0x14a77880">entity</a>-attribute-value (<a href="http://dbpedia.org/resource/Entity-attribute-value_model" id="link-id0x5f978e88">EAV</a>, i.e., triple) store on a generic <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x391bdcd8">RDBMS</a>. The sem-web world has been there, trust me. We came out well because we put all inside the RDBMS, lowest level, which you can't do unless you own the RDBMS. Source access is not enough; you also need the <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x138a3a00">knowledge</a>.</p> <p>Technicalities aside, the question is one of proprietary vs. standards-based. This is not only so with software components, where standards have consistently demonstrated benefits, but now also with the data. <a href="http://www.zemanta.com/" id="link-id0x5f92cb38">Zemanta</a> and <a href="http://www.opencalais.com/" id="link-id0x139c3200">OpenCalais</a> serving <a href="http://dbpedia.org/resource/DBpedia" id="link-id0x1731dc78">DBpedia</a> URIs are examples. Even in entirely closed applications, there is benefit in reusing open vocabularies and identifiers: One does not need to create a secret language for writing a secret memo.</p> <p>Where data is a carrier of value, its value is enhanced by it being easy to repurpose (i.e., standard vocabularies) and to discover (i.e., data set metadata). As on the web, so on the enterprise <a href="http://dbpedia.org/resource/Intranet" id="link-id0x1324ada8">intranet</a>. In this lies the strength of RDF as opposed to proprietary flexible database schemes. This is a qualitative distinction.</p> <p align="center"> <a href="http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/LinkingOpenData" id="link-id117178a8"><img src="http://www.openlinksw.com/images/logos/LoDLogo.gif" alt="Linking Open Data project logo" /> </a> <br /> <a href="http://dbpedia.org/resource/In_hoc_signo_vinces" id="link-id115f47e8"><i>In hoc signo vinces.</i> </a> </p> <p>In this light, we welcome the <a href="http://semanticweb.org/wiki/VoiD" id="link-id0x67cf560">voiD</a> (<a href="http://semanticweb.org/wiki/VoiD" id="link-id0x1898c908">VOcabulary of Interlinked Data</a>), which is the first promise of making federatable data discoverable. Now that there is a point of focus for these efforts, the needed expressivity will no doubt accrete around the voiD core.</p> <p>For data as a service, we clearly see the value of open terminologies as prerequisites for service interchangeability, i.e., creating a marketplace. <a href="http://dbpedia.org/resource/XML" id="link-id0x1588d6a8">XML</a> is for the transaction; RDF is for the discovery, query, and analytics. As with databases in general, first there was the transaction; then there was the query. Same here. For monetizing the query, there are models ranging from renting data sets and server capacity in the clouds to hosted services where one pays for processing past a certain quota. For the hosted case, we just removed a major barrier to offering unlimited query against unlimited data when we completed the <a href="http://www.openlinksw.com/weblog/oerling/?id=1374" id="link-id110b8668">Virtuoso Anytime</a> feature. With this, the user gets what is found within a set time, which is already something, and in case of needing more, one can pay for the usage. Of course, we do not forget advertising. When data has explicit semantics, contextuality is better than with keywords.</p> <p>For these visions to materialize on top of the linked data platform, linked data must join the world of data. This means messaging that is geared towards the database public. They know the problem, but the RDF proposition is still not well enough understood for it to connect.</p> <p>For the relational IT world, we offer passage to the data web and its promise of integration through RDF mapping. We are also bringing out new Microsoft Entity <a href="http://dbpedia.org/resource/ADO.NET_Entity_Framework" id="link-id0x13a50fd8">Framework</a> components. This goes in the direction of defining a unified database frontier with RDF and non-RDF entity models side by side.</p> <p>For <a href="http://www.openlinksw.com/dataspace/organization/openlink#this" id="link-id0x1d2ea7f0">OpenLink Software</a>, 2008 was about developing technology for scale, RDF as well as generic relational. We did show a tiny preview with the <a href="http://challenge.semanticweb.org/" id="link-id0x658fbc8">Billion Triples Challenge</a> demo. Now we are set to come out with the real thing, featuring, among other things, faceted search at the billion triple scale. We <a href="http://www.openlinksw.com/blog/kidehen@openlinksw.com/blog/?id=1489" id="link-id150c6090">started offering ready-to-go Virtuoso-hosted linked open data sets</a> on Amazon EC2 in December. Now we continue doing this based on our next-generation server, as well as make Virtuoso 6 Cluster commercially available. Technical specifics are amply discussed on this <a href="http://dbpedia.org/resource/Blog" id="link-id0x1424ec20">blog</a>. There are still some new technology things to be developed this year; first among these are strong <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x14b8ca88">SPARQL</a> federation, and on-the-fly resizing of server clusters. On the research partnerships side, we have an EU grant for working with the OntoWiki project from the University of Leipzig, and we are partners in DERI's <a href="https://lion.deri.ie/" id="link-id115c02f8">LÃon project</a>. These will provide platforms for further demonstrating the "web" in data web, as in web-scale smart databasing.</p> <p>2009 will see change through scale. The things that exist will start interconnecting and there will be emergent value. Deployments will be larger and scale will be readily available through a services model or by installation at one's own facilities. We may see the start of Search becoming Find, like <a href="http://myopenlink.net/dataspace/person/kidehen#this" id="link-id14e43050">Kingsley</a> says, meaning semantics of data guiding search. Entity extraction will multiply data volumes and bring parts of the data web to real time.</p> <p>Exciting 2009 to all.</p>
Virtuoso 6 FAQ directory
http://www.openlinksw.com/blog/vdb/blog/?date=2008-12-18#1507
2008-12-18T15:46:18Z
2008-12-22T14:30:07.000004-05:00
<p>We have received various inquiries on high-end metadata stores. I will here go through some salient questions. The requested features include:</p> <ul> <li>Scaling to trillions of triples</li> <li>Running on clusters of commodity servers</li> <li>Running in federated environments, possibly over wide area networks</li> <li>Built-in inference</li> <li>Transactions</li> <li>Security</li> <li>Support for extra triple level metadata, such as security attributes</li> </ul> <p>Q: What is the storage cost per triple? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#StorageCostPerTriple" id="link-id147f61e8">answer</a> </p> <p>Q: What is the cost to insert a triple? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#TripleInsertionCost" id="link-id112e2488">answer</a> </p> <p>Q: What is the cost to delete a triple? (For the insertion itself, as well as for updating any indices) <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#TripleDeletionCost" id="link-id11728528">answer</a> </p> <p>Q: What is the cost to search on a given property? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#PropertySearchCost" id="link-id1586e360">answer</a> </p> <p>Q: What <a href="http://dbpedia.org/resource/Data" id="link-id14688e38">data</a> types are supported? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#SupportedDataTypes" id="link-id1593dbf0">answer</a> </p> <p>Q: What inferencing is supported? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#SupportedInferencing" id="link-id112f3248">answer</a> </p> <p>Q: Is the inferencing dynamic or is an extra step required before inferencing can be used? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#InferencingDynamism" id="link-id1477e2e0">answer</a> </p> <p>Q: Do you support <a href="http://dbpedia.org/resource/Full_text_search" id="link-id1177b198">full text search</a>? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#FullTextSearchSupport" id="link-id1543b170">answer</a> </p> <p>Q: What programming interfaces are supported? Do you support standard <a href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id14bb69c0">SPARQL protocol</a>? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#SupportedProgrammingInterfaces" id="link-id14d4eb18">answer</a> </p> <p>Q: How can data be partitioned across multiple servers? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#MultipleServerDataPartitioning" id="link-id13722e00">answer</a> </p> <p>Q: How many triples can a single server handle? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#SingleServerTripleLimits" id="link-id14046e58">answer</a> </p> <p>Q: What is the performance impact of going from the billion to the trillion triples? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#PerformanceImpactBillionToTrillion" id="link-id113cfc10">answer</a> </p> <p>Q: Do you support additional metadata for triples, such as timestamps, security tags etc? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#TripleMetadataSupport" id="link-id14c75fa8">answer</a> </p> <p>Q: Should we use <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id11342010">RDF</a> for our large metadata store? What are the alternatives? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#LargeMetadataStoreFormat" id="link-id1478db38">answer</a> </p> <p>Q: How multithreaded is <a href="http://virtuoso.openlinksw.com" id="link-id1651d028">Virtuoso</a>? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#VirtuosoMultiThreading" id="link-id152ad310">answer</a> </p> <p>Q: Can multiple servers run off a single shared disk database? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#MultipleServersOneDiskDatabase" id="link-id14d9d528">answer</a> </p> <p>Q: Can Virtuoso run on a SAN? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#VirtuosoOnSAN" id="link-id111b55d0">answer</a> </p> <p>Q: How does Virtuoso join across partitions? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#CrossPartitionJoins" id="link-id11094db8">answer</a> </p> <p>Q: Does Virtuoso support federated triple stores? If there are multiple <a href="http://dbpedia.org/resource/SPARQL" id="link-id19156b48">SPARQL</a> end points, can Virtuoso be used to do queries joining between these? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#FederatedTripleStoresAndQueries" id="link-id15447ef8">answer</a> </p> <p>Q: How many servers can a cluster contain? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#ClusterServerLimit" id="link-id125fe0d0">answer</a> </p> <p>Q: How do I reconfigure a cluster, adding and removing machines, etc? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#ClusterReconfiguration" id="link-id1150c448">answer</a> </p> <p>Q: How will Virtuoso handle regional clusters? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#RegionalClustering" id="link-id1596ca48">answer</a> </p> <p>Q: Is there a mechanism for terminating long running queries? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#TerminatingLongRunningQueries" id="link-id116bbd60">answer</a> </p> <p>Q: Can the user be asynchronously notified when a long running query terminates? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#AsynchNotificationOfQueryTermination" id="link-id15a59a50">answer</a> </p> <p>Q: How many concurrent queries can Virtuoso handle? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#ConcurrentQueryLimits" id="link-id110a8c00">answer</a> </p> <p>Q: What is the relative performance of SPARQL queries vs. native relational queries <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#RelativePerformanceSparqlVsSql" id="link-id110914f8">answer</a> </p> <p>Q: Does Virtuoso support property tables? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#PropertyTableSupport" id="link-id1581f8c8">answer</a> </p> <p>Q: What performance metrics does Virtuoso offer? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#PerformanceMetricSupport" id="link-id14e92300">answer</a> </p> <p>Q: What support do you provide for concurrency/multithreading operation? Is your interface thread-safe? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#ConcurrencyAndThreadSafety" id="link-id15964b80">answer</a> </p> <p>Q: What level of ACID properties are supported? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#AcidComplianceLevel" id="link-id11035ac0">answer</a> </p> <p>Q: Do you provide the ability to atomically add a set of triples, where either all are added or none are added? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#AtomicTripleInsertion" id="link-id15290e68">answer</a> </p> <p>Q: Do you provide the ability to add a set of triples, respecting the isolation property (so concurrent accessors either see none of the triple values, or all of them)? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#IsolationDuringInsertion" id="link-id15855df0">answer</a> </p> <p>Q: What is the time to start a database, create/open a graph? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#StartupTimes" id="link-id14227f40">answer</a> </p> <p>Q: What sort of security features are built into Virtuoso? <a href="http://virtuoso.openlinksw.com/Whitepapers/html/Virt6FAQ.html#BuiltInSecurity" id="link-id11927810">answer</a> </p>
Virtuoso RDF: A Getting Started Guide for the Developer
http://www.openlinksw.com/blog/vdb/blog/?date=2008-12-17#1505
2008-12-17T12:31:34Z
2008-12-17T12:41:27.000006-05:00
<p>It is a long standing promise of mine to dispel the false impression that using <a href="http://virtuoso.openlinksw.com/" id="link-id113506d0">Virtuoso</a> to work with <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id115d9528">RDF</a> is complicated.</p> <p>The purpose of this presentation is to show a programmer how to put RDF into Virtuoso and how to query it. This is done programmatically, with no confusing user interfaces.</p> <p>You should have a Virtuoso Open Source tree built and installed. We will look at the LUBM benchmark demo that comes with the package. All you need is a Unix shell. Running the shell under emacs (<code>m-x shell</code>) is the best. But the open source <code>isql</code> utility should have command line editing also. The emacs shell is however convenient for cutting and pasting things between shell and files.</p> <p>To get started, cd into <code>binsrc/tests/lubm</code>.</p> <p>To verify that this works, you can do </p> <blockquote> <pre>./test_server.sh virtuoso-t</pre></blockquote> <p>This will test the server with the LUBM queries. This should report 45 tests passed. After this we will do the tests step-by-step.</p> <h2>Loading the <a href="http://dbpedia.org/resource/Data" id="link-id10f7bd90">Data</a> </h2> <p>The file <code>lubm-load.sql</code> contains the commands for loading the LUBM single university qualification database.</p> <p>The data files themselves are in <code>lubm_8000</code>, 15 files in RDFXML.</p> <p>There is also a little ontology called <code>inf.nt</code>. This declares the subclass and subproperty relations used in the benchmark.</p> <p>So now let's go through this procedure.</p> <p>Start the server:</p> <blockquote> <pre>$ virtuoso-t -f & </pre></blockquote> <p>This starts the server in foreground mode, and puts it in the background of the shell.</p> <p>Now we connect to it with the isql utility.</p> <blockquote> <pre>$ isql 1111 dba dba </pre></blockquote> <p>This gives a <code>SQL></code> prompt. The default username and password are both <code>dba</code>.</p> <p>When a command is <a href="http://dbpedia.org/resource/SQL" id="link-id1176ce70">SQL</a>, it is entered directly. If it is <a href="http://dbpedia.org/resource/SPARQL" id="link-id156df468">SPARQL</a>, it is prefixed with the keyword <code>sparql</code>. This is how all the SQL clients work. Any SQL client, such as any <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id152d0a00">ODBC</a> or <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id157ad6a0">JDBC</a> application, can use SPARQL if the SQL string starts with this keyword.</p> <p>The <code>lubm-load.sql</code> file is quite self-explanatory. It begins with defining an SQL procedure that calls the RDF/XML load function, <code>DB..RDF_LOAD_RDFXML</code>, for each file in a directory.</p> <p>Next it calls this function for the <code>lubm_8000</code> directory under the server's working directory.</p> <blockquote> <pre>sparql CLEAR GRAPH <lubm>; sparql CLEAR GRAPH <inf>; load_lubm ( server_root() || '/lubm_8000/' ); </pre></blockquote> <p>Then it verifies that the right number of triples is found in the <lubm> graph.</p> <blockquote> <pre>sparql SELECT COUNT(*) FROM <lubm> WHERE { ?x ?y ?z } ; </pre></blockquote> <p>The echo commands below this are interpreted by the isql utility, and produce output to show whether the test was passed. They can be ignored for now.</p> <p>Then it adds some implied <code>subOrganizationOf</code> triples. This is part of setting up the LUBM test database.</p> <blockquote> <pre>sparql PREFIX ub: <http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl#> INSERT INTO GRAPH <lubm> { ?x ub:subOrganizationOf ?z } FROM <lubm> WHERE { ?x ub:subOrganizationOf ?y . ?y ub:subOrganizationOf ?z . }; </pre></blockquote> <p>Then it loads the ontology file, <code>inf.nt</code>, using the Turtle load function, <code>DB.DBA.TTLP</code>. The arguments of the function are the text to load, the default namespace prefix, and the <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id15835550">URI</a> of the target graph.</p> <blockquote> <pre>DB.DBA.TTLP ( file_to_string ( 'inf.nt' ), 'http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl', 'inf' ) ; sparql SELECT COUNT(*) FROM <inf> WHERE { ?x ?y ?z } ; </pre></blockquote> <p>Then we declare that the triples in the <code><inf></code> graph can be used for inference at run time. To enable this, a SPARQL query will declare that it uses the <code>'inft'</code> rule set. Otherwise this has no effect.</p> <blockquote> <pre>rdfs_rule_set ('inft', 'inf'); </pre></blockquote> <p>This is just a log checkpoint to finalize the work and truncate the transaction log. The server would also eventually do this in its own time.</p> <blockquote> <pre>checkpoint; </pre></blockquote> <p>Now we are ready for querying.</p> <h2>Querying the Data</h2> <p>The queries are given in 3 different versions: The first file, <code>lubm.sql</code>, has the queries with most inference open coded as <code>UNIONs</code>. The second file, <code>lubm-inf.sql</code>, has the inference performed at run time using the ontology <a href="http://dbpedia.org/resource/Information" id="link-id1109faf0">information</a> in the <code><inf></code> graph we just loaded. The last, <code>lubm-phys.sql</code>, relies on having the entailed triples physically present in the <code><lubm></code> graph. These entailed triples are inserted by the SPARUL commands in the <code>lubm-cp.sql</code> file.</p> <p>If you wish to run all the commands in a SQL file, you can type <code>load <filename>;</code> (e.g., <code>load lubm-cp.sql;</code>) at the <code>SQL></code> prompt. If you wish to try individual statements, you can paste them to the command line.</p> <p>For example: </p> <blockquote> <pre>SQL> sparql PREFIX ub: <http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl#> SELECT * FROM <lubm> WHERE { ?x a ub:Publication . ?x ub:publicationAuthor <http://www.Department0.University0.edu/AssistantProfessor0> }; VARCHAR _______________________________________________________________________ http://www.Department0.University0.edu/AssistantProfessor0/Publication0 http://www.Department0.University0.edu/AssistantProfessor0/Publication1 http://www.Department0.University0.edu/AssistantProfessor0/Publication2 http://www.Department0.University0.edu/AssistantProfessor0/Publication3 http://www.Department0.University0.edu/AssistantProfessor0/Publication4 http://www.Department0.University0.edu/AssistantProfessor0/Publication5 6 Rows. -- 4 msec. </pre></blockquote> <p>To stop the server, simply type <code>shutdown;</code> at the <code>SQL></code> prompt.</p> <p>If you wish to use a <a href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id11384668">SPARQL protocol</a> end point, just enable the HTTP listener. This is done by adding a stanza like â</p> <blockquote> <pre>[HTTPServer] ServerPort = 8421 ServerRoot = . ServerThreads = 2 </pre></blockquote> <p>â to the end of the <code>virtuoso.ini</code> file in the <code>lubm</code> directory. Then shutdown and restart (type <code>shutdown;</code> at the <code>SQL></code> prompt and then <code>virtuoso-t -f &</code> at the shell prompt).</p> <p>Now you can connect to the end point with a web browser. The <a href="http://dbpedia.org/resource/Uniform_Resource_Locator" id="link-id113d02d8">URL</a> is <code>http://localhost:8421/sparql</code>. Without parameters, this will show a human readable form. With parameters, this will execute SPARQL.</p> <p>We have shown how to load and query RDF with Virtuoso using the most basic SQL tools. Next you can access RDF from, for example, <a href="http://dbpedia.org/resource/PHP" id="link-id142d0ba0">PHP</a>, using the PHP ODBC interface.</p> <p>To see how to use <a href="http://jena.sourceforge.net/" id="link-id117074f0">Jena</a> or <a href="http://sourceforge.net/projects/sesame/" id="link-id1103c9b0">Sesame</a> with Virtuoso, look at <a href="http://docs.openlinksw.com/virtuoso/rdfnativestorageproviders.html" id="link-id15488ce8">Native RDF Storage Providers</a>. To see how RDF data types are supported, see <a href="http://docs.openlinksw.com/virtuoso/VirtuosoDriverJDBC.html#jdbcrdf" id="link-id15784a40">Extension datatype for RDF</a> </p> <p>To work with large volumes of data, you must add memory to the configuration file and use the row-autocommit mode, i.e., do <code>log_enable (2);</code> before the load command. Otherwise Virtuoso will do the entire load as a single transaction, and will run out of rollback space. See <a href="http://docs.openlinksw.com/virtuoso/" id="link-id111410f0">documentation</a> for more.</p>
See the Lite: Embeddable/Background Virtuoso starts at 25MB
http://www.openlinksw.com/blog/vdb/blog/?date=2008-12-17#1503
2008-12-17T09:34:12Z
2008-12-17T12:03:49-05:00
<p>We have received many requests for an embeddable-scale <a href="http://virtuoso.openlinksw.com" id="link-id0x1cd69650">Virtuoso</a>. In response to this, we have added a Lite mode, where the initial size of a server process is a tiny fraction of what the initial size would be with default settings. With 2MB of disk cache buffers (ini file setting, <code>NumberOfBuffers = 256</code>), the process size stays under 30MB on 32-bit Linux.</p> <p>The value of this is that one can now have <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1ce89340">RDF</a> and full text indexing on the desktop without running a Java VM or any other memory-intensive software. And of course, all of <a href="http://dbpedia.org/resource/SQL" id="link-id0x1cfc9288">SQL</a> (transactions, stored procedures, etc.) is in the same embeddably-sized container.</p> <p>The Lite executable is a full Virtuoso executable; the Lite mode is controlled by a switch in the configuration file. The executable size is about 10MB for 32-bit Linux. A database created in the Lite mode will be converted into a fully-featured database (tables and indexes are added, among other things) if the server is started with the Lite setting "off"; functionality can be reverted to Lite mode, though it will now consume somewhat more memory, etc.</p> <p>Lite mode offers full SQL and <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1c511da8">SPARQL</a>/SPARUL (via SPASQL), but disables all <a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x1dac1950">HTTP</a>-based services (WebDAV, application hosting, etc.). Clients can still use all typical database access mechanisms (i.e., <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0xb19a488">ODBC</a>, <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0x1d93ee40">JDBC</a>, OLE-DB, <a href="http://dbpedia.org/resource/ADO.NET" id="link-id0x1ce391c0">ADO</a>.<a href="http://dbpedia.org/resource/.NET_Framework" id="link-id0xacf1168">NET</a>, and XMLA) to connect, including the <a href="http://jena.sourceforge.net/" id="link-id0xaaf5b58">Jena</a> and <a href="http://sourceforge.net/projects/sesame/" id="link-id0x1b1e4328">Sesame</a> frameworks for RDF. ODBC now offers full support of RDF <a href="http://dbpedia.org/resource/Data" id="link-id0x1cfc9f78">data</a> types for <a href="http://dbpedia.org/resource/C%2B%2B" id="link-id0xa6059d8">C</a>-based clients. A Redland-compatible API also exists, for use with Redland v1.0.8 and later. </p> <p>Especially for embedded use, we now allow restricting the listener to be a Unix socket, which allows client connections only from the localhost.</p> <p>Shipping an embedded Virtuoso is easy. It just takes one executable and one configuration file. Performance is generally comparable to "normal" mode, except that Lite will be somewhat less scalable on multicore systems.</p> <p>The Lite mode will be included in the next Virtuoso 5 Open Source release.</p>
"E Pluribus Unum", or "Inversely Functional Identity", or "Smooshing Without the Stickiness" (re-updated)
http://www.openlinksw.com/blog/vdb/blog/?date=2008-12-16#1499
2008-12-16T14:14:43Z
2008-12-16T15:01:36.000003-05:00
<p>What a terrible word, smooshing... I have understood it to mean that when you have two names for one thing, you give each all the attributes of the other. This smooshes them together, makes them interchangeable.</p> <p>This is complex, so I will begin with the point and the interested may read on for the details and implications. Starting with soon to be released version 6, <a href="http://virtuoso.openlinksw.com" id="link-id15718cb8">Virtuoso</a> allows you to say that two things, if they share a uniquely identifying property, are the same. Examples of uniquely identifying properties would be a book's ISBN number, or a person's social security plus full name. In relational language this is a <i>unique key</i>, and in <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id145ed998">RDF</a> parlance, an <i>inverse functional property</i>.</p> <p>In most systems, such problems are dealt with as a preprocessing step before querying. For example, all the items that are considered the same will get the same properties or at load time all identifiers will be normalized according to some application rules. This is good if the rules are clear and understood. This is so in closed situations, where things tend to have standard identifiers to begin with. But on the open web this is not so clear cut.</p> <p>In this post, we show how to do these things <i>ad hoc</i>, without materializing anything. At the end, we also show how to materialize identity and what the consequences of this are with open web <a href="http://dbpedia.org/resource/Data" id="link-id11726358">data</a>. We use real live web crawls from the <a href="http://challenge.semanticweb.org/" id="link-id14f40448">Billion Triples Challenge</a> data set.</p> <p>On the <a href="http://dbpedia.org/resource/Linked_Data" id="link-id156e2b10">linked data</a> <a href="http://dbpedia.org/resource/Giant_Global_Graph" id="link-id1106ce08">web</a>, there are independently arising descriptions of the same thing and thus arises the need to smoosh, if these are to be somehow integrated. But this is only the beginning of the problems.</p> <p>To address these, we have added the option of specifying that some property will be considered inversely functional in a query. This is done at run time and the property does not really have to be inversely functional in the pure sense. <code>foaf:name</code> will do for an example. This simply means that for purposes of the query concerned, two subjects which have at least one <code>foaf:name</code> in common are considered the same. In this way, we can join between FOAF files. With the same database, a query about music preferences might consider having the same name as "same enough," but a query about criminal prosecution would obviously need to be more precise about sameness.</p> <p>Our ontology is defined like this:</p> <blockquote> <pre>-- Populate a named graph with the triples you want to use in query time inferencing<br /> ttlp ( ' @prefix foaf: <xmlns="http" xmlns.com="xmlns.com" foaf="foaf"> </> @prefix owl: <xmlns="http" www.w3.org="www.w3.org" owl="owl"> </> foaf:mbox_sha1sum a owl:InverseFunctionalProperty . foaf:name a owl:InverseFunctionalProperty . ', 'xx', 'b3sifp' );<br /> -- Declare that the graph contains an ontology for use in query time inferencing <br /> rdfs_rule_set ( 'http://example.com/rules/b3sifp#', 'b3sifp' ); </pre></blockquote> <p>Then use it:</p> <blockquote> <pre>sparql DEFINE input:inference "http://example.com/rules/b3sifp#" SELECT DISTINCT ?k ?f1 ?f2 WHERE { ?k foaf:name ?n . ?n bif:contains "'Kjetil Kjernsmo'" . ?k foaf:knows ?f1 . ?f1 foaf:knows ?f2 };<br /> VARCHAR VARCHAR VARCHAR ______________________________________ _______________________________________________ ______________________________<br /> http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/dajobe http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/net_twitter http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/amyvdh http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/pom http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/mattb http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/davorg http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/distobj http://www.kjetil.kjernsmo.net/foaf#me http://norman.walsh.name/knows/who/robin-berjon http://twitter.com/perigrin .... </pre></blockquote> <p>Without the inference, we get no matches. This is because the data in question has one graph per FOAF file, and blank nodes for persons. No graph references any person outside the ones in the graph. So if somebody is mentioned as known, then without the inference there is no way to get to what that person's FOAF file says, since the same individual will be a different blank node there. The declaration in the context named <code>b3sifp</code> just means that all things with a matching <code>foaf:name</code> or <code>foaf:mbox_sha1sum</code> are the same.</p> <p>Sameness means that two are the same for purposes of <code>DISTINCT</code> or <code>GROUP BY</code>, and if two are the same, then both have the <code>UNION</code> of all of the properties of both.</p> <p>If this were a naive smoosh, then the individuals would have all the same properties but would not be the same for <code>DISTINCT</code>.</p> <p>If we have complex application rules for determining whether individuals are the same, then one can materialize <code>owl:sameAs</code> triples and keep them in a separate graph. In this way, the original data is not contaminated and the materialized volume stays reasonable â nothing like the blow-up of duplicating properties across instances.</p> <p>The pro-smoosh argument is that if every duplicate makes exactly the same statements, then there is no great blow-up. Best and worst cases will always depend on the data. In rough terms, the more <i>ad hoc</i> the use, the less desirable the materialization. If the usage pattern is really set, then a relational-style application-specific representation with identity resolved at load time will perform best. We can do that too, but so can others.</p> <p>The principal point is about agility as concerns the inference. Run time is more agile than materialization, and if the rules change or if different users have different needs, then materialization runs into trouble. When talking web scale, having multiple users is a given; it is very uneconomical to give everybody their own copy, and the likelihood of a user accessing any significant part of the corpus is minimal. Even if the queries were not limited, the user would typically not wait for the answer of a query doing a scan or aggregation over 1 billion <a href="http://dbpedia.org/resource/Blog" id="link-id1156a550">blog</a> posts or something of the sort. So queries will typically be selective. Selective means that they do not access all of the data, hence do not benefit from ready-made materialization for things they do not even look at. </p> <p>The exception is corpus-wide statistics queries. But these will not be done in interactive time anyway, and will not be done very often. Plus, since these do not typically run all in memory, these are disk bound. And when things are disk bound, size matters. Reading extra entailment on the way is just a performance penalty.</p> <p>Enough talk. Time for an experiment. We take the Yahoo and Falcon web crawls from the Billion Triples Challenge set, and do two things with the FOAF data in them:</p> <ol> <li>Resolve identity at insert time. We remove duplicate person URIs, and give the single <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id11317008">URI</a> all the properties of all the duplicate URIs. We expect these to be most often repeats. If a person references another person, we normalize this reference to go to the single URI of the referenced person.</li> <li>Give every duplicate URI of a person all the properties of all the duplicates. If these are the same value, the data should not get much bigger, or so we think.</li> </ol> <p>For the experiment, we will consider two people the same if they have the same <code>foaf:name</code> and are both instances of <code>foaf:Person</code>. This gets some extra hits but should not be statistically significant.</p> <p>The following is a commented <a href="http://dbpedia.org/resource/SQL" id="link-id110945b0">SQL</a> script performing the smoosh. We play with internal IDs of things, thus some of these operations cannot be done in SPARQL alone. We use SPARQL where possible for readability. As the documentation states, <code>iri_to_id</code> converts from the qualified name of an IRI to its ID and <code>id_to_iri</code> does the reverse.</p> <p>We count the triples that enter into the smoosh:</p> <blockquote> <pre>-- the name is an existence because else we'd get several times more due to -- the names occurring in many graphs <br /> sparql SELECT COUNT(*) WHERE { { SELECT DISTINCT ?person WHERE { ?person a foaf:Person } } . FILTER ( bif:exists ( SELECT (1) WHERE { ?person foaf:name ?nn } ) ) . ?person ?p ?o };<br /> -- We get 3284674 </pre></blockquote> <p>We make a few tables for intermediate results.</p> <blockquote> <pre>-- For each distinct name, gather the properties and objects from -- all subjects with this name <br /> CREATE TABLE name_prop ( np_name ANY, np_p IRI_ID_8, np_o ANY, PRIMARY KEY ( np_name, np_p, np_o ) ); ALTER INDEX name_prop ON name_prop PARTITION ( np_name VARCHAR (-1, 0hexffff) );<br /> -- Map from name to canonical IRI used for the name <br /> CREATE TABLE name_iri ( ni_name ANY PRIMARY KEY, ni_s IRI_ID_8 ); ALTER INDEX name_iri ON name_iri PARTITION ( ni_name VARCHAR (-1, 0hexffff) );<br /> -- Map from person IRI to canonical person IRI<br /> CREATE TABLE pref_iri ( i IRI_ID_8, pref IRI_ID_8, PRIMARY KEY ( i ) ); ALTER INDEX pref_iri ON pref_iri PARTITION ( i INT (0hexffff00) );<br /> -- a table for the materialization where all aliases get all properties of every other <br /> CREATE TABLE smoosh_ct ( s IRI_ID_8, p IRI_ID_8, o ANY, PRIMARY KEY ( s, p, o ) ); ALTER INDEX smoosh_ct ON smoosh_ct PARTITION ( s INT (0hexffff00) );<br /> -- disable transaction log and enable row auto-commit. This is necessary, otherwise -- bulk operations are done transactionally and they will run out of rollback space.<br /> LOG_ENABLE (2);<br /> -- Gather all the properties of all persons with a name under that name. -- INSERT SOFT means that duplicates are ignored <br /> INSERT SOFT name_prop SELECT "n", "p", "o" FROM ( sparql DEFINE output:valmode "LONG" SELECT ?n ?p ?o WHERE { ?x a foaf:Person . ?x foaf:name ?n . ?x ?p ?o } ) xx ;<br /> -- Now choose for each name the canonical IRI <br /> INSERT INTO name_iri SELECT np_name, ( SELECT MIN (s) FROM rdf_quad WHERE o = np_name AND p = IRI_TO_ID ('http://xmlns.com/foaf/0.1/name') ) AS mini FROM name_prop WHERE np_p = IRI_TO_ID ('http://xmlns.com/foaf/0.1/name') ;<br /> -- For each person IRI, map to the canonical IRI of that person <br /> INSERT SOFT pref_iri (i, pref) SELECT s, ni_s FROM name_iri, rdf_quad WHERE o = ni_name AND p = IRI_TO_ID ('http://xmlns.com/foaf/0.1/name') ;<br /> -- Make a graph where all persons have one iri with all the properties of all aliases -- and where person-to-person refs are canonicalized<br /> INSERT SOFT rdf_quad (g,s,p,o) SELECT IRI_TO_ID ('psmoosh'), ni_s, np_p, COALESCE ( ( SELECT pref FROM pref_iri WHERE i = np_o ), np_o ) FROM name_prop, name_iri WHERE ni_name = np_name OPTION ( loop, quietcast ) ;<br /> -- A little explanation: The properties of names are copied into rdf_quad with the name -- replaced with its canonical IRI. If the object has a canonical IRI, this is used as -- the object, else the object is unmodified. This is the COALESCE with the sub-query.<br /> -- This takes a little time. To check on the progress, take another connection to the -- server and do <br /> STATUS ('cluster');<br /> -- It will return something like -- Cluster 4 nodes, 35 s. 108 m/s 1001 KB/s 75% cpu 186% read 12% clw threads 5r 0w 0i -- buffers 549481 253929 d 8 w 0 pfs<br /> -- Now finalize the state; this makes it permanent. Else the work will be lost on server -- failure, since there was no transaction log <br /> CL_EXEC ('checkpoint');<br /> -- See what we got<br /> sparql SELECT COUNT (*) FROM <psmoosh> WHERE {?s ?p ?o};<br /> -- This is 2253102<br /> -- Now make the copy where all have the properties of all synonyms. This takes so much -- space we do not insert it as RDF quads, but make a special table for it so that we can -- run some statistics. This saves time.<br /> INSERT SOFT smoosh_ct (s, p, o) SELECT s, np_p, np_o FROM name_prop, rdf_quad WHERE o = np_name AND p = IRI_TO_ID ('http://xmlns.com/foaf/0.1/name') ;<br /> -- as above, INSERT SOFT so as to ignore duplicates <br /> SELECT COUNT (*) FROM smoosh_ct;<br /> -- This is 167360324<br /> -- Find out where the bloat comes from <br /> SELECT TOP 20 COUNT (*), ID_TO_IRI (p) FROM smoosh_ct GROUP BY p ORDER BY 1 DESC; </pre></blockquote> <p>The results are:</p> <blockquote> <pre>54728777 http://www.w3.org/2002/07/owl#sameAs 48543153 http://xmlns.com/foaf/0.1/knows 13930234 http://www.w3.org/2000/01/rdf-schema#seeAlso 12268512 http://xmlns.com/foaf/0.1/interest 11415867 http://xmlns.com/foaf/0.1/nick 6683963 http://xmlns.com/foaf/0.1/weblog 6650093 http://xmlns.com/foaf/0.1/depiction 4231946 http://xmlns.com/foaf/0.1/mbox_sha1sum 4129629 http://xmlns.com/foaf/0.1/homepage 1776555 http://xmlns.com/foaf/0.1/holdsAccount 1219525 http://xmlns.com/foaf/0.1/based_near 305522 http://www.w3.org/1999/02/22-rdf-syntax-ns#type 274965 http://xmlns.com/foaf/0.1/name 155131 http://xmlns.com/foaf/0.1/dateOfBirth 153001 http://xmlns.com/foaf/0.1/img 111130 http://www.w3.org/2001/vcard-rdf/3.0#ADR 52930 http://xmlns.com/foaf/0.1/gender 48517 http://www.w3.org/2004/02/skos/core#subject 45697 http://www.w3.org/2000/01/rdf-schema#label 44860 http://purl.org/vocab/bio/0.1/olb </pre></blockquote> <p>Now compare with the predicate distribution of the smoosh with identities canonicalized </p> <blockquote> <pre>sparql SELECT COUNT (*) ?p FROM <psmoosh> WHERE { ?s ?p ?o } GROUP BY ?p ORDER BY 1 DESC LIMIT 20;</pre></blockquote> <p>Results are:</p> <blockquote> <pre>748311 http://xmlns.com/foaf/0.1/knows 548391 http://xmlns.com/foaf/0.1/interest 140531 http://www.w3.org/2000/01/rdf-schema#seeAlso 105273 http://www.w3.org/1999/02/22-rdf-syntax-ns#type 78497 http://xmlns.com/foaf/0.1/name 48099 http://www.w3.org/2004/02/skos/core#subject 45179 http://xmlns.com/foaf/0.1/depiction 40229 http://www.w3.org/2000/01/rdf-schema#comment 38272 http://www.w3.org/2000/01/rdf-schema#label 37378 http://xmlns.com/foaf/0.1/nick 37186 http://dbpedia.org/property/abstract 34003 http://xmlns.com/foaf/0.1/img 26182 http://xmlns.com/foaf/0.1/homepage 23795 http://www.w3.org/2002/07/owl#sameAs 17651 http://xmlns.com/foaf/0.1/mbox_sha1sum 17430 http://xmlns.com/foaf/0.1/dateOfBirth 15586 http://xmlns.com/foaf/0.1/page 12869 http://dbpedia.org/property/reference 12497 http://xmlns.com/foaf/0.1/weblog 12329 http://blogs.yandex.ru/schema/foaf/school </pre></blockquote> <p>We can drop the <code>owl:sameAs</code> triples from the count, so the bloat is a bit less by that but it still is tens of times larger than the canonicalized copy or the initial state.</p> <p>Now, when we try using the psmoosh graph, we still get different results from the results with the original data. This is because <code>foaf:knows</code> relations to things with no <code>foaf:name</code> are not represented in the smoosh. The exist:</p> <blockquote> <pre>sparql SELECT COUNT (*) WHERE { ?s foaf:knows ?thing . FILTER ( !bif:exists ( SELECT (1) WHERE { ?thing foaf:name ?nn } ) ) };<br /> -- 1393940 </pre></blockquote> <p>So the smoosh graph is not an accurate rendition of the social network. It would have to be smooshed further to be that, since the data in the sample is quite irregular. But we do not go that far here.</p> <p>Finally, we calculate the smoosh blow up factors. We do not include <code>owl:sameAs</code> triples in the counts.</p> <blockquote> <pre>select (167360324 - 54728777) / 3284674.0; 34.290022997716059<br /> select 2229307 / 3284674.0; = 0.678699621332284 </pre></blockquote> <p>So, to get a smoosh that is not really the equivalent of the original, either multiply the original triple count by 34 or 0.68, depending on whether synonyms are collapsed or not.</p> <p>Making the smooshes does not take very long, some minutes for the small one. Inserting the big one would be longer, a couple of hours maybe. It was 33 minutes for filling the <code>smoosh_ct</code> table. The metrics were not with optimal tuning so the performance numbers just serve to show that smooshing takes time. Probably more time than allowable in an interactive situation, no matter how the process is optimized.</p>
Virtuoso Anytime: No Query Is Too Complex (updated)
http://www.openlinksw.com/blog/vdb/blog/?date=2008-12-11#1495
2008-12-11T16:13:10Z
2008-12-12T10:29:23-05:00
<p>A persistent argument against the <a href="http://dbpedia.org/resource/Linked_Data" id="link-id1199d5f8">linked data</a> <a href="http://dbpedia.org/resource/Giant_Global_Graph" id="link-id116f2730">web</a> has been the cost, scalability, and vulnerability of <a href="http://dbpedia.org/resource/SPARQL" id="link-id14e423c0">SPARQL</a> end points, should the linked data web gain serious mass and traffic.</p> <p>As we are on the brink of hosting the whole <a href="http://dbpedia.org/resource/DBpedia" id="link-id1376a8b0">DBpedia</a> <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id113c8d20">Linked Open Data</a> cloud in <a href="http://virtuoso.openlinksw.com" id="link-id11425a78">Virtuoso</a> Cluster, we have had to think of what we'll do if, for example, somebody decides to count all the triples in the set.</p> <p>How can we encourage clever use of <a href="http://dbpedia.org/resource/Data" id="link-id116f1210">data</a>, yet not die if somebody, whether through malice, lack of understanding, or simple bad luck, submits impossible queries?</p> <p>Restricting the language is not the way; any language beyond text search can express queries that will take forever to execute. Also, just returning a timeout after the first second (or whatever arbitrary time period) leaves people in the dark and does not produce an impression of responsiveness. So we decided to allow arbitrary queries, and if a quota of time or resources is exceeded, we return partial results and indicate how much processing was done.</p> <p>Here we are looking for the top 10 people whom people claim to know without being known in return, like this:</p> <blockquote> <pre>SQL> sparql SELECT ?celeb, COUNT (*) WHERE { ?claimant foaf:knows ?celeb . FILTER (!bif:exists ( SELECT (1) WHERE { ?celeb foaf:knows ?claimant } ) ) } GROUP BY ?celeb ORDER BY DESC 2 LIMIT 10;<br /> celeb callret-1 VARCHAR VARCHAR ________________________________________ _________<br /> http://twitter.com/BarackObama 252 http://twitter.com/brianshaler 183 http://twitter.com/newmediajim 101 http://twitter.com/HenryRollins 95 http://twitter.com/wilw 81 http://twitter.com/stevegarfield 78 http://twitter.com/cote 66 mailto:adam.westerski@deri.org 66 mailto:michal.zaremba@deri.org 66 http://twitter.com/dsifry 65<br /> *** Error S1TAT: [Virtuoso Driver][Virtuoso Server]RC...: Returning incomplete results, query interrupted by result timeout. Activity: 1R rnd 0R seq 0P disk 1.346KB / 3 messages<br /> SQL> sparql SELECT ?celeb, COUNT (*) WHERE { ?claimant foaf:knows ?celeb . FILTER (!bif:exists ( SELECT (1) WHERE { ?celeb foaf:knows ?claimant } ) ) } GROUP BY ?celeb ORDER BY DESC 2 LIMIT 10;<br /> celeb callret-1 VARCHAR VARCHAR ________________________________________ _________<br /> http://twitter.com/JasonCalacanis 496 http://twitter.com/Twitterrific 466 http://twitter.com/ev 442 http://twitter.com/BarackObama 356 http://twitter.com/laughingsquid 317 http://twitter.com/gruber 294 http://twitter.com/chrispirillo 259 http://twitter.com/ambermacarthur 224 http://twitter.com/t 219 http://twitter.com/johnedwards 188<br /> *** Error S1TAT: [Virtuoso Driver][Virtuoso Server]RC...: Returning incomplete results, query interrupted by result timeout. Activity: 329R rnd 44.6KR seq 342P disk 638.4KB / 46 messages</pre></blockquote> <p>The first query read all data from disk; the second run had the working set from the first and could read some more before time ran out, hence the results were better. But the response time was the same.</p> <p>If one has a query that just loops over consecutive joins, like in basic SPARQL, interrupting the processing after a set time period is simple. But such queries are not very interesting. To give meaningful partial answers with nested aggregation and sub-queries requires some more tricks. The basic idea is to terminate the innermost active sub-query/aggregation at the first timeout, and extend the timeout a bit so that accumulated results get fed to the next aggregation, like from the <code>GROUP BY</code> to the <code>ORDER BY</code>. If this again times out, we continue with the next outer layer. This guarantees that results are delivered if there were any results found for which the query pattern is true. False results are not produced, except in cases where there is comparison with a count and the count is smaller than it would be with the full evaluation.</p> <p>One can also use this as a basis for paid services. The cutoff does not have to be time; it can also be in other units, making it insensitive to concurrent usage and variations of working set.</p> <p>This system will be deployed on our <a href="http://challenge.semanticweb.org/" id="link-id11500a58">Billion Triples Challenge</a> <a href="http://b3s.openlinksw.com/" id="link-id11683120">demo instance</a> in a few days, after some more testing. When Virtuoso 6 ships, all <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id1157a500">LOD</a> Cloud AMIs and OpenLink-hosted LOD Cloud SPARQL endpoints will have this enabled by default. (AMI users will be able to disable the feature, if desired.) The feature works with Virtuoso 6 in both single server and cluster deployment.</p>
An Example of RDF Scalability
http://www.openlinksw.com/blog/vdb/blog/?date=2008-11-27#1488
2008-11-27T11:23:47Z
2008-12-01T12:09:55.000008-05:00
<p>We hear it to exhaustion, where is <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x14e828d8">RDF</a> scalability? We have been suggesting for a while that this is a solved question. I will here give some concrete numbers to back this.</p> <p>The scalability dream is to add hardware and get increased performance in proportion to the power the added component has when measured by itself. A corollary dream is to take scalability effects that are measured in a simple task and see them in a complex task.</p> <p>Below we show how we do 3.3 million random triple lookups per second on two 8 core commodity servers producing complete results, joining across partitions. On a single 4 core server, the figure is about 1 million lookups per second. With a single thread, it is about 250K lookups per second. This is the good case. But even our worse case is quite decent.</p> <p>We took a simple <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x14fef850">SPARQL</a> query, counting how many people say they reciprocally know each other. In the <a href="http://challenge.semanticweb.org/" id="link-id0x1bca04d0">Billion Triples Challenge</a> <a href="http://dbpedia.org/resource/Data" id="link-id0x1be84e88">data</a> set, there are 25M <code>foaf:knows</code> quads of which 92K are reciprocal. <i>Reciprocal</i> here means that when x knows y in some graph, y knows x in the same or any other graph.</p> <pre>SELECT COUNT (*) WHERE { ?p1 foaf:knows ?p2 . ?p2 foaf:knows ?p1 }</pre> <p>There is no guarantee that the triple of <code>x knows y</code> is in the same partition as the triple y knows x. Thus the join is randomly distributed, n partitions to n partitions.</p> <p>We left this out of the Billion Triples Challenge demo because this did not run fast enough for our liking. Since then, we have corrected this.</p> <p>If run on a single thread, this query would be a loop over all the quads with a predicate of <code>foaf:knows</code>, and an inner loop looking for a quad with 3 of 4 fields given (<code>SPO</code>). If we have a partitioned situation, we have a loop over all the <code>foaf:knows</code> quads in each partition, and an inner lookup looking for the reciprocal <code>foaf:knows</code> quad in whatever partition it may be found.</p> <p>We have implemented this with two different message patterns: </p> <ol> <li> <p> <b>Centralized:</b> One process reads all the <code>foaf:knows</code> quads from all processes. Every 50K quads, it sends a batch of reciprocal quad checks to each partition that could contain a reciprocal quad. Each partition keeps the count of found reciprocal quads, and these are gathered and added up at the end.</p> </li> <li> <p> <b>Symmetrical:</b> Each process reads the <code>foaf:knows</code> quads in its partition, and sends a batch of checks to each process that could have the reciprocal <code>foaf:knows</code> quad every 50K quads. At the end, the counts are gathered from all partitions. There is some additional control traffic but we do not go into its details here.</p> </li> </ol> <p>Below is the result measured on 2 machines each with 2 x Xeon 5345 (quad core; total 8 cores), 16G RAM, and each machine running 6 <a href="http://virtuoso.openlinksw.com" id="link-id0x16642a90">Virtuoso</a> instances. The interconnect is dual 1-Gbit ethernet. Numbers are with warm cache.</p> <blockquote> <code>Centralized: 35,543 msec, 728,634 sequential + random lookups per second <br /> Cluster 12 nodes, 35 s. 1072 m/s 39,085 KB/s 316% cpu ... <br /> <br /> Symmetrical: 7706 msec, 3,360,740 sequential + random lookups per second <br /> Cluster 12 nodes, 7 s. 572 m/s 16,983 KB/s 1137% cpu ...</code> </blockquote> <p>The second line is the summary from the cluster status report for the duration of the query. The interesting numbers are the KB/s and the %CPU. The former is the cross-sectional data transfer rate for intra-cluster communication; the latter is the consolidated CPU utilization, where a constantly-busy core counts for 100%. The point to note is that the symmetrical approach takes 4x less real time with under half the data transfer rate. Further, when using multiple machines, the speed of a single interface does not limit the overall throughput as it does in the centralized situation.</p> <p>These figures represent the best and worst cases of distributed <code>JOIN</code>ing. If we have a straight sequence of <code>JOIN</code>s, with single pattern optionals and existences and the order in which results are produced is not significant (i.e., there is aggregation, existence test, or <code>ORDER BY</code>), the symmetrical pattern is applicable. On the other hand, if there are multiple triple pattern optionals, complex sub-queries, <code>DISTINCT</code>s in the middle of the query, or results have to be produced in the order of an index, then the centralized approach must be used at least part of the time.</p> <p>Also, if we must make transitive closures, which can be thought of as an extension of a <code>DISTINCT</code> in a subquery, we must pass the data through a single point before moving the bindings to the next <code>JOIN</code> in the sequence. This happens for example in resolving <code><a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x14e1a160">owl</a>:sameAs</code> at run time. However, the good news is that performance does not fall much below the centralized figure even when there are complex nested structures with intermediate transitive closures, <code>DISTINCT</code>s, complex existence tests, etc., that require passing all intermediate results through a central point. No matter the complexity, it is always possible to vector some tens-of-thousands of variable bindings into a single message exchange. And if there are not that many intermediate results, then single query execution time is not a problem anyhow.</p> <p>For our sample query, we would get still more speed by using a partitioned hash join, filling the hash from the <code>foaf:knows</code> relations and then running the <code>foaf:knows</code> relations through the hash. If the hash size is right, a hash lookup is somewhat better than an index lookup. The problem is that when the hash join is not the right solution, it is an expensive mistake: the best case is good; the worst case is very bad. But if there is no index then hash join is better than nothing. One problem of hash joins is that they make temporary data structures which, if large, will skew the working set. One must be quite sure of the cardinality before it is safe to try a hash join. So we do not do hash joins with RDF, but we do use them sometimes with relational data. </p> <p>These same methods apply to relational data just as well. This does not make generic RDF storage outperform an application-specific relational representation on the same platform, as the latter benefits from all the same optimizations, but in terms of sheer numbers, this makes RDF representation an option where it was not an option before. RDF is all about not needing to design the schema around the queries, and not needing to limit what joins with what else.</p>
Virtuoso Vs. MySQL: Setting the Berlin Record Straight (update 2)
http://www.openlinksw.com/blog/vdb/blog/?date=2008-11-20#1485
2008-11-20T11:06:11Z
2008-11-24T10:15:11.000021-05:00
<p>In the context of the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0xa322b58">Berlin SPARQL Benchmark</a>, I have repeatedly written about measurement procedures and steady state. The point is that the numbers at larger scales are unreliable due to cache behavior if one is not careful about measurement and does not have adequate warmup. Thus it came to pass that one cut of the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x9524730">BSBM</a> paper had 3 seconds for <a href="http://dbpedia.org/resource/MySQL" id="link-id0x2ba8db0">MySQL</a> and 100 for <a href="http://virtuoso.openlinksw.com" id="link-id0xa9137d0">Virtuoso</a>, basically through ignoring cache effects.</p> <p>So we decided to do it ourselves.</p> <p>The score is (updated with revised <code>innodb_buffer_pool_size</code> setting, based on advice noted down below):</p> <table border="1" cellspacing="2" cellpadding="5"> <tr> <th>n-clients</th> <th>Virtuoso</th> <th>MySQL <br /> (with increased buffer pool size)</th> <th>MySQL <br /> (with default buffer poll size)</th> </tr> <tr align="right"> <td>1</td> <td> 41,161.33</td> <td> 27,023.11 </td> <td> 12,171.41</td> </tr> <tr align="right"> <td>4</td> <td> 127,918.30</td> <td> (pending) </td> <td> 37,566.82</td> </tr> <tr align="right"> <td>8</td> <td> 218,162.29 </td> <td> 105,524.23 </td> <td> 51,104.39 </td> </tr> <tr align="right"> <td>16</td> <td> 214,763.58 </td> <td> 98,852.42 </td> <td> 47,589.18 </td> </tr> </table> <p>The metric is the query mixes per hour from the BSBM test driver output. For the interested, the complete output is <a href="http://www.openlinksw.com/weblog/oerling/texts/bsbmres.txt" id="link-id1119f770">here</a>.</p> <p>The benchmark is pure <a href="http://dbpedia.org/resource/SQL" id="link-id0x2b61c88">SQL</a>, nothing to do with <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x17a6d408">SPARQL</a> or <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x9a0a968">RDF</a>.</p> <p>The hardware is 2 x Xeon 5345 (2 x quad core, 2.33 GHz), 16 G RAM. The OS is 64-bit Debian Linux.</p> <p>The benchmark was run at a scale of 200,000. Each run had 2000 warm-up query mixes and 500 measured query mixes, which gives steady state, eliminating any effects of OS disk cache and the like. Both databases were configured to use 8G for disk cache. The test effectively runs from memory. We ran an analyze table on each MySQL table but noticed that this had no effect. Virtuoso does the stats sampling on the go; possibly MySQL also since the explicit stats did not make any difference. The MySQL tables were served by the InnoDB engine. MySQL appears to cache results of queries in some cases. This was not apparent in the tests.</p> <p>The versions are 5.09 for Virtuoso and 5.1.29 for MySQL. You can download and examine --</p> <ul> <li> <a href="http://www.openlinksw.com/weblog/oerling/texts/virtuoso.ini" id="link-id14fe17f0">Virtuoso configuration file</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/texts/my.cnf" id="link-id116fe490">MySQL configuration file</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/texts/create_tables_and_rdf_view.sql" id="link-id14ce9268">Table definitions & RDF views</a> </li> <li> <a href="http://www.openlinksw.com/weblog/oerling/texts/mysqlinx.sql" id="link-id1535e298">Indexes on MySQL tables</a> </li> </ul> <p> <strike>MySQL ought to do better. We suspect that here, just as in the TPC-D experiment we made way back, the query plans are not quite right. Also we rarely saw over 300% CPU utilization for MySQL. It is possible there is a config parameter that affects this. The public is invited to tell us about such.</strike> </p> <p> <b>Update:</b> </p> <p>Andreas Schultz of the BSBM team advised us to increase the <code>innodb_buffer_pool_size</code> setting in the MySQL config. We did and it produced some improvement. Indeed, this is more like it, as we now see CPU utilization around 700% instead of the 300% in the previously published run, which rendered it suspect. Also, our experiments with TPC-D led us to expect better. We ran these things a few times so as to have warm cache.</p> <p>On the first run, we noticed that the Innodb warm up time was somewhere well in excess of 2000 query mixes. Another time, we should make a graph of throughput as a function of time for both MySQL and Virtuoso. We recently made a greedy prefetch hack that should give us some mileage there. For the next BSBM, all we can advise is to run larger scale system for half an hour first and then measure and then measure again. If the second measurement is the same as the first then it is good.</p> <p>As always, since MySQL is not our specialty, we confidently invite the public to tell us how to make it run faster. So, unless something more turns up, our next trial is a revisit of <a href="http://dbpedia.org/resource/TPC-H" id="link-id0x17a20498">TPC-H</a>.</p>
ISWC 2008: Some Questions
http://www.openlinksw.com/blog/vdb/blog/?date=2008-11-04#1481
2008-11-04T15:54:42Z
2008-11-04T14:37:01-05:00
<h2>Inference: Is it always forward chaining?</h2> <p>We got a number of questions about <a href="http://virtuoso.openlinksw.com" id="link-id0x13c64b60">Virtuoso</a>'s inference support. It seems that we are the odd one out, as we do not take it for granted that inference ought to consist of materializing entailment.</p> <p>Firstly, of course one can materialize all one wants with Virtuoso. The simplest way to do this is using SPARUL. With the recent transitivity extensions to <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x14d17778">SPARQL</a>, it is also easy to materialize implications of transitivity with a single statement. Our point is that for trivial entailment such as subclass, sub-property, single transitive property, and <a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x128e55d0">owl</a>:sameAs, we do not require materialization, as we can resolve these at run time also, with backward-chaining built into the engine.</p> <p>For more complex situations, one needs to materialize the entailment. At the present time, we know how to generalize our transitive feature to run arbitrary backward-chaining rules, including recursive ones. We could have a sort of Datalog backward-chaining embedded in our <a href="http://dbpedia.org/resource/SQL" id="link-id0x12614770">SQL</a>/SPARQL and could run this with good parallelism, as the transitive feature already works with clustering and partitioning without dying of message latency. Exactly when and how we do this will be seen. Even if users want entailment to be materialized, such a rule system could be used for producing the materialization at good speed.</p> <p>We had a word with <a href="http://web.comlab.ox.ac.uk/people/Ian.Horrocks/" id="link-id117c99d0">Ian Horrocks</a> on the question. He noted that it is often naive on behalf of the community to tend to equate description of semantics with description of algorithm. The <a href="http://dbpedia.org/resource/Data" id="link-id0x145b2980">data</a> need not always be blown up.</p> <p>The advantage of not always materializing is that the working set stays better. Once the working set is no longer in memory, response times jump disproportionately. Also, if the data changes or is retracted or is unreliable, one can end up doing a lot of extra work with materialization. Consider the effect of one malicious sameAs statement. This can lead to a lot of effects that are hard to retract. On the other hand, if running in memory with static data such as the LUBM benchmark, the queries run some 20% faster if entailment subclasses and sub-properties are materialized rather than done at run time.</p> <h2>Genetic Algorithms for SPARQL?</h2> <p>Our compliments for the wildest idea of the conference go to <a href="http://www.eyaloren.org/" id="link-id1a203af8">Eyal Oren</a>, <a href="http://www.few.vu.nl/~cgueret/" id="link-id16208758">Christophe Guéret</a>, and <a href="http://www.few.vu.nl/~schlobac/" id="link-id111923e0">Stefan Schlobach</a>, <i>et al</i>, for their <a href="http://www.informatik.uni-trier.de/~ley/db/conf/semweb/iswc2008.html#OrenGS08" id="link-id11793540">paper on using genetic algorithms for guessing how variables in a SPARQL query ought to be instantiated</a>. Prisoners of our "conventional wisdom" as we are, this might never have occurred to us.</p> <h2>Schema Last?</h2> <p>It is interesting to see how the industry comes to the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x12b57e90">semantic web</a> conferences talking about schema last while at the same time the traditional semantic web people stress enforcing schema constraints and making more predictably performing and database friendlier logics. So do the extremes converge.</p> <p>There is a point to schema last. <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x12a8ff48">RDF</a> is very good for getting a view of ad hoc or unknown data. One can just load and look at what there is. Also, additions of unforeseen optional properties or relations to the schema are easy and efficient. However, it seems that a really high traffic online application would always benefit from having some application specific data structures. Such could also save considerably in hardware.</p> <p>It is not a sharp divide between RDF and relational application oriented representation. We have the capabilities in our RDB to RDF mapping. We just need to show this and have SPARUL and data loading</p>
ISWC 2008: RDB2RDF Face-to-Face
http://www.openlinksw.com/blog/vdb/blog/?date=2008-11-04#1477
2008-11-04T13:26:19Z
2008-11-04T17:20:35-05:00
<p>The W3C's RDB-to-<a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x153bdcf8">RDF</a> mapping incubator group (<a href="http://www.w3.org/2005/Incubator/rdb2rdf/" id="link-id0x13e3e6b8">RDB2RDF XG</a>) met in <a href="http://dbpedia.org/resource/Karlsruhe" id="link-id0x15236b08">Karlsruhe</a> after <a href="http://iswc2008.semanticweb.org/" id="link-id0x2450fba8">ISWC 2008</a>.</p> <p>The meeting was about writing a charter for a working group that would define a standard for mapping relational databases to RDF, either for purposes of import into RDF stores or of query mapping from <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x14c84338">SPARQL</a> to <a href="http://dbpedia.org/resource/SQL" id="link-id0x146db368">SQL</a>. There was a lot of agreement and the meeting even finished ahead of the allotted time.</p> <h2>Whose Identifiers?</h2> <p>There was discussion concerning using the <a href="http://dbpedia.org/resource/Entity" id="link-id0x12c15e58">Entity</a> Name Service from the Okkam project for assigning URIs to entities mapped from relational databases. This makes sense when talking about long-lived, legal entities, such as people or companies or geography. Of course, there are cases where this makes no sense; for example, a purchase order or maintenance call hardly needs an identifier registered with the ENS. The problem is, in practice, a CRM could mention customers that have an ENS registered ID (or even several such IDs) and others that have none. Of course, the CRM's reference cannot depend on any registration. Also, even when there is a stable <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x12b7b5c0">URI</a> for the entity, a CRM may need a key that specifies some administrative subdivision of the customer.</p> <p>Also we note that an on-demand RDB-to-RDF mapping may have some trouble dealing with "same as" assertions. If names that are anything other than string forms of the keys in the system must be returned, there will have to be a lookup added to the RDB. This is an administrative issue. Certainly going over the network to ask for names of items returned by queries has a prohibitive cost. It would be good for ad hoc integration to use shared URIs when possible. The trouble of adding and maintaining lookups for these, however, makes this more expensive than just mapping to RDF and using literals for joining between independently maintained systems.</p> <h2> <a href="http://dbpedia.org/resource/XML" id="link-id0x14bf7da0">XML</a> or RDF?</h2> <p>We talked about having a language for human consumption and another for discovery and machine processing of mappings. Would this latter be XML or RDF based? Describing every detail of syntax for a mapping as RDF is really tedious. Also such descriptions are very hard to query, just as <a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x1493ffc0">OWL</a> ontologies are. One solution is to have opaque strings embedded into RDF, just like XSLT has <a href="http://dbpedia.org/resource/XPath" id="link-id0x1400fe98">XPath</a> in string form embedded into XML. Maybe it will end up in this way here also. Having a complete XML mapping of the parse tree for mappings, XQueryX-style, could be nice for automatic generation of mappings with XSLT from an XML view of the <a href="http://dbpedia.org/resource/Information" id="link-id0x14c846d8">information</a> schema. But then XSLT can also produce text, so an XML syntax that has every detail of a mapping language as distinct elements is not really necessary for this.</p> <p>Another matter is then describing the RDF generated by the mapping in terms of RDFS or OWL. This would be a by-product of declaring the mapping. Most often, I would presume the target ontology to be given, though, reducing the need for this feature. But if RDF mapping is used for discovery of <a href="http://dbpedia.org/resource/Data" id="link-id0x14f6f128">data</a>, such a description of the exposed data is essential.</p> <h2>Interoperability</h2> <p>We agreed with <a href="http://www.informatik.uni-leipzig.de/~auer/foaf.rdf#me" id="link-id0x1e776730">Sören Auer</a> that we could make <a href="http://virtuoso.openlinksw.com" id="link-id0x1477ad18">Virtuoso</a>'s mapping language compatible with <a href="http://triplify.org/" id="link-id0x15514388">Triplify</a>. Triplify is very simple, extraction only, no SPARQL, but does have the benefit of expressing everything in SQL. As it happens, I would be the last person to tell a web developer what language to program in. So if it is SQL, then let it stay SQL. Technically, a lot of the information the Virtuoso mapping expresses is contained in the Triplify SQL statements, but not all. Some extra declarations are needed still but can have reasonable defaults.</p> <p>There are two ways of stating a mapping. Virtuoso starts with the triple and says which tables and columns will produce the triple. Triplify starts with the SQL statement and says what triples it produces. These are fairly equivalent. For the web developer, the latter is likely more self-evident, while the former may be more compact and have less repetition.</p> <p>Virtuoso and Triplify alone would give us the two interoperable implementations required from a working group, supposing the language were annotations on top of SQL. This would be a guarantee of delivery, as we would be close enough to the result from the get go.</p> <h2>Related Web resources</h2> <ul> <li> <a href="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSSQL2RDF" id="link-id14e27040">OpenLink Virtuoso: Open-Source Edition: Mapping SQL Data to RDF</a> </li> <li> <a href="http://virtuoso.openlinksw.com/Whitepapers/pdf/Virtuoso_SQL_to_RDF_Mapping.pdf" id="link-id1baad3a8">Virtuoso RDF Views â Getting Started Guide (PDF)</a> </li> </ul>
ISWC 2008: The Scalable Knowledge Systems Workshop
http://www.openlinksw.com/blog/vdb/blog/?date=2008-11-03#1473
2008-11-03T13:16:47Z
2008-11-03T12:33:54-05:00
<p>Mike Dean of <a href="http://dbpedia.org/resource/BBN_Technologies" id="link-id0x25699878">BBN Technologies</a> opened the Scalable <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1ed01750">Knowledge</a> Systems Workshop with an invited talk. He reminded us of the facts of nature as concern the cost of distributed computing and running out of space for the working set. Developers in the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x21fbb9a8">semantic web</a> field deplorably often ignore these facts, or alternatively recognize them and admit that they are unbeatable, that one just can't join across partitions.</p> <p>I gave a talk about the <a href="http://virtuoso.openlinksw.com" id="link-id0x20b6e020">Virtuoso</a> Cluster edition, wherein I repeated essentially the same ground facts as Mike and outlined how we (in spite of these) profit from distributed memory multiprocessing. To those not intimate with these questions, let me affirm that deriving benefit from threading in a symmetric multiprocessor box, let alone a cluster connected by a network, totally depends on having many relatively long running things going at a time and blocking as seldom as possible.</p> <p>Further, Mike Dean talked about <a href="http://www.asio.bbn.com/" id="link-id0x222252f0">ASIO</a>, the BBN suite of semantic web tools. His most challenging statement was about the storage engine, a network-database-inspired triple-store using memory-mapped files. </p> <p>Will the <a href="http://dbpedia.org/resource/CODASYL" id="link-id0x222d8730">CODASYL</a> days come back, and will the linked list on disk be the way to store triples/quads? I would say that this will have, especially with a memory-mapped file, probably a better best-case as a B-tree but that this also will be less predictable with fragmentation. With Virtuoso, using a B-tree index, we see about 20-30% of CPU time spent on index lookup when running LUBM queries. With a disk-based memory-mapped linked-list storage, we would see some improvements in this while getting hit probably worse than now in the case of fragmentation. Plus compaction on the fly would not be nearly as easy and surely far less local, if there were pointers between pages. So it is my intuition that trees are a safer bet with varying workloads while linked lists can be faster in a query-dominated in-memory situation.</p> <p>Chris Bizer presented the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x22e41c40">Berlin SPARQL Benchmark</a> (<a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1c909960">BSBM</a>), which has already been discussed here in some detail. He did acknowledge that the next round of the race must have a real steady-state rule. This just means that the benchmark must be run long enough for the system under test to reach a state where the cache is full and the performance remains indefinitely at the same level. Reaching steady state can take 20-30 minutes in some cases.</p> <p>Regardless of steady state, BSBM has two generally valid conclusions: </p> <ol> <li>mapping relational to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x21d01890">RDF</a>, where possible, is faster than triple storage; and </li> <li>the equivalent relational solution can be some 10x faster than the pure triples representation.</li> </ol> <p>Mike Dean asked whether BSBM was a case of a setup to have triple stores fail. Not necessarily, I would say; we should understand that one motivation of BSBM is testing mapping technologies. Therefore it must have a workload where mapping makes sense. Of course there are workloads where triples are unchallenged â take the <a href="http://challenge.semanticweb.org/" id="link-id0x1feb9250">Billion Triples Challenge</a> <a href="http://dbpedia.org/resource/Data" id="link-id0x1fe12b60">data</a> set for one.</p> <p>Also, with BSBM, once should note that the query optimization time plays a fairly large role since most queries touch relatively little data. Also, even if the scale is large, the working set is not nearly the size of the database. This in fact penalizes mapping technologies against native <a href="http://dbpedia.org/resource/SQL" id="link-id0x1e275c88">SQL</a> since the difference there is compiling the query, especially since parameters are not used. So, Chris, since we both like to map, let's make a benchmark that shows mapping closer to native SQL.</p> <h2>Bridging the 10x Gap?</h2> <p>When we run Virtuoso relational against Virtuoso triple store with the <a href="http://dbpedia.org/resource/TPC-H" id="link-id0x22046d88">TPC-H</a> workload, we see that the relational case is significantly faster. These are long queries, thus query optimization time is negligible; we are here comparing memory-based access times. Why is this? The answer is that a single index lookup gives multiple column values with almost no penalty for the extra column. Also, since the number of total joins is lower, the overhead coming from moving from join to next join is likewise lower. This is just a meter of count of executed instructions.</p> <p>A column store joins in principle just as much as a triple store. However, since the BI workload often consists of scanning over large tables, the joins tend to be local, the needed lookup can often use the previous location as a starting point. A triple store can do the same if queries have high locality. We do this in some SQL situations and can try this with triples also. The RDF workload is typically more random in its access pattern, though. The other factor is the length of control path. A column store has a simpler control flow if it knows that the column will have exactly one value per row. With RDF, this is not a given. Also, the column store's row is identified by a single number and not a multipart key. These two factors give the column store running with a fixed schema some edge over the more generic RDF quad store.</p> <p>There was some discussion on how much closer a triple store could come to a relational one. Some gains are undoubtedly possible. We will see. For the ideal row store workload, the <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x22f837c0">RDBMS</a> will continue to have some edge. Large online systems typically have a large part of the workload that is simple and repetitive. There is nothing to prevent one having special indices for supporting such workload, even while retaining the possibility of arbitrary triples elsewhere. Some degree of application-specific data structure does make sense. We just need to show how this is done. In this way, we have a continuum and not an either/or choice of triples vs. tables.</p> <h2>Scale, Where Next?</h2> <p>Concerning the future direction of the workshop, there were a few directions suggested. One of the more interesting ones was Mike Dean's suggestion about dealing with a large volume of same-as assertions, specifically a volume where materializing all the entailed triples was no longer practical. Of course, there is the question of scale. This time, we were the only ones focusing on a parallel database with no restrictions on joining.</p>
Virtuoso - Are We Too Clever for Our Own Good? (updated)
http://www.openlinksw.com/blog/vdb/blog/?date=2008-10-26#1467
2008-10-26T12:15:35Z
2008-10-27T12:07:58-04:00
<p>"Physician, heal thyself," it is said. We profess to say what the messaging of the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x1b4a25f0">semantic web</a> ought to be, but is our own perfect?</p> <p>I will here engage in some critical introspection as well as amplify on some answers given to <a href="http://virtuoso.openlinksw.com" id="link-id0x1e4f9928">Virtuoso</a>-related questions in recent times.</p> <p>I use some conversations from the <a href="http://dbpedia.org/resource/Vienna" id="link-id0x1e6c0ca8">Vienna</a> <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x1e56df88">Linked Data</a> Practitioners meeting as a starting point. These views are mine and are limited to the Virtuoso server. These do not apply to the <a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id0x1e680440">ODS</a> (<a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id0x1e140068">OpenLink Data Spaces</a>) applications line, <a href="http://oat.openlinksw.com/" id="link-id0x1f4ba630">OAT</a> (<a href="http://oat.openlinksw.com/" id="link-id0x1ba4bac8">OpenLink Ajax Toolkit</a>), or <a href="http://ode.openlinksw.com/" id="link-id0x1d4159b0">ODE</a> (<a href="http://ode.openlinksw.com/" id="link-id0x1e973c80">OpenLink Data Explorer</a>).</p> <h3>"It is not always clear what the main thrust is, we get the impression that you are spread too thin," said <a href="http://www.informatik.uni-leipzig.de/~auer/foaf.rdf#me" id="link-id0x1f8bafe0">Sören Auer</a>.</h3> <p>Well, personally, I am all for core competence. This is why I do not participate in all the online conversations and groups as much as I could, for example. Time and energy are critical resources and must be invested where they make a difference. In this case, the real core competence is running in the database race. This in itself, come to think of it, is a pretty broad concept.</p> <p>This is why we put a lot of emphasis on Linked Data and the <a href="http://dbpedia.org/resource/Data" id="link-id0x200bd1f0">Data</a> Web for now, as this is the emerging game. This is a deliberate choice, not an outside imperative or built-in limitation. More specifically, this means exposing any pre-existing relational data as linked data plus being the definitive <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1fb03528">RDF</a> store.</p> <p>We can do this because we own our database and <a href="http://dbpedia.org/resource/SQL" id="link-id0x1e7dcc70">SQL</a> and data access middleware and have a history of connecting to any <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x1e9baf18">RDBMS</a> out there.</p> <p>The principal message we have been hearing from the RDF field is the call for scale of triple storage. This is even louder than the call for relational mapping. We believe that in time mapping will exceed triple storage as such, once we get some real production strength mappings deployed, enough to outperform RDF warehousing.</p> <p>There are also RDF middleware things like RDF-ization and demand-driven web harvesting (i.e, the so-called Sponger). These are <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1f5f6b78">SPARQL</a> options, thus accessed via standard interfaces. We have little desire to create our own languages or APIs, or to tell people how to program. This is why we recently introduced <a href="http://sourceforge.net/projects/sesame/" id="link-id0x206818c8">Sesame</a>- and <a href="http://jena.sourceforge.net/" id="link-id0x202b3348">Jena</a>-compatible APIs to our RDF store. From what we hear, these work. On the other hand, we do not hesitate to move beyond the standards when there is obvious value or necessity. This is why we brought SPARQL up to and beyond SQL expressivity. It is not a case of E3 (Embrace, Extend, Extinguish).</p> <p>Now, this message could be better reflected in our material on the web. This <a href="http://dbpedia.org/resource/Blog" id="link-id0x1c82e508">blog</a> is a rather informal step in this direction; more is to come. For now we concentrate on delivering.</p> <p>The conventional communications wisdom is to split the message by target audience. For this, we should split the RDF, relational, and web services messages from each other. We believe that a challenger, like the semantic web technology stack, must have a compelling message to tell for it to be interesting. This is not a question of research prototypes. The new technology cannot lack something the installed technology takes for granted.</p> <p>This is why we do not tend to show things like how to insert and query a few triples: No business out there will insert and query triples for the sake of triples. There must be a more compelling story â for example, turning the whole world into a database. This is why our examples start with things like turning the <a href="http://dbpedia.org/resource/TPC-H" id="link-id0x20832510">TPC-H</a> database into RDF, queries and all. Anything less is not interesting. Why would an enterprise that has business intelligence and integration issues way more complex than the rather stereotypical TPC-H even look at a technology that pretends to be all for integration and all for expressivity of queries, yet cannot answer the first question of the entry exam?</p> <p>The world out there is complex. But maybe we ought to make some simple tutorials? So, as a call to the people out there, tell us what a good tutorial would be. The question is more about figuring out what is out there and adapting these and making a sort of compatibility list. Jena and Sesame stuff ought to run as is. We could offer a webinar to all the data web luminaries showing how to promote the data web message with Virtuoso. After all, why not show it on the best platform?</p> <h3>"You are arrogant. When I read your papers or documentation, the impression I get is that you say you are smart and the reader is stupid."</h3> <p>We should answer in multiple parts.</p> <p>For general collateral, like web sites and documentation:</p> <p>The web site gives a confused product image. For the Virtuoso product, we should divide at the top into</p> <ul> <li> Data web and RDF - Host linked data, expose relational assets as linked data;</li> <li> Relational Database - Full function, high performance, open source, Federated/Virtual Relational DBMS, expose heterogeneous RDB assets through one point of contact for integration;</li> <li> Web Services - access all the above over standard protocols, dynamic web pages, web hosting.</li> </ul> <p>For each point, one simple statement. We all know what the above things mean?</p> <p>Then we add a new point about scalability that impacts all the above, namely the Virtuoso version 6 Cluster, meaning that you can do all these things at 10 to 1000 times the scale. This means this much more data or in some cases this much more requests per second. This too is clear.</p> <p>Far as I am concerned, hosting Java or .<a href="http://dbpedia.org/resource/.NET_Framework" id="link-id0x20283a88">NET</a> does not have to be on the front page. Also, we have no great interest in going against <a href="http://dbpedia.org/resource/Apache" id="link-id0x2024a068">Apache</a> when it comes to a web server only situation. The fact that we have a web listener is important for some things but our claim to fame does not rest on this.</p> <p>Then for documentation and training materials: The documentation should be better. Specifically it should have more of a how-to dimension since nobody reads the whole thing anyhow. About online tutorials, the order of presentation should be different. They do not really reflect what is important at the present moment either.</p> <p>Now for conference papers: Since taking the data web as a focus area, we have submitted some papers and had some rejected because these do not have enough references and do not explain what is obvious to ourselves.</p> <p>I think that the communications failure in this case is that we want to talk about end to end solutions and the reviewers expect research. For us, the solution is interesting and exists only if there is an adequate functionality mix for addressing a specific use case. This is why we do not make a paper about query cost model alone because the cost model, while indispensable, is a thing that is taken for granted where we come from. So we mention RDF adaptations to cost model, as these are important to the whole but do not find these to be the justification for a whole paper. If we made papers on this basis, we would have to make five times as many. Maybe we ought to.</p> <h3>"Virtuoso is very big and very difficult"</h3> <p>One thing that is not obvious from the Virtuoso packaging is that the minimum installation is an executable under 10MB and a config file. Two files.</p> <p>This gives you SQL and SPARQL out of the box. Adding <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0x1ee61058">ODBC</a> and <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0x1b8c31c0">JDBC</a> clients is as simple as it gets. After this, there is basic database functionality. Tuning is a matter of a few parameters that are explained on this blog and elsewhere. Also, the full scale installation is available as an Amazon EC2 image, so no installation required.</p> <p>Now for the difficult side:</p> <p>Use SQL and SPARQL; use stored procedures whenever there is server side business logic. For some time critical web pages, use VSP. Do not use VSPX. Otherwise, use whatever you are used to â <a href="http://dbpedia.org/resource/PHP" id="link-id0x20a13c00">PHP</a> or Java or anything else. For web services, simple is best. Stick to basics. "The engineer is one who can invent a simple thing." Use SQL statements rather than admin UI.</p> <p>Know that you can start a server with no database file and you get an initial database with nothing extra. The demo database, the way it is produced by installers is cluttered.</p> <p>We should put this into a couple of use case oriented how-tos.</p> <p>Also, we should create a network of "friendly local virtuoso geeks" for providing basic training and services so we do not have to explain these things all the time. To all you data-web-ers out there â please sign up and we will provide instructions, etc. Contact Yrjänä Rankka (ghard[at-sign]openlinksw.com), or go through the mailing lists; do not contact me directly.</p> <h3>"OK, we understand that you may be good at the large end of the spectrum but how do you reconcile this with the lightweight or embedded end, like the semantic desktop?"</h3> <p>Now, what is good for one end is usually good for the other. Namely, a database, no matter the scale, needs to have space efficient storage, fast index lookup, and correct query plans. Then there are things that occur only at the high-end, like clustering, but these are separate things. For embedding, the initial memory footprint needs to be small. With Virtuoso, this is accomplished by leaving out some 200 built-in tables and 100,000 lines of SQL procedures that are normally in by default, supporting things such as DAV and diverse other protocols. After all, if SPARQL is all one wants these are not needed.</p> <p>If one really wants to do one's server logic (like web listener and thread dispatching) oneself, this is not impossible but requires some advice from us. On the other hand, if one wants to have logic for security close to the data, then using stored procedures is recommended; these execute right next to the data, and support inline SPARQL and SQL. Depending on the license status of the other code, some special licensing arrangements may apply.</p> <p>We are talking about such things with different parties at present.</p> <h3>"How webby are you? What is webby?"</h3> <p>"Webby means distributed, heterogeneous, open; not monolithic consolidation of everything."</p> <p>We are philosophically webby. We come from open standards; we are after all called OpenLink; our history consists of connecting things. We believe in choice â the user should be able to pick the best of breed for components and have them work together. We cannot and do not wish to force replacement of existing assets. Transforming data on the fly and connecting systems, leaving data where it originally resides, is the first preference. For the data web, the first preference is a federation of independent SPARQL end points. When there is harvesting, we prefer to do it on demand, as with our Sponger. With the immense amount of data out there we believe in finding what is relevant <i>when</i> it is relevant, preferably close at hand, leveraging things like social networks. With a data web, many things which are now siloized, such as marketplaces and social networks, will return to the open.</p> <p>Google-style crawling of everything becomes less practical if one needs to run complex <i>ad hoc</i> queries against the mass of data. For these types of scenarios, if one needs to warehouse, the data cloud will offer solutions where one pays for database on demand. While we believe in loosely coupled federation where possible, we have serious work on the scalability side for the data center and the compute-on-demand cloud.</p> <h3>"How does OpenLink see the next five years unfolding?"</h3> <p>Personally, I think we have the basics for the birth of a new inflection in the <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1fb9ae58">knowledge</a> economy. The <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x1f07c648">URI</a> is the unit of exchange; its value and competitive edge lie in the data it links you with. A name without context is worth little, but as a name gets more use, more <a href="http://dbpedia.org/resource/Information" id="link-id0x1f007d60">information</a> can be found through that name. This is anything from financial statistics, to legal precedents, to news reporting or government data. Right now, if the SEC just added one line of markup to the XBRL template, this would instantaneously make all SEC-mandated reporting into linked data via GRDDL.</p> <p>The URI is a carrier of brand. An information brand gets traffic and references, and this can be monetized in diverse ways. The key word is <i>context</i>. Information overload is here to stay, and only better context offers the needed increase in productivity to stay ahead of the flood.</p> <p>Semantic technologies on the whole can help with this. Why these should be semantic web or data web technologies as opposed to just semantic is the linked data value proposition. Even smart islands are still islands. Agility, scale, and scope, depend on the possibility of combining things. Therefore common terminologies and dereferenceability and discoverability are important. Without these, we are at best dealing with closed systems even if they were smart. The expert systems of the 1980s are a case in point.</p> <p>Ever since the .com era, the <a href="http://dbpedia.org/resource/Uniform_Resource_Locator" id="link-id0x2048e670">URL</a> has been a brand. Now it becomes a URI. Thus, entirely hiding the URI from the user experience is not always desirable. The URI is a sort of handle on the provenance and where more can be found; besides, people are already used to these.</p> <p>With linked data, information value-add products become easy to build and deploy. They can be basically just canned SPARQL queries combining data in a useful and insightful manner. And where there is traffic there can be monetization, whether by advertizing, subscription, or other means. Such possibilities are a natural adjunct to the blogosphere. To publish analysis, one no longer needs to be a think tank or media company. We could call this scenario the birth of a meshup economy.</p> <p>For OpenLink itself, this is our roadmap. The immediate future is about getting our high end offerings like clustered RDF storage generally available, both on the cloud and for private data centers. Ourselves, we will offer the whole <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1c696170">Linked Open Data</a> cloud as a database. The single feature to come in version 2 of this is fully automatic partitioning and repartitioning for on-demand scale; now, you have to choose how many partitions you have.</p> <p>This makes some things possible that were hard thus far.</p> <p>On the mapping front, we go for real-scale data integration scenarios where we can show that SPARQL can unify terms and concepts across databases, yet bring no added cost for complex queries. Enterprises can use their existing warehouses and have an added level of abstraction, the possibility of cross systems interlinking, the advantages of using the same taxonomies and ontologies across systems, and so forth.</p> <p>Then there will be developments in the direction of smarter web harvesting on demand with the Virtuoso <a href="http://virtuoso.openlinksw.com/Whitepapers/html/VirtSpongerWhitePaper.html" id="link-id0x206ab780">Sponger</a>, and federation of heterogeneous SPARQL end points. The federation is not so unlike clustering, except the time scales are 2 orders of magnitude longer. The work on SPARQL end point statistics and data set description and discovery is a good development in the community.</p> <p>Then there will be NLP integration, as exemplified by the Open Calais linked data wrapper and more.</p> <p>Can we pull this off or is this being spread too thin? We know from experience that all this can be accomplished. Scale is already here; we show it with the billion triples set. Mapping is here; we showed it last in the Berlin Benchmark. We will also show some TPC-H results after we get a little quiet after the ISWC event. Then there is ongoing maintenance but with this we have shown a steady turnaround and quick time to fix for pretty much anything.</p>
State of the Semantic Web, Part 2 - The Technical Questions (updated)
http://www.openlinksw.com/blog/vdb/blog/?date=2008-10-26#1466
2008-10-26T12:02:43Z
2008-10-27T11:28:14-04:00
<p>Here I will talk about some more technical questions that came up. This is mostly general; <a href="http://virtuoso.openlinksw.com" id="link-id0x205901a0">Virtuoso</a> specific questions and answers are separate. </p> <h3>"How to Bootstrap? Where will the triples come from?"</h3> <p>There are already wrappers producing <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x13519ac8">RDF</a> from many applications. Since any structured or semi-structured <a href="http://dbpedia.org/resource/Data" id="link-id0x1c93b418">data</a> can be converted to RDF and often there is even a pre-existing terminology for the application domain, the availability of the data <i>per se</i> is not the concern.</p> <p>The triples may come from any application or database, but they will not come from the end user directly. There was a good talk about photograph annotation in <a href="http://dbpedia.org/resource/Vienna" id="link-id0x1ea9d150">Vienna</a>, describing many ways of deriving metadata for photos. The essential wisdom is annotating on the spot and wherever possible doing so automatically. The consumer is very unlikely to go annotate photos after the fact. Further, one can infer that photos made with the same camera around the same time are from the same location. There are other such heuristics. In this use case, the end user does not need to see triples. There is some benefit though in using commonly used geographical terminology for linking to other data sources.</p> <h3>"How will one develop applications?"</h3> <p>I'd say one will develop them much the same way as thus far. In <a href="http://dbpedia.org/resource/PHP" id="link-id0x207fca00">PHP</a>, for example. Whether one's query language is <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x20a5fde0">SPARQL</a> or <a href="http://dbpedia.org/resource/SQL" id="link-id0x1a0bb5e0">SQL</a> does not make a large difference in how basic web UI is made.</p> <p>A SPARQL end-point is no more an end-user item than a SQL command-line is.</p> <p>A common mistake among techies is that they think the data structure and user experience can or ought to be of the same structure. The UI dialogs do not, for example, have to have a 1:1 correspondence with SQL tables.</p> <p>The idea of generating UI from data, whether relational or data-web, is so seductive that generation upon generation of developers fall for it, repeatedly. Even I, at OpenLink, after supposedly having been around the block a couple of times made some experiments around the topic. What does make sense is putting a thin wrapper or HTML around the application, using XSLT and such for formatting. Since the model does allow for unforeseen properties of data, one can build a viewer for these alongside the regular forms. For this, Ajax technologies like <a href="http://oat.openlinksw.com/" id="link-id0x1e91d118">OAT</a> (the <a href="http://oat.openlinksw.com/" id="link-id0x174b7950">OpenLink AJAX Toolkit</a>) will be good.</p> <p>The UI ought not to completely hide the URIs of the data from the user. It should offer a drill down to faceted views of the triples for example. Remember when Xerox talked about graphical user interfaces in 1980? "Don't mode me in" was the slogan, as I recall.</p> <p>Since then, we have vacillated between modal and non-modal interaction models. Repetitive workflows like order entry go best modally and are anyway being replaced by web services. Also workflows that are very infrequent benefit from modality; take personal network setup wizards, for example. But enabling the <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1ea14610">knowledge</a> worker is a domain that by its nature must retain some respect for human intelligence and not kill this by denying access to the underlying data, including provenance and URIs. Face it: the world is not getting simpler. It is increasingly data dependent and when this is so, having semantics and flexibility of access for the data is important.</p> <p>For a real-time task-oriented user interface like a fighter plane cockpit, one will not show URIs unless specifically requested. For planning fighter sorties though, there is some potential benefit in having all data such as friendly and hostile assets, geography, organizational structure, etc., as <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x207bcd20">linked data</a>. It makes for more flexible querying. Linked data does not <i>per se</i> mean open, so one can be joinable with open data through using the same identifiers even while maintaining arbitrary levels of security and compartmentalization.</p> <p>For automating tasks that every time involve the same data and queries, RDF has no intrinsic superiority. Thus the user interfaces in places where RDF will have real edge must be more capable of <i>ad hoc</i> viewing and navigation than regular real-time or line of business user interfaces.</p> <p>The <a href="http://ode.openlinksw.com/" id="link-id0x2083a6f0">OpenLink Data Explorer</a> idea of a "data behind the web page" view goes in this direction. Read the web as before, then hit a switch to go to the data view. There are and will be separate clarifications and demos about this.</p> <h3>"What of the proliferation of standards? Does this not look too tangled, no clear identity? How would one know where to begin?"</h3> <p>When <a href="http://www.w3.org/2001/sw/sweo/" id="link-id0x1e8eac68">SWEO</a> was beginning, there was an endlessly protracted discussion of the so-called layer cake. This acronym jungle is not good messaging. Just say linked, flexibly repurpose-able data, and rich vocabularies and structure. Just the right amount of structure for the application, less rigid and easier to change than relational.</p> <p>Do not even mention the different serialization formats. Just say that it fits on top of the accepted web infrastructure â <a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x1e3806b8">HTTP</a>, URIs, and <a href="http://dbpedia.org/resource/XML" id="link-id0x1f547288">XML</a> where desired.</p> <p>It is misleading to say inference is a box at some specific place in the diagram. Inference of different types may or may not take place at diverse points, whether presentation or storage, on demand or as a preprocessing step. Since there is structure and semantics, inference is possible if desired.</p> <h3>"Can I make a social network application in RDF only, with no <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x20553ee0">RDBMS</a>?"</h3> <p>Yes, in principle, but what do you have in mind? The answer is very context dependent. The person posing the question had an E-learning system in mind, with things such as course catalogues, course material, etc. In such a case, RDF is a great match, especially since the user count will not be in the millions. No university has that many students and anyway they do not hang online browsing the course catalogue.</p> <p>On the other hand, if I think of making a social network site with RDF as the exclusive data model, I see things that would be very inefficient. For example, keeping a count of logins or the last time of login would be by default several times less efficient than with a RDBMS.</p> <p>If some application is really large scale and has a knowable workload profile, like any social network does, then some task-specific data structure is simply economical. This does not mean that the application language cannot be SPARQL but this means that the storage format must be tuned to favor some operations over others, relational style. This is a matter of cost more than of feasibility. Ten servers cost less than a hundred and have failures ten times less frequently.</p> <p>In the near term we will see the birth of an application paradigm for the data web. The data will be open, exposed, first-class citizen; yet the user experience will not have to be in a 1:1 image of the data.</p>
State of the Semantic Web, Part 1 - Sociology, Business, and Messaging (update 2)
http://www.openlinksw.com/blog/vdb/blog/?date=2008-10-24#1460
2008-10-24T10:19:03Z
2008-10-27T11:28:07-04:00
<p>I was in <a href="http://dbpedia.org/resource/Vienna" id="link-id0x1f18a540">Vienna</a> for the <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x1ec788a0">Linked Data</a> Practitioners gathering this week. Danny Ayers asked me if I would <a href="http://dbpedia.org/resource/Blog" id="link-id0x20838238">blog</a> about the State of the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x20694ed8">Semantic Web</a> or write the <i>This Week's Semantic Web</i> column. I don't have the time to cover all that may have happened during the past week but I will editorialize about the questions that again were raised in Vienna. How these things relate to <a href="http://virtuoso.openlinksw.com" id="link-id0x20b1cd38">Virtuoso</a> will be covered separately. This is about the overarching questions of the times, not the finer points of geek craft.</p> <p> <a href="http://www.informatik.uni-leipzig.de/~auer/foaf.rdf#me" id="link-id0x1ff31b30">Sören Auer</a> asked me to say a few things about relational to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1f8118e0">RDF</a> mapping. I will cite some highlights from this, as they pertain to the general scene. There was an "open hacking" session Wednesday night featuring lightning talks. I will use some of these too as a starting point.</p> <h3>The messaging?</h3> <p>The <a href="http://www.w3.org/2001/sw/sweo/" id="link-id0x1dc39210">SWEO</a> (Semantic Web Education and Outreach) interest group of the W3C spent some time looking for an elevator pitch for the Semantic Web. It became "<a href="http://dbpedia.org/resource/Data" id="link-id0x1f24dd98">Data</a> Unleashed." Why not? Let's give this some context.</p> <p>So, if we are holding a <i>Semantic Web 101</i> session, where should we begin? I hazard to guess that we should not begin by writing a FOAF file in Turtle by hand, as this is one thing that is not likely to happen in the real world.</p> <p>Of course, the social aspect of the Data Web is the most immediately engaging, so a demo might be to go make an account with <a href="http://myopenlink.net/" id="link-id0x1f5e0198">myopenlink</a>.<a href="http://dbpedia.org/resource/.NET_Framework" id="link-id0x1ec49a00">net</a> and see that after one has entered the data one normally enters for any social network, one has become a Data Web citizen. This means that one can be found, just like this, with a query against the set of data spaces hosted on the system. Then we just need a few pages that repurpose this data and relate it to other data. We show some samples of queries like this in our <a href="http://challenge.semanticweb.org/" id="link-id0x1ee35f70">Billion Triples Challenge</a> demo. We will make a webcast about this to make it all clearer.</p> <p>Behold: The Data Web is about the world becoming a database; writing <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x20644808">SPARQL</a> queries or triples is incidental. You will write FOAF files by hand just as little as you now write <a href="http://dbpedia.org/resource/SQL" id="link-id0x1fd9fbc0">SQL</a> insert statements for filling in your account <a href="http://dbpedia.org/resource/Information" id="link-id0x1dfd3540">information</a> on Myspace.</p> <p>Every time there is a major shift in technology, this shift needs to be motivated by addressing a new class of problem. This means doing something that could not be done before. The last time this happened was when the relational database became the dominant IT technology. At that time, the questions involved putting the enterprise in the database and building a cluster of Line Of Business (LOB) applications around the database. The argument for the <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x1e920868">RDBMS</a> was that you did not have to constrain the set of queries that might later be made, when designing the database. In other words, it was making things more <i>ad hoc</i>. This was opposed then on grounds of being less efficient than the hierarchical and network databases which the relational eventually replaced.</p> <p>Today, the point of the Data Web is that you do not have to constrain what your data can join or integrate with, when you design your database. The counter-argument is that this is slow and geeky and not scalable. See the similarity?</p> <p>A difference is that we are not specifically aiming at replacing the RDBMS. In fact, if you know exactly what you will query and have a well defined workload, a relational representation optimized for the workload will give you about 10x the performance of the equivalent RDF warehouse. OLTP remains a relational-only domain.</p> <p>However, when we are talking about doing queries and analytics against the Web, or even against more than a handful of relational systems, the things which make RDBMS good become problematic.</p> <h3>What is the business value of this?</h3> <p>The most reliable of human drives is the drive to make oneself known. This drives all, from any social scene to business communications to politics. Today, when you want to proclaim you exist, you do so first on the Web. The Web did not become the prevalent media because business loved it for its own sake, it became prevalent because business could not afford not to assert their presence there. If anything, the Web eroded the communications dominance of a lot of players, which was not welcome but still had to be dealt with, by embracing the Web.</p> <p>Today, in a world driven by data, the Data Web will be catalyzed by similar factors: If your data is not there, you will not figure in query results. Search engines will play some role there but also many social applications will have reports that are driven by published data. Also consider any e-commerce, any marketplace, and so forth. The Data Portability movement is a case in point: Users want to own their own content; silo operators want to capitalize on holding it. Right now, we see these things in silos; the Data Web will create bridges between these, and what is now in silo data centers will be increasingly available on an ad hoc basis with Open Data.</p> <p>Again, we see a movement from the specialized to the generic: What LinkedIn does in its data center can be done with ad hoc queries with <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1e715138">linked open data</a>. Of course, LinkedIn does these things somewhat more efficiently because their system is built just for this task, but the linked data approach has the built-in readiness to join with everything else at almost no cost, without making a new data warehouse for each new business question.</p> <p>We could call this the sociological aspect of the thing. Getting to more concrete business, we see an economy that, we could say, without being alarmists, is confronted with some issues. Well, generally when times are bad, this results in consolidation of property and power. Businesses fail and get split up and sold off in pieces, government adds controls and regulations and so forth. This means ad hoc data integration, as control without data is just pretense. If times are lean, this also means that there is little readiness to do wholesale replacement of systems, which will take years before producing anything. So we must play with what there is and make it deliver, in ways and conditions that were not necessarily anticipated. The agility of the Data Web, if correctly understood, can be of great benefit there, especially on the reporting and business intelligence side. Specifically mapping line-of-business systems into RDF on the fly will help with integration, making the specialized warehouse the slower and more expensive alternative. But this too is needed at times.</p> <p>But for the RDF community to be taken seriously there, the messaging must be geared in this direction. Writing FOAF files by hand is not where you begin the pitch. Well, what is more natural then having a global, queriable information space, when you have a global information driven economy?</p> <p>The Data Web is about making this happen. First with doing this in published generally available data; next with the enterprises having their private data for their own use but still linking toward the outside, even though private data stays private: You can still use standard terms and taxonomies, where they apply, when talking of proprietary information.</p> <h3>But let's get back to more specific issues</h3> <p>At the lightning talks in Vienna, one participant said, "Man's enemy is not the lion that eats men, it's his own brother. Semantic Web's enemy is the <a href="http://dbpedia.org/resource/XML" id="link-id0x1aeb61b8">XML</a> Web services stack that ate its lunch." There is some truth to the first part. The second part deserves some comment. The Web services stack is about transactions. When you have a fixed, often repeating task, it is a natural thing to make this a Web service. Even though SOA is not really prevalent in enterprise IT, it has value in things like managing supply-chain logistics with partners, etc. Lots of standard messages with unambiguous meaning. To make a parallel with the database world: first there was OLTP; then there was business intelligence. Of course, you must first have the transactions, to have something to analyze.</p> <p>SOA is for the transactions; the Data Web is for integration, analysis, and discovery. It is the <i>ad hoc</i> component of the real time enterprise, if you will. It is not a competitor against a transaction oriented SOA. In fact, RDF has no special genius for transactions. Another mistake that often gets made is stretching things beyond their natural niche. Doing transactions in RDF is this sort of over-stretching without real benefit.</p> <p>"I made an ontology and it really did solve a problem. How do I convince the enterprise people, the MBA who says it's too complex, the developer who says it is not what he's used to, and so on?"</p> <p>This is an education question. One of the findings of SWEO's enterprise survey was that there was awareness that difficult problems existed. There were and are corporate ontologies and taxonomies, diversely implemented. Some of these needs are recognized. RDF based technologies offer to make these more open standards based. open standards have proven economical in the past. What we also hear is that major enterprises do not even know what their information and human resources assets are: Experts can't be found even when they are in the next department, or reports and analysis gets buried in wikis, spreadsheets, and emails.</p> <p>Just as when SQL took off, we need vendors to do workshops on getting started with a technology. The affair in Vienna was a step in this direction. Another type of event specially focusing on vertical problems and their Data Web solutions is a next step. For example, one could do a workshop on integrating supply chain information with Data Web technologies. Or one on making enterprise <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1fbd3398">knowledge</a> bases from HR, CRM, office automation, wikis, etc. The good thing is that all these things are additions to, not replacements of, the existing mission-critical infrastructure. And better use of what you already have ought to be the theme of the day.</p>
Virtuoso Cluster Paper Update
http://www.openlinksw.com/blog/vdb/blog/?date=2008-10-02#1451
2008-10-02T10:02:33Z
2008-10-03T04:38:06-04:00
<p>An updated version of the paper about <a href="http://virtuoso.openlinksw.com" id="link-id0xc0abc50">Virtuoso</a> Cluster is available at <a href="http://www.openlinksw.com/weblog/oerling/2008webscale_rdf.pdf" id="link-id16459248">2008webscale_rdf.pdf</a> </p>
Virtuoso Update, Billion Triples and Outlook
http://www.openlinksw.com/blog/vdb/blog/?date=2008-10-02#1450
2008-10-02T10:02:32Z
2008-10-02T12:47:07.000004-04:00
<div> <div style="display:none;">Virtuoso Update, Billion Triples and Outlook</div> <p>I will say a few things about what we have been doing and where we can go.</p> <p>Firstly, we have a fairly scalable platform with <a href="http://virtuoso.openlinksw.com" id="link-id0x1aa82dc0">Virtuoso</a> 6 Cluster. It was most recently tested with the workload discussed in the previous <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1445" id="link-id1638a5b8">Billion Triples post</a>.</p> <p>There is an updated version of <a href="http://www.openlinksw.com/weblog/oerling/2008webscale_rdf.pdf" id="link-id16280a68">the paper about this</a>. This will be presented at the web scale workshop of ISWC 2008 in Karlsruhe.</p> <p>Right now, we are polishing some things in Virtuoso 6 -- some optimizations for smarter balancing of interconnect traffic over multiple network interfaces, and some more <a href="http://dbpedia.org/resource/SQL" id="link-id0x1abd3f38">SQL</a> optimizations specific to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1adbe410">RDF</a>. The must-have basics, like parallel running of sub-queries and aggregates, and all-around unrolling of loops of every kind into large partitioned batches, is all there and proven to work.</p> <p>We spent a lot of time around the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1aaa0e78">Berlin SPARQL Benchmark</a> story, so we got to the more advanced stuff like the <a href="http://challenge.semanticweb.org/" id="link-id0x1a860a50">Billion Triples Challenge</a> rather late. We did along the way also run <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1a27f2a8">BSBM</a> with an <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x1ad5c918">Oracle</a> back-end, with Virtuoso mapping <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1cf0e4a0">SPARQL</a> to SQL. This merits its own analysis in the near future. This will be the basic how-to of mapping OLTP systems to RDF. Depending on the case, one can use this for lookups in real-time or ETL.</p> <p>RDF will deliver value in complex situations. An example of a complex relational mapping use case came from Ordnance Survey, presented at the <a href="http://www.w3.org/2005/Incubator/rdb2rdf/" id="link-id0x1ab96bb0">RDB2RDF XG</a>. Examples of complex warehouses include the <a href="http://neurocommons.org/page/Main_Page" id="link-id0x1adb2db0">Neurocommons</a> database, the Billion Triples Challenge, and the <a href="http://www.garlik.com/" id="link-id0x1925c7b0">Garlik DataPatrol</a>.</p> <p>In comparison, the Berlin workload is really simple and one where RDF is not at its best, as amply discussed on the <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x1c6d1480">Linked Data</a> forum. BSBM's primary value is as a demonstrator for the basic mapping tasks that will be repeated over and over for pretty much any online system when presence on the <a href="http://dbpedia.org/resource/Data" id="link-id0x1a937400">data</a> web becomes as indispensable as presence on the HTML web.</p> <p>I will now talk about the complex warehouse/web-harvesting side. I will come to the mapping in another post.</p> <p>Now, all the things shown in the <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1445" id="link-id14de1d18">Billion Triples post</a> can be done with a relational system specially built for each purpose. Since we are a general purpose <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x1a457c70">RDBMS</a>, we use this capability where it makes sense. For example, storing statistics about which tags or interests occur with which other tags or interests as RDF blank nodes makes no sense. We do not even make the experiment; we know ahead of time that the result is at least an order of magnitude in favor of the relational row-oriented solution in both space and time.</p> <p>Whenever there is a data structure specially made for answering one specific question, like joint occurrence of tags, RDB and mapping is the way to go. With Virtuoso, this can fully-well coexist with physical triples, and can still be accessed in SPARQL and mixed with triples. This is territory that we have not extensively covered yet, but we will be giving some examples about this later.</p> <p>The real value of RDF is in agility. When there is no time to design and load a new warehouse for every new question, RDF is unparalleled. Also SPARQL, once it has the necessary extensions of aggregating and sub-queries, is nicer than SQL, especially when we have sub-classes and sub-properties, transitivity, and "same as" enabled. These things have some run time cost and if there is a report one is hitting absolutely all the time, then chances are that resolving terms and identity at load-time and using materialized views in SQL is the reasonable thing. If one is inventing a new report every time, then RDF has a lot more convenience and flexibility.</p> <p>We are just beginning to explore what we can do with data sets such as the online conversation space, linked data, and the open ontologies of <a href="http://umbel.org/about/" id="link-id0x1aa5ea18">UMBEL</a> and <a href="http://dbpedia.org/resource/Cyc" id="link-id0x1a631a20">OpenCyc</a>. It is safe to say that we can run with real world scale without loss of query expressivity. There is an incremental cost for performance but this is not prohibitive. Serving the whole billion triples set from memory would cost about $32K in hardware. $8K will do if one can wait for disk part of the time. One can use these numbers as a basis for costing larger systems. For online search applications, one will note that running the indexes pretty much from memory is necessary for flat response time. For back office analytics this is not necessarily as critical. It all depends on the use case.</p> <p>We expect to be able to combine geography, social proximity, subject matter, and <a href="http://dbpedia.org/resource/Named_entity_recognition" id="link-id0x1aebdcc8">named entities</a>, with hierarchical taxonomies and traditional full text, and to present this through a simple user interface.</p> <p>We expect to do this with online response times if we have a limited set of starting points and do not navigate more than 2 or 3 steps from each starting point. An example would be to have a full text pattern and news group, and get the cloud of interests from the authors of matching posts. Another would be to make a faceted view of the properties of the 1000 people most closely connected to one person.</p> <p>Queries like finding the fastest online responders to questions about romance across the global board-scape, or finding the person who initiates the most long running conversations about crime, take a bit longer but are entirely possible.</p> <p>The genius of RDF is to be able to do these things within a general purpose database, ad hoc, in a single query language, mostly without materializing intermediate results. Any of these things could be done with arbitrary efficiency in a custom built system. But what is special now is that the cost of access to this type of <a href="http://dbpedia.org/resource/Information" id="link-id0x1ab88490">information</a> and far beyond drops dramatically as we can do these things in a far less labor intensive way, with a general purpose system, with no redesigning and reloading of warehouses at every turn. The query becomes a commodity.</p> <p>Still, one must know what to ask. In this respect, the self-describing nature of RDF is unmatched. A query like <i>list the top 10 attributes with the most distinct values for all persons</i> cannot be done in SQL. SQL simply does not allow the columns to be variable.</p> <p>Further, we can accept queries as text, the way people are used to supplying them, and use structure for drill-down or result-relevance, and also recognize named entities and subject matter concepts in query text. Very simple NLP will go a long way towards keeping SPARQL out of the user experience.</p> <p>The other way of keeping query complexity hidden is to publish hand-written SPARQL as parameter-fed canned reports.</p> <p>Between now and ISWC 2008, the last week of October, we will put out demos showing some of these things. Stay tuned.</p> </div>
OpenLink Software's Virtuoso Submission to the Billion Triples Challenge
http://www.openlinksw.com/blog/vdb/blog/?date=2008-09-30#1446
2008-09-30T16:24:34Z
2008-10-03T06:20:48.000094-04:00
<div> <h2>Introduction</h2> <p>We use <a href="http://virtuoso.openlinksw.com" id="link-id0xb03e418">Virtuoso</a> 6 Cluster Edition to demonstrate the following:</p> <ul> <li>Text and structured <a href="http://dbpedia.org/resource/Information" id="link-id0xbd9dae8">information</a> based lookups</li> <li>Analytics queries</li> <li>Analysis of co-occurrence of features like interests and tags.</li> <li>Dealing with identity of multiple IRI's (<a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0xb383dd8">owl</a>:sameAs)</li> </ul> <p>The demo is based on a set of canned <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xbda6298">SPARQL</a> queries that can be invoked using the <a href="http://ode.openlinksw.com/" id="link-id0xbb292f0">OpenLink Data Explorer</a> (<a href="http://ode.openlinksw.com/" id="link-id0xc263528">ODE</a>) Firefox extension.</p> <p>The demo queries can also be run directly against the SPARQL end point.</p> <p>The demo is being worked on at the time of submission and may be shown online by appointment.</p> <p>Automatic annotation of the <a href="http://dbpedia.org/resource/Data" id="link-id0xa173378">data</a> based on <a href="http://dbpedia.org/resource/Named_entity_recognition" id="link-id0xbdda558">named entity extraction</a> is being worked on at the time of this submission. By the time of ISWC 2008 the set of sample queries will be enhanced with queries based on extracted <a href="http://dbpedia.org/resource/Named_entity_recognition" id="link-id0xa66fbe0">named entities</a> and their relationships in the <a href="http://umbel.org/about/" id="link-id0xa06e2c8">UMBEL</a> and Open CYC ontologies. </p> <p>Also examples involving owl:sameAs are being added, likewise with similarity metrics and search hit scores.</p> <h2>The Data</h2> <p>The database consists of the billion triples data sets and some additions like Umbel. Also the Freebase extract is newer than the challenge original.</p> <p>The triple count is 1115 million.</p> <p>In the case of web harvested resources, the data is loaded in one graph per resource.</p> <p>In the case of larger data sets like <a href="http://dbpedia.org/resource/DBpedia" id="link-id0xc2bf770">Dbpedia</a> or the US census, all triples of the provenance share a data set specific graph.</p> <p>All string literals are additionally indexed in a full text index. No stop words are used.</p> <p>Most queries do not specify a graph. Thus they are evaluated against the union of all the graphs in the database. The indexing scheme is SPOG, GPOS, POGS, OPGS. All indices ending in S are bitmap indices. </p> <h2>The Queries </h2> <p>The demo uses Virtuoso SPARQL extensions in most queries. These extensions consist on one hand of well known <a href="http://dbpedia.org/resource/SQL" id="link-id0xaf8cb40">SQL</a> features like aggregation with grouping and existence and value subqueries and on the other of <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xafdceb8">RDF</a> specific features. The latter include run time RDFS and OWL inferencing support and backward chaining subclasses and transitivity. </p> <h3>Simple Lookups</h3> <pre>sparql select ?s ?p (bif:search_excerpt (bif:vector ('<a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0xbb64dd0">semantic', 'web</a>'), ?o)) where { ?s ?p ?o . filter (bif:contains (?o, "'semantic web'")) } limit 10 ; </pre> <p>This looks up triples with semantic web in the object and makes a search hit summary of the literal, highlighting the search terms. </p> <pre>sparql select ?tp count(*) where { ?s ?p2 ?o2 . ?o2 a ?tp . ?s foaf:nick ?o . filter (bif:contains (?o, "plaid_skirt")) } group by ?tp order by desc 2 limit 40 ; </pre> <p>This looks at what sorts of things are referenced by the properties of the foaf handle plaid_skirt.</p> <p>What are these things called?</p> <pre>sparql select ?lbl count(*) where { ?s ?p2 ?o2 . ?o2 rdfs:label ?lbl . ?s foaf:nick ?o . filter (bif:contains (?o, "plaid_skirt")) } group by ?lbl order by desc 2 ; </pre> <p>Many of these things do not have a rdfs:label. Let us use a more general concept of lable which groups dc:title, foaf:name and other name-like properties together. The subproperties are resolved at run time, there is no materialization. </p> <pre>sparql define input:inference 'b3s' select ?lbl count(*) where { ?s ?p2 ?o2 . ?o2 b3s:label ?lbl . ?s foaf:nick ?o . filter (bif:contains (?o, "plaid_skirt")) } group by ?lbl order by desc 2 ; </pre> <p>We can list sources by the topics they contain. Below we look for graphs that mention terrorist bombing. </p> <pre>sparql select ?g count(*) where { graph ?g { ?s ?p ?o . filter (bif:contains (?o, "'terrorist bombing'")) } } group by ?g order by desc 2 ; </pre> <p>Now some web 2.0 tagging of search results. The <a href="http://dbpedia.org/resource/Tag" id="link-id0xa8b89f8">tag</a> cloud of "computer"</p> <pre>sparql select ?lbl count (*) where { ?s ?p ?o . ?o bif:contains "computer" . ?s sioc:topic ?tg . optional { ?tg rdfs:label ?lbl } } group by ?lbl order by desc 2 limit 40 ; </pre> <p>This query will find the posters who talk the most about sex.</p> <pre>sparql select ?auth count (*) where { ?d dc:creator ?auth . ?d ?p ?o filter (bif:contains (?o, "sex")) } group by ?auth order by desc 2 ; </pre> <h3>Analytics </h3> <p>We look for people who are joined by having relatively uncommon interests but do not know each other.</p> <pre>sparql select ?i ?cnt ?n1 ?n2 ?p1 ?p2 where { { select ?i count (*) as ?cnt where { ?p foaf:interest ?i } group by ?i } filter ( ?cnt > 1 && ?cnt < 10) . ?p1 foaf:interest ?i . ?p2 foaf:interest ?i . filter (?p1 != ?p2 && !bif:exists ((select (1) where {?p1 foaf:knows ?p2 })) && !bif:exists ((select (1) where {?p2 foaf:knows ?p1 }))) . ?p1 foaf:nick ?n1 . ?p2 foaf:nick ?n2 . } order by ?cnt limit 50 ; </pre> <p>The query takes a fairly long time, mostly spent counting the interested in 25M interest triples. It then takes people that share the interest and checks that neither claims to know the other. It then sorts the results rarest interest first. The query can be written more efficently but is here just to show that database-wide scans of the population are possible ad hoc. </p> <p>Now we go to SQL to make a tag co-occurrence matrix. This can be used for showing a Technorati-style related tags line at the bottom of a search result page. This showcases the use of SQL together with SPARQL. The half-matrix of tags t1, t2 with the co-occurrence count at the intersection is much more efficiently done in SQL, specially since it gets updated as the data changes. This is an example of materialized intermediate results based on warehoused RDF. </p> <pre>create table tag_count (tcn_tag iri_id_8, tcn_count int, primary key (tcn_tag)); alter index tag_count on tag_count partition (tcn_tag int (0hexffff00)); create table tag_coincidence (tc_t1 iri_id_8, tc_t2 iri_id_8, tc_count int, tc_t1_count int, tc_t2_count int, primary key (tc_t1, tc_t2)) alter index tag_coincidence on tag_coincidence partition (tc_t1 int (0hexffff00)); create index tc2 on tag_coincidence (tc_t2, tc_t1) partition (tc_t2 int (0hexffff00)); </pre> <p>How many times each topic is mentioned?</p> <pre> insert into tag_count select * from (sparql define output:valmode "LONG" select ?t count (*) as ?cnt where { ?s sioc:topic ?t } group by ?t) xx option (quietcast); </pre> <p>Take all t1, t2 where t1 and t2 are tags of the same subject, store only the permutation where the internal id of t1 < that of t2.</p> <pre>insert into tag_coincidence (tc_t1, tc_t2, tc_count) select "t1", "t2", cnt from (select "t1", "t2", count (*) as cnt from (sparql define output:valmode "LONG" select ?t1 ?t2 where { ?s sioc:topic ?t1 . ?s sioc:topic ?t2 }) tags where "t1" < "t2" group by "t1", "t2") xx where isiri_id ("t1") and isiri_id ("t2") option (quietcast); </pre> <p>Now put the individual occurrence counts into the same table with the co-occurrence. This denormalization makes the related tags lookup faster. </p> <pre>update tag_coincidence set tc_t1_count = (select tcn_count from tag_count where tcn_tag = tc_t1), tc_t2_count = (select tcn_count from tag_count where tcn_tag = tc_t2); </pre> <p>Now each tag_coincidence row has the joint occurrence count and individual occurrence counts. A single select will return a Technorati-style related tags listing. </p> <p>To show the <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x9d4bc60">URI</a>'s of the tags: </p> <pre>select top 10 id_to_iri (tc_T1), id_to_iri (tc_t2), tc_count from tag_coincidence order by tc_count desc; </pre> <h3>Social Networks </h3> <p>We look at what interests people have </p> <pre>sparql select ?o ?cnt where { { select ?o count (*) as ?cnt where { ?s foaf:interest ?o } group by ?o } filter (?cnt > 100) } order by desc 2 limit 100 ; </pre> <p>Now the same for the Harry Potter fans </p> <pre>sparql select ?i2 count (*) where { ?p foaf:interest <<a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0xba0b390">http</a>://www.livejournal.com/interests.bml?int=harry+potter> . ?p foaf:interest ?i2 } group by ?i2 order by desc 2 limit 20 ; </pre> <p>We see whether knows relations are symmmetrical. We return the top n people that others claim to know without being reciprocally known.</p> <pre>sparql select ?celeb, count (*) where { ?claimant foaf:knows ?celeb . filter (!bif:exists ((select (1) where { ?celeb foaf:knows ?claimant }))) } group by ?celeb order by desc 2 limit 10 ; </pre> <p>We look for a well connected person to start from.</p> <pre>sparql select ?p count (*) where { ?p foaf:knows ?k } group by ?p order by desc 2 limit 50 ; </pre> <p>We look for the most connected of the many online identities of Stefan Decker.</p> <pre>sparql select ?sd count (distinct ?xx) where { ?sd a foaf:Person . ?sd ?name ?ns . filter (bif:contains (?ns, "'Stefan Decker'")) . ?sd foaf:knows ?xx } group by ?sd order by desc 2 ; </pre> <p>We count the transitive closure of Stefan Decker's connections </p> <pre>sparql select count (*) where { { select * where { ?s foaf:knows ?o } } option (transitive, t_distinct, t_in(?s), t_out(?o)) . filter (?s = <mailto:stefan.decker@deri.org>) } ; </pre> <p>Now we do the same while following owl:sameAs links.</p> <pre>sparql define input:same-as "yes" select count (*) where { { select * where { ?s foaf:knows ?o } } option (transitive, t_distinct, t_in(?s), t_out(?o)) . filter (?s = <mailto:stefan.decker@deri.org>) } ; </pre> <h2>Demo System</h2> <p>The system runs on Virtuoso 6 Cluster Edition. The database is partitioned into 12 partitions, each served by a distinct server process. The system demonstrated hosts these 12 servers on 2 machines, each with 2 xXeon 5345 and 16GB memory and 4 SATA disks. For scaling, the processes and corresponding partitions can be spread over a larger number of machines. If each ran on its own server with 16GB RAM, the whole data set could be served from memory. This is desirable for search engine or fast analytics applications. Most of the demonstrated queries run in memory on second invocation. The timing difference between first and second run is easily an order of magnitude. </p> </div>
Requirements for Relational-to-RDF Mapping
http://www.openlinksw.com/blog/vdb/blog/?date=2008-09-08#1436
2008-09-08T09:41:25Z
2008-09-08T15:03:09-04:00
<div> <div style="display:none;">Requirements for Relational-to-RDF Mapping</div> <p>Many of you will know about the W3C relational-to-<a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1e1be0a8">RDF</a> mapping incubator activity. The group is planning to suggest forming a working group for drawing up a specification for relational-to-RDF mapping.</p> <p>To this effect, I recently summarized the group discussions and some of our own experiences around the topic at <<a href="http://esw.w3.org/topic/Rdb2RdfXG/ReqForMappingByOErling" id="link-id146030e8">http://esw.w3.org/topic/Rdb2RdfXG/ReqForMappingByOErling</a>>.</p> <p>I will here discuss this less formally and more in the light of our own experience. A working group goal statement must be neutral vis à vis the following points, even if any working group will unavoidably encounter these issues on the way. A <a href="http://dbpedia.org/resource/Blog" id="link-id0x1e6b3950">blog</a> post on the other hand can be more specific.</p> <p>I gave a talk to the <a href="http://www.w3.org/2005/Incubator/rdb2rdf/" id="link-id0xa0932c68">RDB2RDF XG</a> this spring, with these <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VirtPresentations/Relational2RDF.ppt" id="link-id14572540">slides</a>.</p> <p>The main point is that people would really like to map on-the-fly, if they only could. Making an RDF warehouse is not of value in itself, but it is true that in some cases this cannot be avoided.</p> <p>At first sight, one would think that a mapping specification could be neutral as regards whether one stores the mapped triples as triples or makes them on demand. There is almost no comparison between the complexity of doing non-trivial mappings on-the-fly versus mapping as ETL. Some of this complexity spills over into the requirements for a mapping language. </p> <h2>Eliminating JOINs</h2> <p>We expect to have a situation where one virtual triple can have many possible sources. The mapping is a union of mapped databases. Any integration scenario will have this feature. In such a situation, if we are <code>JOIN</code>ing using such triples, we end up with <code>UNION</code>s of all databases that could produce the triples in question. This is generally not desired. Therefore, in the on-demand mapping case, there must be a lot of type inference logic that is not relevant in the ETL scenario.</p> <p>To make the point clearer, suppose a query like "list the organizations whose representatives have published about <i>xx</i>." Suppose that there are three databases mapped, all of which have a table of organizations, a table of persons with affiliation to organizations, a table of publications by these persons, and finally a table of tags for the publications. Now, we want the laboratories that have published with articles with <a href="http://dbpedia.org/resource/Tag" id="link-id0xa0977bf0">tag</a> <i>XX</i>. It is a matter of common sense in this scenario that a publication will have the author and the author's affiliation in the same database. However, the RDB-to-RDF mapping does not necessarily know this, if all that it is told is that a table makes IRIs of publications by applying a certain pattern to the primary key of the publications table. To infer what needs to be inferred, the system must realize that IRIs from one mapping are disjoint from IRIs from another: A paper in database <i>X</i> will usually not have an author in database <i>Y</i>. The IDs in database <i>Y</i>, even if perchance equal to the IDs in <i>X</i>, do not mean the same thing, and there is no point joining across databases by them.</p> <p>This entire question is a non-issue in the ETL scenario, but is absolutely vital in the real-time mapping. This is also something that must be stated, at least implicitly, in any mapping. If a mapping translates keys of one place to IRIs with one pattern, and keys from another using another pattern, it must be inferable from the patterns whether the sets of IRIs will be disjoint.</p> <p>This is critical. Otherwise we will be joining everything to everything else, and there will be orders of magnitude of penalty compared to hand-crafted <a href="http://dbpedia.org/resource/SQL" id="link-id0xa09490f8">SQL</a> over the same <a href="http://dbpedia.org/resource/Data" id="link-id0xa095efd0">data</a> sources.</p> <h2>Expectations and Limitations on Queries</h2> <p> <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1e360230">SPARQL</a> queries translate quite well to SQL when there is only one table that can produce a triple with a subject of a given class, when there are few columns that can map to a given predicate, and when classes and predicates are literals in the query.</p> <p> <a href="http://virtuoso.openlinksw.com" id="link-id0x1f5edb30">Virtuoso</a> has some SQL extensions for dealing with breaking a wide table into a row per column. This facilitates dealing with predicates that are not known at query compile time. If the table in question is not managed by Virtuoso, Virtuoso's SQL virtualization/federation takes care of the matter. If a mapping system goes directly to third-party SQL, no such tricks can be used.</p> <p>The above example suggests that for supporting on-the-fly mapping without relying on owning the SQL underneath, some subsets of SPARQL may have to be defined. For example, one will probably have to require that all predicates be literals. The alternative is prohibitive run-time cost and complexity.</p> <p>But we must not lose the baby with the bath-water. Aside from offering global identifiers, RDF's attractions include subclasses and sub-predicates. In relational terms, these translate to <code>UNION</code>s and do involve some added cost. A mapping system just has to have means of dealing with this cost, and of recognizing cases where this cost is prohibitive. Some further work is likely to be required for defining well-behaved subsets of SPARQL and mappings.</p> <h2>ETL Ou Ne Pas ETL?</h2> <p>Whether to warehouse or not? If one has hundreds of sources, of which some are not even relational, some ETL would seem necessary. Kashiup Vipul gave a position paper at last year's RDB-to-RDF mapping workshop in Cambridge, Massachusetts, about a system of relational mapping and on-demand RDF-izers of diverse semi-structured biomedical data, e.g., spreadsheets. The issue certainly exists, and any mapping work will likely encounter integration scenarios where one part is fairly neatly mapped from relational stores, and another part comes from a less structured repository of ETLed physical triples.</p> <p>Our take is that if something is a large or very large relational store, then map; else, ETL. With Virtuoso, we can mix mapped and local triples, but this is not a generally available feature of triple stores and standardization will likely have to wait until there are more implementations.</p> <h2>Conclusions</h2> <ul> <li>If you map on demand, watch out for an explosion of <code>UNION</code>s when integrating sources that talk of similar things.</li> <li>If you integrate lots of sources, some ETL is likely unavoidable. Look for ways of dealing with part ETL, part mapping. ETLing everything is not always best or even possible.</li> <li>If you map a single fairly-clean RDB to RDF, mapping will work well, potentially much faster than triple storage. Higher storage density and more data per index lookup on the relational side.</li> <li>If you map on demand, some restrictions to SPARQL may be practically necessary. These have to do with variables in predicate position, variables in class position, etc. Individual implementations may support these, but standardization will likely have to put limits on them.</li> </ul> <p>This was a quick summary, by no means comprehensive, on what an eventual RDB2RDF working group would come across. This is a sort of addendum to the requirements I outlined on the ESW wiki.</p> </div>
Transitivity and Graphs for SQL
http://www.openlinksw.com/blog/vdb/blog/?date=2008-09-08#1435
2008-09-08T09:41:24Z
2008-09-08T15:43:07-04:00
<div> <div style="display:none;">Transitivity and Graphs for SQL</div> <h2>Background</h2> <p>I have mentioned on a couple of prior occasions that basic graph operations ought to be integrated into the <a href="http://dbpedia.org/resource/SQL" id="link-id0xa1a18c58">SQL</a> query language.</p> <p>The history of databases is by and large about moving from specialized applications toward a generic platform. The introduction of the DBMS itself is the archetypal example. It is all about extracting the common features of applications and making these the features of a platform instead.</p> <p>It is now time to apply this principle to graph traversal.</p> <p>The rationale is that graph operations are somewhat tedious to write in a parallelize-able, latency-tolerant manner. Writing them as one would for memory-based <a href="http://dbpedia.org/resource/Data" id="link-id0xaf8c730">data</a> structures is easier but totally unscalable as soon as there is any latency involved, i.e., disk reads or messages between cluster peers.</p> <p>The ad-hoc nature and very large volume of <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xae41ef0">RDF</a> data makes this a timely question. Up until now, the answer to this question has been to materialize any implied facts in RDF stores. If <i>a</i> was part of <i>b</i>, and <i>b</i> part of <i><a href="http://dbpedia.org/resource/C_(programming_language)" id="link-id0xac9d8790">c</a></i>, the implied fact that <i>a</i> is part of <i>c</i> would be inserted explicitly into the database as a pre-query step.</p> <p>This is simple and often efficient, but tends to have the downside that one makes a specialized warehouse for each new type of query. The activity becomes less ad-hoc.</p> <p>Also, this becomes next to impossible when the scale approaches web scale, or if some of the data is liable to be on-and-off included-into or excluded-from the set being analyzed. This is why with <a href="http://virtuoso.openlinksw.com" id="link-id0xb68f9d0">Virtuoso</a> we have tended to favor inference on demand ("backward chaining") and mapping of relational data into RDF without copying.</p> <p>The SQL world has taken steps towards dealing with recursion with the <code>WITH - UNION</code> construct which allows definition of recursive views. The idea there is to define, for example, a tree walk as a <code>UNION</code> of the data of the starting node plus the recursive walk of the starting node's immediate children.</p> <p>The main problem with this is that I do not very well see how a SQL optimizer could effectively rearrange queries involving <code>JOIN</code>s between such recursive views. This model of recursion seems to lose SQL's non-procedural nature. One can no longer easily rearrange <code>JOIN</code>s based on what data is given and what is to be retrieved. If the recursion is written from root to leaf, it is not obvious how to do this from leaf to root. At any rate, queries written in this way are so complex to write, let alone optimize, that I decided to take another approach.</p> <p>Take a question like "list the parts of products of category <i>C</i> which have materials that are classified as toxic." Suppose that the product categories are a tree, the product parts are a tree, and the materials classification is a tree taxonomy where "toxic" has a multilevel substructure.</p> <p>Depending on the count of products and materials, the query can be evaluated as either going from products to parts to materials and then climbing up the materials tree to see if the material is toxic. Or one could do it in reverse, starting with the different toxic materials, looking up the parts containing these, going to the part tree to the product, and up the product hierarchy to see if the product is in the right category. One should be able to evaluate the identical query either way depending on what indices exist, what the cardinalities of the relations are, and so forth â regular cost based optimization.</p> <p>Especially with RDF, there are many problems of this type. In regular SQL, it is a long-standing cultural practice to flatten hierarchies, but this is not the case with RDF.</p> <p>In Virtuoso, we see <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xb3bdcc0">SPARQL</a> as reducing to SQL. Any RDF-oriented database-engine or query-optimization feature is accessed via SQL. Thus, if we address run-time-recursion in the Virtuoso query engine, this becomes, <i>ipso facto</i>, an SQL feature. Besides, we remember that SQL is a much more mature and expressive language than the current SPARQL recommendation.</p> <h2> SQL and Transitivity </h2> <p>We will here look at some simple social network queries. A later article will show how to do more general graph operations. We extend the SQL derived table construct, i.e., <code>SELECT</code> in another <code>SELECT</code>'s <code>FROM</code> clause, with a <code>TRANSITIVE</code> clause.</p> <p>Consider the data:</p> <blockquote> <pre><code>CREATE TABLE "knows" ("p1" INT, "p2" INT, PRIMARY KEY ("p1", "p2") ); ALTER INDEX "knows" ON "knows" PARTITION ("p1" INT); CREATE INDEX "knows2" ON "knows" ("p2", "p1") PARTITION ("p2" INT); </code> </pre></blockquote> <p>We represent a social network with the many-to-many relation "knows". The persons are identified by integers.</p> <blockquote> <pre><code>INSERT INTO "knows" VALUES (1, 2); INSERT INTO "knows" VALUES (1, 3); INSERT INTO "knows" VALUES (2, 4);</code> </pre> <pre><code>SELECT * FROM (SELECT TRANSITIVE T_IN (1) T_OUT (2) T_DISTINCT "p1", "p2" FROM "knows" ) "k" WHERE "k"."p1" = 1;</code></pre></blockquote> <p>We obtain the result:</p> <blockquote> <table width="100"> <tr> <th align="center" width="50">p1</th> <th align="center" width="50">p2</th> </tr> <tr> <td align="center">1</td> <td align="center">3</td> </tr> <tr> <td align="center">1</td> <td align="center">2</td> </tr> <tr> <td align="center">1</td> <td align="center">4</td> </tr> </table> </blockquote> <p>The operation is reversible:</p> <blockquote> <pre><code>SELECT * FROM (SELECT TRANSITIVE T_IN (1) T_OUT (2) T_DISTINCT "p1", "p2" FROM "knows" ) "k" WHERE "k"."p2" = 4; </code> </pre> <table width="100"> <tr> <th align="center" width="50">p1</th> <th align="center" width="50">p2</th> </tr> <tr> <td align="center">2</td> <td align="center">4</td> </tr> <tr> <td align="center">1</td> <td align="center">4</td> </tr> </table> </blockquote> <p>Since now we give <i>p2</i>, we traverse from <i>p2</i> towards <i>p1</i>. The result set states that 4 is known by 2 and 2 is known by 1.</p> <p>To see what would happen if <i>x</i> knowing <i>y</i> also meant <i>y</i> knowing <i>x</i>, one could write:</p> <blockquote> <pre><code>SELECT * FROM (SELECT TRANSITIVE T_IN (1) T_OUT (2) T_DISTINCT "p1", "p2" FROM (SELECT "p1", "p2" FROM "knows" UNION ALL SELECT "p2", "p1" FROM "knows" ) "k2" ) "k" WHERE "k"."p2" = 4;</code> </pre> <table width="100"> <tr> <th align="center" width="50">p1</th> <th align="center" width="50">p2</th> </tr> <tr> <td align="center">2</td> <td align="center">4</td> </tr> <tr> <td align="center">1</td> <td align="center">4</td> </tr> <tr> <td align="center">3</td> <td align="center">4</td> </tr> </table> </blockquote> <p>Now, since we know that 1 and 4 are related, we can ask how they are related.</p> <blockquote> <pre><code>SELECT * FROM (SELECT TRANSITIVE T_IN (1) T_OUT (2) T_DISTINCT "p1", "p2", T_STEP (1) AS "via", T_STEP ('step_no') AS "step", T_STEP ('path_id') AS "path" FROM "knows" ) "k" WHERE "p1" = 1 AND "p2" = 4;</code> </pre> <table width="250"> <tr> <th align="center" width="50">p1</th> <th align="center" width="50">p2</th> <th align="center" width="50">via</th> <th align="center" width="50">step</th> <th align="center" width="50">path</th> </tr> <tr> <td align="center">1</td> <td align="center">4</td> <td align="center">1</td> <td align="center">0</td> <td align="center">0</td> </tr> <tr> <td align="center">1</td> <td align="center">4</td> <td align="center">2</td> <td align="center">1</td> <td align="center">0</td> </tr> <tr> <td align="center">1</td> <td align="center">4</td> <td align="center">4</td> <td align="center">2</td> <td align="center">0</td> </tr> </table> </blockquote> <p>The two first columns are the ends of the path. The next column is the person that is a step on the path. The next one is the number of the step, counting from 0, so that the end of the path that corresponds to the end condition on the column designated as input, i.e., <i>p1</i>, has number 0. Since there can be multiple solutions, the last column is a sequence number allowing distinguishing multiple alternative paths from each other.</p> <p>For LinkedIn users, the friends ordered by distance and descending friend count query, which is at the basis of most LinkedIn search result views can be written as: </p> <blockquote> <pre><code>SELECT p2, dist, (SELECT COUNT (*) FROM "knows" "c" WHERE "c"."p1" = "k"."p2" ) FROM (SELECT TRANSITIVE t_in (1) t_out (2) t_distinct "p1", "p2", t_step ('step_no') AS "dist" FROM "knows" ) "k" WHERE "p1" = 1 ORDER BY "dist", 3 DESC;</code> </pre> <table width="150"> <tr> <th align="center" width="50">p2</th> <th align="center" width="50">dist</th> <th align="center" width="50">aggregate</th> </tr> <tr> <td align="center">2</td> <td align="center">1</td> <td align="center">1</td> </tr> <tr> <td align="center">3</td> <td align="center">1</td> <td align="center">0</td> </tr> <tr> <td align="center">4</td> <td align="center">2</td> <td align="center">0</td> </tr> </table> </blockquote> <h2>How?</h2> <p>The queries shown above work on Virtuoso v6. When running in cluster mode, several thousand graph traversal steps may be proceeding at the same time, meaning that all database access is parallelized and that the algorithm is internally latency-tolerant. By default, all results are produced in a deterministic order, permitting predictable slicing of result sets.</p> <p>Furthermore, for queries where both ends of a path are given, the optimizer may decide to attack the path from both ends simultaneously. So, supposing that every member of a social network has an average of 30 contacts, and we need to find a path between two users that are no more than 6 steps apart, we begin at both ends, expanding each up to 3 levels, and we stop when we find the first intersection. Thus, we reach 2 * 30^3 = 54,000 nodes, and not 30^6 = 729,000,000 nodes.</p> <p>Writing a generic database driven graph traversal framework on the application side, say in Java over <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0xa8a9ef8">JDBC</a>, would easily be over a thousand lines. This is much more work than can be justified just for a one-off, ad-hoc query. Besides, the traversal order in such a case could not be optimized by the DBMS.</p> <h2>Next</h2> <p>In a future <a href="http://dbpedia.org/resource/Blog" id="link-id0xb526a40">blog</a> post I will show how this feature can be used for common graph tasks like critical path, itinerary planning, traveling salesman, the 8 queens chess problem, etc. There are lots of switches for controlling different parameters of the traversal. This is just the beginning. I will also give examples of the use of this in SPARQL.</p> </div>
Epistemology of the Sponger, or How Virtuoso Drives a Web Query
http://www.openlinksw.com/blog/vdb/blog/?date=2008-09-05#1432
2008-09-05T09:20:56Z
2008-09-05T16:04:28-04:00
<div> <div style="display:none;">Epistemology of the Sponger, or How Virtuoso Drives a Web Query</div> <p> <a href="http://virtuoso.openlinksw.com" id="link-id0x1ed6cf28">Virtuoso</a> has an extensive collection of <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1f8d1f78">RDF</a>-izers called Sponger Cartridges. These take a web resource in one of 30+ formats (so far) and extract RDF from it. The Virtuoso <a href="http://virtuoso.openlinksw.com/Whitepapers/html/VirtSpongerWhitePaper.html" id="link-id0x1edc90e8">Sponger</a> is a device which evaluates a query and along the way, finds dereferenceable links, dereferences them, and iteratively re-evaluates the query, until either nothing new is found or some limit is reached.</p> <p>We could call this <i>query-driven crawling</i>. The idea is intuitive â what one looks for, determines what one finds.</p> <p>This does however raise certain questions pertaining to the nature and ultimate possibility of <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1f836b68">knowledge</a>, i.e., epistemology.</p> <p>The process of querying could be said to go from the few to the many, just like the process of harvesting <a href="http://dbpedia.org/resource/Data" id="link-id0x1edb1648">data</a> from the web, the way any search engine does. One follows links or makes joins and thereby increases one's reach.</p> <p>The difference is that a query has no <i>a priori</i> direction. If I ask for the phone numbers of my friends and there are no phone numbers in the database, then it is valid to give an empty result without looking at my friends at all. <a href="http://dbpedia.org/resource/Closed_world_assumption" id="link-id0x1edf1f30">Closed world</a>, as it is said. Never mind that the friends would have had a "see also" link to a retrievable document that did have a phone number.</p> <p>The problem is that a query execution plan determines what possible dereferenceable material the query will encounter during its execution. What is worse, a query plan tends toward the minimal, i.e., toward minimizing the chances of encountering something dereferenceable along the way. Where query and crawl appeared to have a similarity, in fact they have two opposite goals.</p> <p>The user generally has no idea of the execution plan. In the general case, the user <i>cannot</i> have an idea of this plan. There are valid, over 40 year old reasons for leaving the query planning to the database. In exceptional situations the user can read or direct these, but this is really quite tedious and requires understanding that is basically never present.</p> <p>So, given a query, how do we find data that will match it, short of having a pre-loaded database of absolutely everything? This is certainly a desirable goal, and all in the <a href="http://dbpedia.org/resource/Open_world_assumption" id="link-id0x1eb46548">open world</a>, distributed spirit of the web.</p> <p>Let us limit ourselves to queries that have some literals in the object or subject positions. A <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1ed293f8">SPARQL</a> query is basically a graph. Its vertices are variables and literals, and its edges are triple patterns. An edge is labeled by a predicate. For now, we will consider the predicate to always be a literal. From each literal, we can draw a tree, following each edge starting at this literal and descending until we find another literal. Each tree is not always a spanning tree of the graph, but all the trees collectively span the graph.</p> <p>Consider the query </p> <blockquote> <code>{ <john> knows ?x . <mary> knows ?x . ?x label ?l }.</code> </blockquote> The starting points are the literals <code>john</code> and <code>mary</code>. The <code>john</code> tree has one child, <code>?x</code>, which has the children <code>mary</code> and <code>?l</code>. One could notate it as <blockquote> <code>{ <john> knows ?x . {{ <mary> knows ?x} UNION {?x label ?l}}}</code> </blockquote> That is, the head first, and if it has more than one child, a union listing them, recursively. <p>If one composed such queries for each literal in the original pattern and evaluated each as a breadth first walk of the tree, no query optimization tricks, and for each binding of each variable, recorded whether there was something to dereference, one would in a finite time have reached all the directly reachable data. Then one could evaluate the original query, using whatever plan was preferred.</p> <p>The check for dereferenceable data applied to each IRI-valued binding formed in the above evaluation, would consist of looking for "see also", "same as", and other such properties of the IRI. It could also consult text based search engines. Since the evaluation is breadth first, it generates a large number of parallel tasks and is fairly latency tolerant, i.e., it will not die if it must retrieve a few pages from remote sources. We will leave the exact rewrite rules for unions, optionals, aggregates, subqueries, and so on, as an exercise; the general idea should be clear enough.</p> <p>We have here shown a way of transforming SPARQL queries in such a way as to guarantee dereferencing of findable links, without requiring the end user to either explicitly specify or understand query plans.</p> <p>The present Sponger does not work exactly in this manner but it will be developed in this direction. Fortunately, the algorithms outlined above are nothing complicated.</p> </div>
A quick look at SP2B, the SPARQL Performance Benchmark
http://www.openlinksw.com/blog/vdb/blog/?date=2008-08-27#1423
2008-08-27T16:03:40Z
2008-09-02T09:49:57-04:00
<div> <div style="display:none;">A quick look at SP2B, the SPARQL Performance Benchmark</div> <p>I finally got around to running the <a href="http://dbis.informatik.uni-freiburg.de/index.php?project=SP2B" id="link-id17bac628">SP<sup>2</sup>B SPARQL Performance Benchmark</a> on the current <a href="http://virtuoso.openlinksw.com" id="link-id0x1dcaaa48">Virtuoso</a> Open Source Edition, v5.0.8.</p> <p>I ran it with the 5M triples scale, which is the highest scale for which the authors give numbers.</p> <p>I got a run time of 25 minutes for the 12 queries, giving an arithmetic mean of the query time of 125 seconds. This is better than the 800 or so seconds that the authors had measured. Also, Q6 of the set had failed for the authors, but we have since fixed this; the fix is in the v5.0.8 cut.</p> <p>I also tried it with a scale of 25M, but this became I/O bound and took a bit longer. I will try this with v6 and v7 cluster later, which are vastly better at anything I/O bound.</p> <p>The machine was a 2GHz Xeon with 8G RAM. The query text was the one from the authors, with an explicit <code>FROM</code> clause added; the client was the command line Interactive <a href="http://dbpedia.org/resource/SQL" id="link-id0x1be2c808">SQL</a> (iSQL).</p> <p>If one does the test with the default index layout without specifying a graph, things will not work very well. Also, returning the million-row results of these queries over the <a href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id0x1d7ac018">SPARQL protocol</a> is not practical.</p> <p>I will say something more about SP<sup>2</sup>B when I get to have a closer look.</p> </div>
Configuring Virtuoso for Benchmarking
http://www.openlinksw.com/blog/vdb/blog/?date=2008-08-25#1419
2008-08-25T14:06:11Z
2008-08-25T15:29:06.000036-04:00
<div> <div style="display:none;">Configuring Virtuoso for Benchmarking</div> <p>I will here summarize what should be known about running benchmarks with <a href="http://virtuoso.openlinksw.com" id="link-id0xc152cf0">Virtuoso</a>.</p> <h2>Physical Memory</h2> <p>For 8G RAM, in the <code>[Parameters]</code> stanza of <code>virtuoso.ini</code>, set â</p> <blockquote> <code> [Parameters]<br /> ...<br /> NumberOfBuffers = 550000 </code> </blockquote> <p>For 16G RAM, double thisâ</p> <blockquote> <code> [Parameters]<br /> ...<br /> NumberOfBuffers = 1100000 </code> </blockquote> <h2>Transaction Isolation</h2> <p>For most cases, certainly all <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xb7ba270">RDF</a> cases, <i>Read Committed</i> should be the default transaction isolation. In the <code>[Parameters]</code> stanza of <code>virtuoso.ini</code>, set â</p> <blockquote> <code> [Parameters]<br /> ...<br /> DefaultIsolation = 2 </code> </blockquote> <h2>Multiuser Workload</h2> <p>If <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0x1a40f308">ODBC</a>, <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0x1e003cf8">JDBC</a>, or similarly connected client applications are used, there must be more <code>ServerThreads</code> available than there will be client connections. In the <code>[Parameters]</code> stanza of <code>virtuoso.ini</code>, set â</p> <blockquote> <code> [Parameters]<br /> ...<br /> ServerThreads = 100 </code> </blockquote> <p>With web clients (unlike ODBC, JDBC, or similar clients), it may be justified to have fewer <code>ServerThreads</code> than there are concurrent clients. The <code>MaxKeepAlives</code> should be the maximum number of expected web clients. This can be more than the <code>ServerThreads</code> count. In the <code>[HTTPServer]</code> stanza of <code>virtuoso.ini</code>, set â</p> <blockquote> <code> [HTTPServer]<br /> ...<br /> ServerThreads = 100 <br /> MaxKeepAlives = 1000 <br /> KeepAliveTimeout = 10 </code> </blockquote> <p> <i><b>Note</b> â The <code>[HTTPServer] ServerThreads</code> are taken from the total pool made available by the <code>[Parameters] ServerThreads</code>. Thus, the <code>[Parameters] ServerThreads</code> should always be at least as large as (and is best set greater than) the <code>[HTTPServer] ServerThreads</code>, and if using the closed-source Commercial Version, should not exceed the licensed thread count.</i> </p> <h2>Disk Use</h2> <p>The basic rule is to use one stripe (file) per distinct physical device (not per file system), using no RAID. For example, one might stripe a database over 6 files (6 physical disks), with an initial size of 60000 pages (the files will grow as needed). </p> <p>For the above described example, in the <code>[Database]</code> stanza of <code>virtuoso.ini</code>, set â</p> <blockquote> <code> [Database]<br /> ...<br /> Striping = 1<br /> MaxCheckpointRemap = 2000000 </code> </blockquote> <p>â and in the <code>[Striping]</code> stanza, on one line per <code>SegmentName</code>, set â</p> <blockquote> <code> [Striping]<br /> ...<br /> Segment1 = 60000 , /virtdev/db/virt-seg1.db = q1 , /data1/db/virt-seg1-str2.db = q2 , /data2/db/virt-seg1-str3.db = q3 , /data3/db/virt-seg1-str4.db = q4 , /data4/db/virt-seg1-str5.db = q5 , /data5/db/virt-seg1-str6.db = q6</code> </blockquote> <p>As can be seen here, each file gets a background IO thread (the <code>= q<i>xxx</i></code> clause). It should be noted that all files on the same physical device should have the same <code>q<i>xxx</i></code> value. This is not directly relevant to the benchmarking scenario above, because we have only one file per device, and thus only one file per IO queue.</p> <h2> <a href="http://dbpedia.org/resource/SQL" id="link-id0xc8b97c0">SQL</a> Optimization</h2> <p>If queries have lots of joins but access little <a href="http://dbpedia.org/resource/Data" id="link-id0x193b2fa8">data</a>, as with the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1b283ca0">Berlin SPARQL Benchmark</a>, the SQL compiler must be told not to look for better plans if the best plan so far is quicker than the compilation time expended so far. Thus, in the <code>[Parameters]</code> stanza of <code>virtuoso.ini</code>, set â</p> <blockquote> <code> [Parameters]<br /> ...<br /> StopCompilerWhenXOverRunTime = 1 </code> </blockquote> </div>
BSBM With Triples and Mapped Relational Data
http://www.openlinksw.com/blog/vdb/blog/?date=2008-08-06#1410
2008-08-06T19:41:50Z
2008-08-06T16:29:44.000003-04:00
<div> <div style="display:none;">BSBM With Triples and Mapped Relational Data</div> <p>The special contribution of the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id10039db0">Berlin SPARQL Benchmark</a> (<a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id106b2538">BSBM</a>) to the <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id101a75f8">RDF</a> world is to raise the question of doing OLTP with <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xae54170">RDF</a>.</p> <p>Of course, here we immediately hit the question of comparisons with relational databases. To this effect, <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0x1e847b08">BSBM</a> also specifies a relational schema and can generate the <a href="http://dbpedia.org/resource/Data" id="link-id1206c378">data</a> as either triples or <a href="http://dbpedia.org/resource/SQL" id="link-id1667f040">SQL</a> inserts.</p> <p>The benchmark effectively simulates the case of exposing an existing <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id10a93518">RDBMS</a> as RDF. <a href="http://www.openlinksw.com/dataspace/organization/openlink#this" id="link-id13e46d80">OpenLink Software</a> calls this <i>RDF Views</i>. <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id12027578">Oracle</a> is beginning to call this <i>semantic covers</i>. The <a href="http://www.w3.org/2005/Incubator/rdb2rdf/" id="link-id161dc678">RDB2RDF XG</a>, a W3C incubator group, has been active in this area since Spring, 2008.</p> <h3>But why an OLTP workload with RDF to begin with?</h3> <p>We believe this is relevant because RDF promises to be the interoperability factor between potentially all of traditional IS. If <a href="http://dbpedia.org/resource/Data" id="link-id0x1e7119d8">data</a> is online for human consumption, it may be online via a <a href="http://dbpedia.org/resource/SPARQL" id="link-id106a8908">SPARQL</a> end-point as well. The economic justification will come from discoverability and from applications integrating multi-source structured data. Online shopping is a fine use case.</p> <p>Warehousing all the world's publishable data as RDF is not our first preference, nor would it be the publisher's. Considerations of duplicate infrastructure and maintenance are reason enough. Consequently, we need to show that mapping can outperform an RDF warehouse, which is what we'll do here.</p> <h3>What We Got </h3> <p>First, we found that <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1400" id="link-id150ea748">making the query plan took much too long</a> in proportion to the run time. With BSBM this is an issue because the queries have lots of joins but access relatively little data. So we made a faster compiler and along the way retouched the cost model a bit.</p> <p>But the really interesting part with BSBM is mapping relational data to RDF. For us, BSBM is a great way of showing that mapping can outperform even the best triple store. A relational row store is as good as unbeatable with the query mix. And when there is a clear mapping, there is no reason the <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xae5aff0">SPARQL</a> could not be directly translated.</p> <p>If Chris Bizer et al launched the mapping ship, we will be the ones to pilot it to harbor!</p> <p>We filled two <a href="http://virtuoso.openlinksw.com" id="link-id12dbdc70">Virtuoso</a> instances with a BSBM200000 data set, for 100M triples. One was filled with physical triples; the other was filled with the equivalent relational data plus mapping to triples. Performance figures are given in "query mixes per hour". (An update or follow-on to this post will provide elapsed times for each test run.)</p> <p>With the unmodified benchmark we got:</p> <blockquote> <table> <tr> <td><i>Physical Triples:</i> </td> <td>Â Â </td> <td>1297 qmph</td> </tr> <tr> <td><i>Mapped Triples:</i> </td> <td>Â Â </td> <td><b>3144 qmph</b> </td> </tr> </table> </blockquote> <p>In both cases, most of the time was spent on Q6, which looks for products with one of three words in the label. We altered Q6 to use text index for the mapping, and altered the databases accordingly. (There is no such thing as an e-commerce site without a text index, so we are amply justified in making this change.)</p> <p>The following were measured on the second run of a 100 query mix series, single test driver, warm cache.</p> <blockquote> <table> <tr> <td><i>Physical Triples:</i> </td> <td>Â Â </td> <td> 5746 qmph</td> </tr> <tr> <td><i>Mapped Triples:</i> </td> <td>Â Â </td> <td> <b>7525 qmph</b> </td> </tr> </table> </blockquote> <p>We then ran the same with 4 concurrent instances of the test driver. The qmph here is 400 / the longest run time.</p> <blockquote> <table> <tr> <td><i>Physical Triples:</i> </td> <td>Â Â </td> <td> 19459 qmph</td> </tr> <tr> <td><i>Mapped Triples:</i> </td> <td>Â Â </td> <td> <b>24531 qmph</b> </td> </tr> </table> </blockquote> <p>The system used was 64-bit Linux, 2GHz dual-Xeon 5130 (8 cores) with 8G RAM. The concurrent throughputs are a little under 4 times the single thread throughput, which is normal for SMP due to memory contention. The numbers do not evidence significant overhead from thread synchronization.</p> <p>The query compilation represents about 1/3 of total server side CPU. In an actual online application of this type, queries would be parameterized, so the throughputs would be accordingly higher. We used the <code>StopCompilerWhenXOverRunTime = 1</code> option here to cut needless compiler overhead, the queries being straightforward enough.</p> <p>We also see that the advantage of mapping can be further increased by more compiler optimizations, so we expect in the end mapping will lead RDF warehousing by a factor of 4 or so.</p> <h3>Suggestions for BSBM</h3> <ul> <li> <p> <b>Reporting Rules.</b> The benchmark spec should specify a form for disclosure of test run data, TPC style. This includes things like configuration parameters and exact text of queries. There should be accepted variants of query text, as with the TPC.</p> </li> <li> <p> <b>Multiuser operation.</b> The test driver should get a stream number as parameter, so that each client makes a different query sequence. Also, disk performance in this type of benchmark can only be reasonably assessed with a naturally parallel multiuser workload.</p> </li> <li> <p> <b>Add business intelligence.</b> SPARQL has aggregates now, at least with <a href="http://jena.sourceforge.net/" id="link-id11a25ac0">Jena</a> and <a href="http://virtuoso.openlinksw.com" id="link-id0xb003180">Virtuoso</a>, so let's use these. The BSBM business intelligence metric should be a separate metric off the same data. Adding synthetic sales figures would make more interesting queries possible. For example, producing recommendations like "customers who bought this also bought xxx."</p> </li> <li> <p> <b>For the SPARQL community</b>, BSBM sends the message that one ought to support parameterized queries and stored procedures. This would be a <a href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id109e2448">SPARQL protocol</a> extension; the SPARUL syntax should also have a way of calling a procedure. Something like <code>select proc (??, ??)</code> would be enough, where <code>??</code> is a parameter marker, like <code>?</code> in <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id13febf48">ODBC</a>/<a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id120416a8">JDBC</a>.</p> </li> <li> <p> <b>Add transactions.</b>Especially if we are contrasting mapping vs. storing triples, having an update flow is relevant. In practice, this could be done by having the test driver send web service requests for order entry and the SUT could implement these as updates to the triples or a mapped relational store. This could use stored procedures or logic in an app server.</p> </li> </ul> <h3>Comments on Query Mix</h3> <p>The time of most queries is less than linear to the scale factor. Q6 is an exception if it is not implemented using a text index. Without the text index, Q6 will inevitably come to dominate query time as the scale is increased, and thus will make the benchmark less relevant at larger scales.</p> <h2>Next</h2> <p>We include the sources of our RDF view definitions and other material for running BSBM with our forthcoming Virtuoso Open Source 5.0.8 release. This also includes all the query optimization work done for BSBM. This will be available in the coming days.</p> </div>
Virtuoso Optimizations for the Berlin SPARQL Benchmark
http://www.openlinksw.com/blog/vdb/blog/?date=2008-07-30#1401
2008-07-30T18:52:11Z
2008-08-06T16:29:42-04:00
<div> <div style="display:none;">Virtuoso Optimizations for the Berlin SPARQL Benchmark </div> <p>We had a look at Chris Bizer's initial results with the <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id105c9f78">Berlin SPARQL Benchmark</a> (<a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id102d62b0">BSBM</a>) on <a href="http://virtuoso.openlinksw.com" id="link-id13eb9780">Virtuoso</a>. The first results were rather bad, as nearly all of the run time was spent optimizing the <a href="http://dbpedia.org/resource/SPARQL" id="link-id14a51258">SPARQL</a> statements and under 10% actually running them.</p> <p>So I spent a couple of days on the <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xaad28d0">SPARQL</a>/<a href="http://dbpedia.org/resource/SQL" id="link-id108745b0">SQL</a> compiler, to the effect of making it do a better guess of initial execution plan and streamlining some operations. In fact, many of the queries in <a href="http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/spec/index.html" id="link-id0xaa230b8">BSBM</a> are not particularly sensitive to execution plan, as they access a very small portion of the database. So to close the matter, I put in a flag that makes the <a href="http://dbpedia.org/resource/SQL" id="link-id0x1e9e8e28">SQL</a> compiler give up on devising new plans if the time of the best plan so far is less than the time spent compiling so far.</p> <p>With these changes, available now as a diff on top of 5.0.7, we run quite well, several times better than initially. With the compiler time cut-off in place (ini parameter <code>StopCompilerWhenXOverRunTime = 1</code>), we get the following times, output from the BSBM test driver:</p> <blockquote> <pre> Starting test... 0: 1031.22 ms, total: 1151 ms 1: 982.89 ms, total: 1040 ms 2: 923.27 ms, total: 968 ms 3: 898.37 ms, total: 932 ms 4: 855.70 ms, total: 865 ms Scale factor: 10000 Number of query mix runs: 5 times min/max Query mix runtime: 0.8557 s / 1.0312 s Total runtime: 4.691 seconds QMpH: 3836.77 query mixes per hour CQET: 0.93829 seconds average runtime of query mix CQET (geom.): 0.93625 seconds geometric mean runtime of query mix Metrics for Query 1: Count: 5 times executed in whole run AQET: 0.012212 seconds (arithmetic mean) AQET(geom.): 0.009934 seconds (geometric mean) QPS: 81.89 Queries per second minQET/maxQET: 0.00684000s / 0.03115700s Average result count: 7.0 min/max result count: 3 / 10 Metrics for Query 2: Count: 35 times executed in whole run AQET: 0.030490 seconds (arithmetic mean) AQET(geom.): 0.029776 seconds (geometric mean) QPS: 32.80 Queries per second minQET/maxQET: 0.02467300s / 0.06753000s Average result count: 22.5 min/max result count: 15 / 30 Metrics for Query 3: Count: 5 times executed in whole run AQET: 0.006947 seconds (arithmetic mean) AQET(geom.): 0.006905 seconds (geometric mean) QPS: 143.95 Queries per second minQET/maxQET: 0.00580000s / 0.00795100s Average result count: 4.0 min/max result count: 0 / 10 Metrics for Query 4: Count: 5 times executed in whole run AQET: 0.008858 seconds (arithmetic mean) AQET(geom.): 0.008829 seconds (geometric mean) QPS: 112.89 Queries per second minQET/maxQET: 0.00804400s / 0.01019500s Average result count: 3.4 min/max result count: 0 / 10 Metrics for Query 5: Count: 5 times executed in whole run AQET: 0.087542 seconds (arithmetic mean) AQET(geom.): 0.087327 seconds (geometric mean) QPS: 11.42 Queries per second minQET/maxQET: 0.08165600s / 0.09889200s Average result count: 5.0 min/max result count: 5 / 5 Metrics for Query 6: Count: 5 times executed in whole run AQET: 0.131222 seconds (arithmetic mean) AQET(geom.): 0.131216 seconds (geometric mean) QPS: 7.62 Queries per second minQET/maxQET: 0.12924200s / 0.13298200s Average result count: 3.6 min/max result count: 3 / 5 Metrics for Query 7: Count: 20 times executed in whole run AQET: 0.043601 seconds (arithmetic mean) AQET(geom.): 0.040890 seconds (geometric mean) QPS: 22.94 Queries per second minQET/maxQET: 0.01984400s / 0.06012600s Average result count: 26.4 min/max result count: 5 / 96 Metrics for Query 8: Count: 10 times executed in whole run AQET: 0.018168 seconds (arithmetic mean) AQET(geom.): 0.016205 seconds (geometric mean) QPS: 55.04 Queries per second minQET/maxQET: 0.01097600s / 0.05066900s Average result count: 12.8 min/max result count: 6 / 20 Metrics for Query 9: Count: 20 times executed in whole run AQET: 0.043813 seconds (arithmetic mean) AQET(geom.): 0.043807 seconds (geometric mean) QPS: 22.82 Queries per second minQET/maxQET: 0.04274900s / 0.04504100s Average result count: 0.0 min/max result count: 0 / 0 Metrics for Query 10: Count: 15 times executed in whole run AQET: 0.030697 seconds (arithmetic mean) AQET(geom.): 0.029651 seconds (geometric mean) QPS: 32.58 Queries per second minQET/maxQET: 0.02072000s / 0.03975700s Average result count: 1.1 min/max result count: 0 / 4 real 0 m 5.485 s user 0 m 2.233 s sys 0 m 0.170 s </pre></blockquote> <p>Of the approximately 5.5 seconds of running five query mixes, the test driver spends 2.2 s. The server side processing time is 3.1 s, of which SQL compilation is 1.35 s. The rest is miscellaneous system time. The measurement is on 64-bit Linux, 2GHz dual-Xeon 5130 (8 cores) with 8G RAM. </p> <p>We note that this type of workload would be done with stored procedures or prepared, parameterized queries in the SQL world.</p> <p>There will be some further tuning still but this addresses the bulk of the matter. There will be a separate message about the patch containing these improvements.</p> </div>
Virtuoso 5.0.7 Release, Now With Jena and Sesame APIs
http://www.openlinksw.com/blog/vdb/blog/?date=2008-07-17#1393
2008-07-17T17:18:09Z
2008-07-17T15:28:22.000002-04:00
<div> <div style="display:none;">Virtuoso 5.0.7 Release, Now With Jena and Sesame APIs</div> <h2>Improvements</h2> <ul> <li> <a href="http://docs.openlinksw.com:80/virtuoso/rdfnativestorageproviders.html" id="link-id13e54d98">Full operation</a> with <a href="http://jena.sourceforge.net/" id="link-id0x11a3d360">Jena</a> and <a href="http://sourceforge.net/projects/sesame/" id="link-id0x1108d428">Sesame</a> <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1288aa00">RDF</a> Frameworks. This fully replaces any previous attempts at interop, and introduces samples and test suites.</li> <li>Better support for alternate RDF indexing schemes</li> <li>Parallel operation of the RDF Sponger, importing multiple sources concurrently.</li> <li>New <a href="http://dbpedia.org/resource/Data" id="link-id0x128a9810">data</a> formats supported for on-demand RDF-ization in the Sponger</li> <li>More efficient support for inference of subclass and sub-property; now capable of efficiently handling taxonomies of tens of thousands of classes</li> <li> <a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x6af0678">OWL</a> <a href="http://docs.openlinksw.com:80/virtuoso/rdfsparqlrule.html#rdfsparqlruleintro" id="link-id104d58d8">equivalentClass and equivalentProperty</a> support.</li> <li> <a href="http://docs.openlinksw.com:80/virtuoso/rdfdatarepresentation.html#rdfdynamiclocal" id="link-id109606a8">Dynamic IRI host part</a> support for mapped data and for metadata of local resources. Renaming the host or using multiple virtual hosts will accept URIs with the right host part and refer to the same thing, no duplicate storage required.</li> <li> <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x12e0cc38">SPARQL</a> optimizations for <code>LIMIT</code> and <code>OFFSET</code> </li> </ul> <h2>Documentation</h2> <ul> <li> <a href="http://docs.openlinksw.com:80/virtuoso/perfdiag.html#perfdiagqueryplans" id="link-id10a56dd0">How to read query plans and how to use the key performance meters</a> </li> <li> <a href="http://docs.openlinksw.com:80/virtuoso/rdfperformancetuning.html#rdfperfcost" id="link-id106cb5c0">How to diagnose SPARQL queries and how to decide what indexing scheme is right for each RDF use case</a> </li> <li>How to debug RDF views</li> <ul> <li> <a href="http://docs.openlinksw.com:80/virtuoso/sparqldebug.html" id="link-id133b4420">Better documentation of SPARQL extensions and options</a> </li> <li> <a href="http://docs.openlinksw.com:80/virtuoso/rdfviews.html#rdfviewnorthwindexample1" id="link-id1060fdd8">A sample of correct RDF view usage with the Northwind demo data</a> </li> </ul> </ul> <h2>Bug Fixes</h2> <ul> <li>Generally improved safety of built-in functions, better argument checking.</li> <li>Verified UTF8 international character support in all RDF use cases, <a href="http://dbpedia.org/resource/SQL" id="link-id0x12839fd0">SQL</a> client/<a href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id0x1288f350">SPARQL protocol</a>/all data formats.</li> </ul> </div>
The DARQ Matter of Federation
http://www.openlinksw.com/blog/vdb/blog/?date=2008-06-09#1381
2008-06-09T14:02:19Z
2008-06-11T15:15:14-04:00
<div> <div style="display:none;">The DARQ Matter of Federation</div> <p>Astronomers propose that the universe is held together, so to speak, by the gravity of invisible "dark matter" spread in interstellar and intergalactic space.</p> <p>For the <a href="http://dbpedia.org/resource/Data" id="link-id0x19dbf410">data</a> web, it will be held together by federation, also an invisible factor. As in Minkowski space, so in <a href="http://dbpedia.org/resource/Cyberspace" id="link-id0x9fc13ff8">cyberspace</a>.</p> <p>To take the astronomical analogy further, putting too much visible stuff in one place makes a black hole, whose chief properties are that it is very heavy, can only get heavier and that nothing comes out.</p> <p> <a href="http://darq.sourceforge.net/" id="link-id0x1d06bd88">DARQ</a> is Bastian Quilitz's federated extension of the <a href="http://jena.sourceforge.net/" id="link-id0x1cf28f70">Jena</a> <a href="http://jena.sourceforge.net/ARQ/" id="link-id0x1cba22c8">ARQ</a> <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x171c7dc8">SPARQL</a> processor. It has existed for a while and was also presented at <a href="http://www.eswc2008.org/" id="link-id0x1ed53cd0">ESWC2008</a>. There is also SPARQL FED from Andy Seaborne, an explicit means of specifying which end point will process which fragment of a distributed SPARQL query. Still, for federation to deliver in an open, decentralized world, it must be transparent. For a specific application, with a predictable workload, it is of course OK to partition queries explicitly.</p> <p>Bastian had split <a href="http://dbpedia.org/resource/DBpedia" id="link-id0x1ce846c0">DBpedia</a> among five <a href="http://virtuoso.openlinksw.com" id="link-id0x1cad0640">Virtuoso</a> servers and was querying this set with DARQ. The end result was that there was a rather frightful cost of federation as opposed to all the data residing in a single Virtuoso. The other result was that if selectivity of predicates was not correctly guessed by the federation engine, the proposition was a non-starter. With correct join order it worked, though.</p> <p>Yet, we really want federation. Looking further down the road, we simply must make federation work. This is just as necessary as running on a server cluster for mid-size workloads.</p> <p>Since we are convinced of the cause, let's talk about the means.</p> <p>For DARQ as it now stands, there's probably an order of magnitude or even more to gain from a couple of simple tricks. If going to a SPARQL end point that is not the outermost in the loop join sequence, batch the requests together in one <a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x19a48280">HTTP</a>/1.1 message. So, if the query is "get me my friends living in cities of over a million people," there will be the fragment "get city where x lives" and later "ask if population of x greater than 1000000". If I have 100 friends, I send the 100 requests in a batch to each eligible server.</p> <p>Further, if running against a server of known brand, use a client-server connection and prepared statements with array parameters. This can well improve the processing speed at the remote end point by another order of magnitude. This gain may however not be as great as the latency savings from message batching. We will provide a sample of how to do this with Virtuoso over <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0x1cf18278">JDBC</a> so Bastian can try this if interested.</p> <p>These simple things will give a lot of mileage and may even decide whether federation is an option in specific applications. For the open web however, these measures will not yet win the day.</p> <p>When federating <a href="http://dbpedia.org/resource/SQL" id="link-id0x1cf7d0e8">SQL</a>, colocation of data is sort of explicit. If two tables are joined and they are in the same source, then the join can go to the source. For SPARQL this is also so but with a twist:</p> <p>If a foaf:Person is found on a given server, this does not mean that the Person's geek code or email hash will be on the same server. Thus <code>{?p name "Johnny" . ?p geekCode ?g . ?p emailHash ?h }</code> does not necessarily denote a colocated join if many servers serve items of the vocabulary.</p> <p>However, in most practical cases, for obtaining a rapid answer, treating this as a colocated fragment will be appropriate. Thus, it may be necessary to be able to declare that geek codes will be assumed colocated with names. This will save a lot of message passing and offer decent, if not theoretically total recall. For search style applications, starting with such assumptions will make sense. If nothing is found, then we can partition each join step separately for the unlikely case that there were a server that gave geek codes but not names.</p> <p>For Virtuoso, we find that a federated query's asynchronous, parallel evaluation model is not so different from that on a local cluster. So the cluster version could have the option of federated query. The difference is that a cluster is local and tightly coupled and predictably partitioned but a federated setting is none of these.</p> <p>For description, we would take DARQ's description model and maybe extend it a little where needed. Also we would enhance the protocol to allow just asking for the query cost estimate given a query with literals specified. We will do this eventually.</p> <p>We would like to talk to Bastian about large improvements to DARQ, specially when working with Virtuoso. We'll see.</p> <p>Of course, one mode of federating is the crawl-as-you-go approach of the Virtuoso <a href="http://virtuoso.openlinksw.com/Whitepapers/html/VirtSpongerWhitePaper.html" id="link-id0x1e163140">Sponger</a>. This will bring in fragments following seeAlso or sameAs declarations or other references. This will however not have the recall of a warehouse or federation over well described SPARQL end-points. But up to a certain volume it has the speed of local storage.</p> <p>The emergence of voiD (Vocabulary of Interlinked Data) is a step in the direction of making federation a reality. There is <a href="http://www.openlinksw.com/weblog/oerling/?id=1377" id="link-id1109a4c8">a separate post</a> about this.</p> </div>
Aspects of RDF to RDF Mapping
http://www.openlinksw.com/blog/vdb/blog/?date=2008-06-09#1380
2008-06-09T14:02:18Z
2008-06-11T13:15:39-04:00
<div> <div style="display:none;">Aspects of RDF to RDF Mapping</div> <p>The W3C has recently launched an <a href="http://www.w3.org/2005/Incubator/rdb2rdf/" id="link-idd763f48">incubator group about mapping relational data to RDF</a>.</p> <p>From participating in the group for the few initial sessions, I get the following impressions.</p> <p>There is a segment of users, for example from the biomedical community, who do heavy duty <a href="http://dbpedia.org/resource/Data" id="link-id0x17f9e6f8">data</a> integration and look to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x17eabf48">RDF</a> for managing complexity. Unifying heterogeneous data under OWL ontologies, reasoning, and data integrity, are points of interest.</p> <p>There is another segment that is concerned with semantifying the document web, which topic includes initiatives such as <a href="http://triplify.org/" id="link-id0x1a25cd28">Triplify</a> and <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x182c41e8">semantic web</a> search such as <a href="http://sindice.org/" id="link-id0x1a29c5e8">Sindice</a>. The emphasis there is on minimizing entry cost and creating critical mass. The next one to come will clean up the semantics, if these need be cleaned up at all.</p> <p>(Some cleanup is taking place with <a href="http://www.mpi-inf.mpg.de/~suchanek/downloads/yago/" id="link-id0x17fd2b70">Yago</a> and <a href="http://zitgist.com/about/" id="link-id0x17e6ab88">Zitgist</a>, but this is a matter for a different post.)</p> <p>Thus, technically speaking, the mapping landscape is diverse, but ETL (extract-transform-load) seems to predominate. The biomedical people make data warehouses for answering specific questions. The web people are interested in putting data out in the expectation that the next player will warehouse it and allow running complex meshups against the whole of the RDF-ized web.</p> <p>As one would expect, these groups see different issues and needs. Roughly speaking, one is about quality and structure and the other is about volume.</p> <p>Where do we stand?</p> <p>We are with the research data warehousers in saying that the mapping question is very complex and that it would indeed be nice to bypass ETL and go to the source <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x182acd68">RDBMS</a>(s) on demand. Projects in this direction are ongoing.</p> <p>We are with the web people in building large RDF stores with scalable query answering for arbitrary RDF, for example, hosting a lot of the Linking Open Data sets, and working with Zitgist.</p> <p>These things are somewhat different.</p> <p>At present, both the research warehousers and the web scalers predominantly go for ETL.</p> <p>This is fine by us as we definitely are in the large RDF store race.</p> <p>Still, mapping has its point. A relational store will perform quite a bit faster than a quad store if it has the right covering indices or application-specific compressed columnar layout. Thus, there is nothing to block us from querying analytics in <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x16c91438">SPARQL</a>, once the obviously necessary extensions of sub-query, expressions and aggregation are in place.</p> <p>To cite an example, the Ordnance Survey of the UK has a GIS system running on <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x17ee37c8">Oracle</a> with an entry pretty much for each mailbox, lamp post, and hedgerow in the country. According to Ordnance Survey, this would be 1 petatriple, 1e15 triples. "Such a big server farm that we'd have to put it on our map," as Jenny Harding put it at <a href="http://www.eswc2008.org/" id="link-id0x1cab6330">ESWC2008</a>. I'd add that an even bigger map entry would be the power plant needed to run the 100,000 or so PCs this would take. This is counting 10 gigatriples per PC, which would not even give very good working sets.</p> <p>So, on-the-fly RDBMS-to-RDF mapping in some cases is simply necessary. Still, the benefits of RDF for integration can be preserved if the translation middleware is smart enough. Specifically, this entails knowing what tables can be joined with what other tables and pushing maximum processing to the RDBMS(s) involved in the query.</p> <p>You can download the slide set I used for the <a href="http://virtuoso.openlinksw.com" id="link-id0xa1fb7e8">Virtuoso</a> presentation for the RDB to RDF mapping incubator group (<a href="http://virtuoso.openlinksw.com/wiki/main/Main/VirtPresentations/Relational2RDF.ppt" id="link-id106f9e88">PPT</a>; <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VirtPresentations" id="link-id10a8dc90">other formats</a> coming soon). The main point is that real integration is hard and needs smart query splitting and optimization, as well as real understanding of the databases and subject matter from the <a href="http://dbpedia.org/resource/Information" id="link-id0x17ee38a0">information</a> architect. Sometimes in the web space it can suffice to put data out there with trivial RDF translation and hope that a search engine or such will figure out how to join this with something else. For the enterprise, things are not so. Benefits are clear if one can navigate between disjoint silos but making this accurate enough for deriving business conclusions, as well as efficient enough for production, is a soluble and non-trivial question.</p> <p>We will show the basics of this with the <a href="http://dbpedia.org/resource/TPC-H" id="link-id0x1844d718">TPC-H</a> mapping, and by joining this with physical triples. We will also make a set of TPC-H format table sets, make mappings between keys in one to keys in the other, and show joins between the two. The SPARQL querying of one such data store is a done deal, including the SPARQL extensions for this. There is even a demo paper, Business Intelligence Extensions for SPARQL (<a href="http://virtuoso.openlinksw.com/wiki/main/Main/VirtPresentations/RDFAndMapped_BI.pdf" id="link-id12ea4b18">PDF</a>; <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VirtPresentations" id="link-id106e1810">other formats</a> coming soon), by us on the subject in the ESWC 2008 proceedings. If there is an issue left, it is just the technicality of always producing <a href="http://dbpedia.org/resource/SQL" id="link-id0x17fc8d60">SQL</a> that looks hand-crafted and hence is better understood by the target RDBMS(s). For example, Oracle works better if one uses an <code>IN</code> sub-query instead of the equivalent existence test.</p> <p>Follow this <a href="http://dbpedia.org/resource/Blog" id="link-id0xa9bcef8">blog</a> for more on the topic; published papers are always a limited view on the matter.</p> </div>
ESWC 2008
http://www.openlinksw.com/blog/vdb/blog/?date=2008-06-09#1379
2008-06-09T14:02:16Z
2008-06-11T13:15:33-04:00
<div> <div style="display:none;">ESWC 2008</div> <p>Yrjänä Rankka and I attended <a href="http://www.eswc2008.org/" id="link-id10b7a038">ESWC2008</a> on behalf of OpenLink.</p> <p>We were invited at the last minute to give a <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id105df758">Linked Open Data</a> talk at Paolo Bouquet's Identity and Reference workshop. We also had a demo of <a href="http://dbpedia.org/resource/SPARQL" id="link-id12eacca0">SPARQL</a> BI (<a href="http://virtuoso.openlinksw.com/wiki/main/Main/VirtPresentations/ESWC2008%20SPARQL%20BI%20OpenLink.ppt" id="link-id10b43e58">PPT</a>); <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VirtPresentations" id="link-id1116d8f0">other formats coming soon</a>), our business intelligence extensions to <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x16c9bfc8">SPARQL</a> as well as joining between relational <a href="http://dbpedia.org/resource/Data" id="link-id10badc40">data</a> mapped to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id108edaf8">RDF</a> and native <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x181a5ed8">RDF</a> <a href="http://dbpedia.org/resource/Data" id="link-id0x17e69910">data</a>. i was also speaking at the social networks panel chaired by Harry Halpin.</p> <p>I have gathered a few impressions that I will share in the next few posts (<a href="http://www.openlinksw.com/weblog/oerling/?id=1375" id="link-id107298e0">1 - RDF Mapping</a>, <a href="http://www.openlinksw.com/weblog/oerling/?id=1376" id="link-id10b3a530">2 - DARQ</a>, <a href="http://www.openlinksw.com/weblog/oerling/?id=1377" id="link-id107290e0">3 - voiD</a>, <a href="http://www.openlinksw.com/weblog/oerling/?id=1378" id="link-id1071a950">4 - Paradigmata</a>). <i>Caveat: This is not meant to be complete or impartial press coverage of the event but rather some quick comments on issues of personal/OpenLink interest. The fact that I do not mention something does not mean that it is unimportant.</i> </p> <h2>The voiD Graph</h2> <p> <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1a87f110">Linked Open Data</a> was well represented, with Chris Bizer, Tom Heath, ourselves and many others. The great advance for <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id108f3c48">LOD</a> this time around is <a href="http://community.linkeddata.org/MediaWiki/index.php?MetaLOD#Kick-off_meeting_at_ESWC08" id="link-id10df9830">voiD, the Vocabulary of Interlinked Datasets</a>, a means to describe what in fact is inside the <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1a089980">LOD</a> cloud, how to join it with what and so forth. Big time important if there is to be a <a href="http://www.openlinksw.com/weblog/oerling/?id=1377" id="link-iddf74578">web of federatable data sources</a>, feeding directly into what we have been saying for a while about SPARQL end-point self-description and discovery. There is reasonable hope of having something by the date of <a href="http://www.linkeddataplanet.com/" id="link-id10dd0848">Linked Data Planet</a> in a couple of weeks.</p> <h2>Federating</h2> <p>Bastian Quilitz gave a talk about his <a href="http://darq.sourceforge.net/" id="link-id108746e8">DARQ</a>, a federated version of Jena's ARQ.</p> <p>Something like <a href="http://darq.sourceforge.net/" id="link-id0x1a2d9860">DARQ</a>'s optimization statistics should make their way into the <a href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id10992348">SPARQL protocol</a> as well as the voiD data set description.</p> <p>We really need federation but more on this in <a href="http://www.openlinksw.com/weblog/oerling/?id=1376" id="link-id1059d688">a separate post</a>.</p> <h2> <a href="http://xsparql.deri.ie/" id="link-id10314308">XSPARQL</a> </h2> <p>Axel Polleres et al had a paper about <a href="http://xsparql.deri.ie/" id="link-id0x1ad77490">XSPARQL</a>, a merge of <a href="http://dbpedia.org/resource/XQuery" id="link-id10b98e90">XQuery</a> and SPARQL. While visiting DERI a couple of weeks back and again at the conference, we talked about OpenLink implementing the spec. It is evident that the engines must be in the same process and not communicate via the <a href="http://www.w3.org/TR/rdf-sparql-protocol/" id="link-id0x17e75190">SPARQL protocol</a> for this to be practical. We could do this. We'll have to see when.</p> <p>Politically, using <a href="http://dbpedia.org/resource/XQuery" id="link-id0x18a9bf10">XQuery</a> to give expressions and XML synthesis to SPARQL would be fitting. These things are needed anyhow, as surely as aggregation and sub-queries but the latter would not so readily come from XQuery. Some rapprochement between RDF and XML folks is desirable anyhow.</p> <h2>Panel: Will the Sem Web Rise to the Challenge of the Social Web?</h2> <p>The social web panel presented the question of whether the sem web was ready for prime time with data portability.</p> <p>The main thrust was expressed in Harry Halpin's rousing closing words: "Men will fight in a battle and lose a battle for a cause they believe in. Even if the battle is lost, the cause may come back and prevail, this time changed and under a different name. Thus, there may well come to be something like our <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id122f4da0">semantic web</a>, but it may not be the one we have worked all these years to build if we do not rise to the occasion before us right now."</p> <p>So, how to do this? Dan Brickley asked the audience how many supported, or were aware of, the latest Web 2.0 things, such as <a href="http://dbpedia.org/page/OAuth" id="link-idf300bc0">OAuth</a> and <a href="http://dbpedia.org/page/OpenID" id="link-id10ce7a40">OpenID</a>. A few were. The general idea was that research (after all, this was a research event) should be more integrated and open to the world at large, not living at the "outdated pace" of a 3 year funding cycle. Stefan Decker of DERI acquiesced in principle. Of course there is impedance mismatch between specialization and interfacing with everything.</p> <p>I said that triples and vocabularies existed, that OpenLink had <a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id1210dbf8">ODS</a> (<a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id11076be8">OpenLink Data Spaces</a>, <a href="http://community.linkeddata.org/" id="link-id10d46710">Community LinkedData</a>) for managing one's data-web presence, but that scale would be the next thing. Rather large scale even, with 100 gigatriples (Gtriples) reached before one even noticed. It takes a lot of PCs to host this, maybe $400K worth at today's prices, without replication. Count 16G ram and a few cores per Gtriple so that one is not waiting for disk all the time.</p> <p>The tricks that Web 2.0 silos do with app-specific data structures and app-specific partitioning do not really work for RDF without compromising the whole point of smooth schema evolution and tolerance of ragged data.</p> <p>So, simple vocabularies, minimal inference, minimal blank nodes. Besides, note that the inference will have to be done at run time, not forward-chained at load time, if only because users will not agree on what sameAs and other declarations they want for their queries. Not to mention spam or malicious sameAs declarations!</p> <p>As always, there was the question of business models for the open data web and for semantic technologies in general. As we see it, <a href="http://dbpedia.org/resource/Information" id="link-id108b7688">information</a> overload is the factor driving the demand. Better contextuality will justify semantic technologies. Due to the large volumes and complex processing, a data-as-service model will arise. The data may be open, but its query infrastructure, cleaning, and keeping up-to-date, can be monetized as services.</p> <h2>Identity and Reference</h2> <p>For the identity and reference workshop, the ultimate question is metaphysical and has no single universal answer, even though people, ever since the dawn of time and earlier, have occupied themselves with the issue. Consequently, I started with the Genesis quote where Adam called things by <i>nominibus suis</i>, off-hand implying that things would have some intrinsic ontologically-due names. This would be among the older references to the question, at least in widely known sources.</p> <p>For present purposes, the consensus seemed to be that what would be considered the same as something else depended entirely on the application. What was similar enough to warrant a sameAs for cooking purposes might not warrant a sameAs for chemistry. In fact, complete and exact sameness for URIs would be very rare. So, instead of making generic weak similarity assertions like similarTo or seeAlso, one would choose a set of strong sameAs assertions and have these in effect for query answering if they were appropriate to the granularity demanded by the application.</p> <p>Therefore sameAs is our permanent companion, and there will in time be malicious and spam sameAs. So, nothing much should be materialized on the basis of sameAs assertions in an <a href="http://dbpedia.org/resource/Open_world_assumption" id="link-id10c4dfd0">open world</a>. For an app-specific warehouse, sameAs can be resolved at load time.</p> <p>There was naturally some apparent tension between the Occam camp of <a href="http://dbpedia.org/resource/Entity" id="link-id105fd240">entity</a> name services and the LOD camp. I would say that the issue is more a perceived polarity than a real one. People will, inevitably, continue giving things names regardless of any centralized authority. Just look at natural language. But having a dictionary that is commonly accepted for established domains of discourse is immensely helpful.</p> <h2>CYC and NLP</h2> <p>The semantic search workshop was interesting, especially CYC's presentation. CYC is, as it were, the grand old man of <a href="http://dbpedia.org/resource/Knowledge" id="link-id10568158">knowledge</a> representation. Over the long term, I would have support of the CYC inference language inside a database query processor. This would mostly be for repurposing the huge <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1acff9d0">knowledge</a> base for helping in search type queries. If it is for transactions or financial reporting, then queries will be <a href="http://dbpedia.org/resource/SQL" id="link-id130a0a80">SQL</a> and make little or no use of any sort of inference. If it is for summarization or finding things, the opposite holds. For scaling, the issue is just making correct cardinality guesses for query planning, which is harder when inference is involved. We'll see.</p> <p>I will also have a closer look at natural language one of these days, quite inevitably, since <a href="http://zitgist.com/about/" id="link-id10795828">Zitgist</a> (for example) is into <a href="http://dbpedia.org/resource/Entity" id="link-id0x18a12918">entity</a> disambiguation.</p> <h2>Scale</h2> <p>Garlic gave a talk about their Data Patrol and QDOS. We agree that storing the data for these as triples instead of 1000 or so constantly changing relational tables could well make the difference between next-to-unmanageable and efficiently adaptive.</p> <p>Garlic probably has the largest triple collection in constant online use to date. We will soon join them with our hosting of the whole LOD cloud and <a href="http://sindice.org/" id="link-id0x17f18a38">Sindice</a>/<a href="http://zitgist.com/about/" id="link-id0x184e9e90">Zitgist</a> as triples.</p> <h2>Conclusions</h2> <p>There is a mood to deliver applications. Consequently, scale remains a central, even the principal topic. So for now we make bigger centrally-managed databases. At the next turn around the corner we will have to turn to federation. The point here is that a planetary-scale, centrally-managed, online system can be made when the workload is uniform and anticipatable, but if it is free-form queries and complex analysis, we have a problem. So we move in the direction of federating and charging based on usage whenever the workload is more complex than making simple lookups now and then.</p> <p>For the <a href="http://virtuoso.openlinksw.com" id="link-id1026ac28">Virtuoso</a> roadmap, this changes little. Next we make data sets available on Amazon EC2, as widely promised at ESWC. With big scale also comes rescaling and repartitioning, so this gets additional weight, as does further parallelizing of single user workloads. As it happens, the same medicine helps for both. At <a href="http://www.linkeddataplanet.com/" id="link-id0x17ff5c20">Linked Data Planet</a>, we will make more announcements.</p> </div>
Virtuoso Cluster Paper
http://www.openlinksw.com/blog/vdb/blog/?date=2008-05-30#1369
2008-05-30T10:02:04Z
2008-05-30T06:02:05-04:00
<div> <div style="display:none;">Virtuoso Cluster Paper</div> <div> <div>We have a new article on <a href="http://virtuoso.openlinksw.com" id="link-id10424890">Virtuoso</a> cluster, submitted to ISWC 2008.</div> <div> Right now we are working on hosting the billion triples challenge <a href="http://dbpedia.org/resource/Data" id="link-id1077f800">data</a> set at Amazon EC2 using <a href="http://virtuoso.openlinksw.com" id="link-id102117f0">Virtuoso</a> Cluster. This will be the first publicly available instance of <a href="http://virtuoso.openlinksw.com" id="link-id0x20387e80">Virtuoso</a> Cluster and all interested may then instantiate their own copy on the EC2 infrastructure. </div> <br /> <div> <a href="http://www.openlinksw.com/weblog/oerling/2008iswc_webscale_rdf.pdf" id="link-id10af2f30">Towards Web Scale RDF</a> <br /> <a href="http://www.openlinksw.com/weblog/oerling/RDFAndMapped_BI.pdf" id="link-idfedf9f0">Integrating Open Sources and Relational Data with SPARQL</a> <br /> <a href="http://www.openlinksw.com/weblog/oerling/bisparql2.pdf" id="link-id106e5418">Business Intelligence Extensions for SPARQL</a> <br /> </div> <br /> <div> Look for a separate announcement in the near future. </div> </div> </div>
DBpedia Benchmark Revisited
http://www.openlinksw.com/blog/vdb/blog/?date=2008-05-09#1359
2008-05-09T19:33:42Z
2008-05-12T11:24:43-04:00
<div> <div style="display:none;">DBpedia Benchmark Revisited</div> <p>We ran the <a href="http://dbpedia.org/resource/DBpedia" id="link-id0x1cd6d0c8">DBpedia</a> benchmark queries again with different configurations of <a href="http://virtuoso.openlinksw.com" id="link-id0x1bf01048">Virtuoso</a>. I had not studied the details of the matter previously but now did have a closer look at the queries.</p> <p>Comparing numbers given by different parties is a constant problem. In the case reported here, we loaded the full DBpedia 3, all languages, with about 198M triples, onto Virtuoso v5 and Virtuoso Cluster v6, all on the same 4 core 2GHz Xeon with 8G RAM. All databases were striped on 6 disks. The Cluster configuration was with 4 processes in the same box.</p> <p>We ran the queries in two variants:</p> <ul> <li>With graph specified in the <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1b9d3ca0">SPARQL</a> <code>FROM</code> clause, using the default indices.</li> <li>With no graph specified anywhere, using an alternate indexing scheme.</li> </ul> <p>The times below are for the sequence of 5 queries; individual query times are not reported. I did not do a line-by-line review of the execution plans since they seem to run well enough. We could get some extra mileage from cost model tweaks, especially for the numeric range conditions, but we will do this when somebody comes up with better times.</p> <p>First, about Virtuoso v5: Because there is a query in the set that specifies no condition on S or O and only P, this simply cannot be done with the default indices. With Virtuoso Cluster v6 it sort-of can, because v6 is more space efficient.</p> <p>So we added the index:</p> <blockquote> <code> create bitmap index <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1c364a58">rdf</a>_quad_pogs on rdf_quad (p, o, g, s); </code> </blockquote> <table> <tr> <td>Â </td> <td align="center"><b>Virtuoso v5 with<br /> gspo, ogps, pogs</b> </td> <td align="center"><b>Virtuoso Cluster v6 with <br />gspo, ogps</b> </td> <td align="center"><b>Virtuoso Cluster v6 with <br />gspo, ogps, pogs</b> </td> </tr> <tr> <td><b>cold</b> </td> <td align="center">210 s</td> <td align="center">136 s</td> <td align="center">33.4 s</td> </tr> <tr> <td><b>warm</b> </td> <td align="center">0.600 s</td> <td align="center">4.01 s</td> <td align="center">0.628 s</td> </tr> </table> <p>OK, so now let us do it without a graph being specified. For all platforms, we drop any existing indices, and --</p> <blockquote> <code> create table r2 (g iri_id_8, s, iri_id_8, p iri_id_8, o any, primary key (s, p, o, g)) <br /> alter index R2 on R2 partition (s int (0hexffff00)); <br /> <br /> log_enable (2); <br /> insert into r2 (g, s, p, o) select g, s, p, o from rdf_quad; <br /> <br /> drop table rdf_quad; <br /> alter table r2 rename RDF_QUAD; <br /> create bitmap index rdf_quad_opgs on rdf_quad (o, p, g, s) partition (o varchar (-1, 0hexffff)); <br /> create bitmap index rdf_quad_pogs on rdf_quad (p, o, g, s) partition (o varchar (-1, 0hexffff)); <br /> create bitmap index rdf_quad_gpos on rdf_quad (g, p, o, s) partition (o varchar (-1, 0hexffff)); </code> </blockquote> <p>The code is identical for v5 and v6, except that with v5 we use <code>iri_id (32 bit)</code> for the type, not <code>iri_id_8 (64 bit)</code>. We note that we run out of IDs with v5 around a few billion triples, so with v6 we have double the ID length and still manage to be vastly more space efficient.</p> <p>With the above 4 indices, we can query the <a href="http://dbpedia.org/resource/Data" id="link-id0x1bae4cd8">data</a> pretty much in any combination without hitting a full scan of any index. We note that all indices that do not begin with s end with s as a bitmap. This takes about 60% of the space of a non-bitmap index for data such as DBpedia.</p> <p>If you intend to do completely arbitrary RDF queries in Virtuoso, then chances are you are best off with the above index scheme.</p> <table> <tr> <td>Â </td> <td align="center"><b> Virtuoso v5 with<br /> gspo, ogps, pogs</b> </td> <td align="center"><b> Virtuoso Cluster v6 with <br /> spog, pogs, opgs, gpos </b> </td> </tr> <tr> <td><b>warm</b> </td> <td align="center">0.595 s</td> <td align="center">0.617 s</td> </tr> </table> <p>The cold times were about the same as above, so not reproduced.</p> <h3>Graph or No Graph?</h3> <p>It is in the SPARQL spirit to specify a graph and for pretty much any application, there are entirely sensible ways of keeping the data in graphs and specifying which ones are concerned by queries. This is why Virtuoso is set up for this by default.</p> <p>On the other hand, for the open web scenario, dealing with an unknown large number of graphs, enumerating graphs is not possible and questions like which graph of which source asserts x become relevant. We have two distinct use cases which warrant different setups of the database, simple as that.</p> <p>The latter use case is not really within the SPARQL spec, so implementations may or may not support this. For example <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x1cd2db78">Oracle</a> or Vertica would not do this well since they partition data according to graph or predicate, respectively. On the other hand, stores that work with one quad table, which is most of the ones out there, should do it maybe with some configuring, as shown above.</p> <p>Frameworks like Jena are not to my <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1b300390">knowledge</a> geared towards having a wildcard for graph, although I would suppose this can be arranged by adding some "super-graph" object, a graph of all graphs. I don't think this is directly supported and besides most apps would not need it.</p> <p>Once the indices are right, there is no difference between specifying a graph and not specifying a graph with the queries considered. With more complex queries, specifying a graph or set of graphs does allow some optimizations that cannot be done with no graph specified. For example, bitmap intersections are possible only when all leading key parts are given.</p> <h3>Conclusions</h3> <p>The best warm cache time is with v5; the five queries run under 600 ms after the first go. This is noted to show that all-in-memory with a single thread of execution is hard to beat.</p> <p>Cluster v6 performs the same queries in 623 ms. What is gained in parallelism is lost in latency if all operations complete in microseconds. On the other hand, Cluster v6 leaves v5 in the dust in any situation that has less than 100% hit rate. This is due to actual benefit from parallelism if operations take longer than a few microseconds, such as in the case of disk reads. Cluster v6 has substantially better data layout on disk, as well as fewer pages to load for the same content.</p> <p>This makes it possible to run the queries without the pogs index on Cluster v6 even when v5 takes prohibitively long.</p> <p>The morale of the story is to have a lot of RAM and space-efficient data representation.</p> <p>The DBpedia benchmark does not specify any random access pattern that would give a measure of sustained throughput under load, so we are left with the extremes of cold and warm cache of which neither is quite realistic.</p> <p>Chris Bizer and I have talked on and off about benchmarks and I have made suggestions that we will see incorporated into the Berlin SPARQL benchmark, which will, I believe, be much more informative.</p> <h3>Appendix: Query Text</h3> <p>For reference, the query texts specifying the graph are below. To run without specifying the graph, just drop the <code>FROM <<a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0x1c371db0">http</a>://dbpedia.org></code> from each query. The returned row counts are indicated below each query's text.</p> <blockquote> <code><pre> sparql SELECT ?p ?o FROM <http://dbpedia.org> WHERE { <http://dbpedia.org/resource/Metropolitan_Museum_of_Art> ?p ?o }; -- 1337 rows sparql PREFIX p: <http://dbpedia.org/property/> SELECT ?film1 ?actor1 ?film2 ?actor2 FROM <http://dbpedia.org> WHERE { ?film1 p:starring <http://dbpedia.org/resource/Kevin_Bacon> . ?film1 p:starring ?actor1 . ?film2 p:starring ?actor1 . ?film2 p:starring ?actor2 . }; -- 23910 rows sparql PREFIX p: <http://dbpedia.org/property/> SELECT ?artist ?artwork ?museum ?director FROM <http://dbpedia.org> WHERE { ?artwork p:artist ?artist . ?artwork p:museum ?museum . ?museum p:director ?director }; -- 303 rows sparql PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ?s ?homepage FROM <http://dbpedia.org> WHERE { <http://dbpedia.org/resource/Berlin> geo:lat ?berlinLat . <http://dbpedia.org/resource/Berlin> geo:long ?berlinLong . ?s geo:lat ?lat . ?s geo:long ?long . ?s foaf:homepage ?homepage . FILTER ( ?lat <= ?berlinLat + 0.03190235436 && ?long >= ?berlinLong - 0.08679199218 && ?lat >= ?berlinLat - 0.03190235436 && ?long <= ?berlinLong + 0.08679199218) }; -- 56 rows sparql PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX p: <http://dbpedia.org/property/> SELECT ?s ?a ?homepage FROM <http://dbpedia.org> WHERE { <http://dbpedia.org/resource/New_York_City> geo:lat ?nyLat . <http://dbpedia.org/resource/New_York_City> geo:long ?nyLong . ?s geo:lat ?lat . ?s geo:long ?long . ?s p:architect ?a . ?a foaf:homepage ?homepage . FILTER ( ?lat <= ?nyLat + 0.3190235436 && ?long >= ?nyLong - 0.8679199218 && ?lat >= ?nyLat - 0.3190235436 && ?long <= ?nyLong + 0.8679199218) }; -- 13 rows </pre> </code> </blockquote> </div>
SPARQL at WWW 2008
http://www.openlinksw.com/blog/vdb/blog/?date=2008-04-30#1354
2008-04-30T16:28:10Z
2008-08-28T11:26:06.000004-04:00
<div> <div style="display:none;">SPARQL at WWW 2008</div> <p>Andy Seaborne and Eric Prud'hommeaux, editors of the <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x183b13a8">SPARQL</a> recommendation, convened a SPARQL birds of a feather session at <a href="http://www2008.org/" id="link-id0xd31c2d0">WWW 2008</a>. The administrative outcome was that implementors could now experiment with extensions, hopefully keeping each other current about their efforts and that towards the end of 2008, a new W3C working group might begin formalizing the experiences into a new SPARQL spec.</p> <p>The session drew a good crowd, including many users and developers. The wishes were largely as expected, with a few new ones added. Many of the wishes already had diverse implementations, however most often without interop. I will below give some comments on the main issues discussed.</p> </div> <li> <p> <b>SPARQL Update</b> - This is likely the most universally agreed upon extension. Implementations exist, largely along the lines of Andy Seaborne's SPARUL spec, which is also likely material for a W3C member submission. The issue is without much controversy; transactions fall outside the scope, which is reasonable enough. With triple stores, we can define things as combinations of inserts and deletes, and isolation we just leave aside. If anything, operating on a transactional platform such as <a href="http://virtuoso.openlinksw.com" id="link-id0x1d442cd8">Virtuoso</a>, one wishes to disable transactions for any operations such as bulk loads and long-running inserts and deletes. Transactionality has pretty much no overhead for a few hundred rows, but for a few hundred million rows the cost of locking and rollback is prohibitive. With Virtuoso, we have a row auto-commit mode which we recommend for use with <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xec62c58">RDF</a>: It commits by itself now and then, optionally keeping a roll forward log, and is transactional enough not to leave half triples around, i.e., inserted in one index but not another.</p> <p>As far as we are concerned, updating physical triples along the SPARUL lines is pretty much a done deal.</p> <p>The matter of updating relational <a href="http://dbpedia.org/resource/Data" id="link-id0x19f995c8">data</a> mapped to RDF is a whole other kettle of fish. On this, I should say that RDF has no special virtues for expressing transactions but rather has a special genius for integration. Updating is best left to web service interfaces that use <a href="http://dbpedia.org/resource/SQL" id="link-id0x125b2b98">SQL</a> on the inside. Anyway, updating union views, which most mappings will be, is complicated. Besides, for transactions, one usually knows exactly what one wishes to update.</p> </li> <li> <p> <b>Full Text</b> - Many people expressed a desire for full text access. Here we run into a deplorable confusion with regexps. The closest SPARQL has to full text in its native form is regexps, but these are not really mappable to full text except in rare special cases and I would despair of explaining to an end user what exactly these cases are. So, in principle, some regexps are equivalent to full text but in practice I find it much preferable to keep these entirely separate.</p> <p>It was noted that what the users want is a text box for search words. This is a front end to the CONTAINS predicate of most SQL implementations. Ours is MS SQL Server compatible and has a SPARQL version called <code>bif:contains</code>. One must still declare which triples one wants indexed for full text, though. This admin overhead seems inevitable, as text indexing is a large overhead and not needed by all applications.</p> <p>Also, text hits are not boolean; usually they come with a hit score. Thus, a SPARQL extension for this could look like </p> <blockquote> <code>select * where { ?thing has_description ?d . ?d ftcontains "gizmo" ftand "widget" score ?score . }</code> </blockquote> <p>This would return all the subjects, descriptions, and scores, from subjects with a has_description property containing widget and gizmo. Extending the basic pattern is better than having the match in a filter, since the match binds a variable.</p> <p>The <a href="http://dbpedia.org/resource/XQuery" id="link-id0xfec6788">XQuery</a>/<a href="http://dbpedia.org/resource/XPath" id="link-id0x1a789e38">XPath</a> groups have recently come up with a full-text spec, so I used their style of syntax above. We already have a full-text extension, as do some others. but for standardization, it is probably most appropriate to take the XQuery work as a basis. The XQuery full-text spec is quite complex, but I would expect most uses to get by with a small subset, and the structure seems better thought out, at first glance, than the more ad-hoc implementations in diverse SQLs.</p> <p>Again, declaring any text index to support the search, as well as its timeliness or transactionality, are best left to implementations.</p> </li> <li> <p> <b>Federation</b> - This is a tricky matter. <a href="http://jena.sourceforge.net/ARQ/" id="link-id0xba487f0">ARQ</a> has a SPARQL extension for sending a nested set of triple patterns to a specific end-point. The <a href="http://darq.sourceforge.net/" id="link-id0xf8a5ab0">DARQ</a> project has something more, including a selectivity model for SPARQL.</p> <p>With federated SQL, life is simpler since after the views are expanded, we have a query where each table is at a known server and has more or less known statistics. Generally, execution plans where as much work as possible is pushed to the remote servers are preferred, and modeling the latencies is not overly hard. With SPARQL, each triple pattern could in principle come from any of the federated servers. Associating a specific end-point to a fragment of the query just passes the problem to the user. It is my guess that this is the best we can do without getting very elaborate, and possibly buggy, end-point content descriptions for routing federated queries.</p> <p>Having said this, there remains the problem of join order. I suggested that we enhance the protocol by allowing asking an end-point for the query cost for a given SPARQL query. Since they all must have a cost model for optimization, this should not be an impossible request. A time cost and estimated cardinality would be enough. Making statistics available <i>Ã la</i> DARQ was also discussed. Being able to declare cardinalities expected of a remote end-point is probably necessary anyway, since not all will implement the cost model interface. For standardization, agreeing on what is a proper description of content and cardinality and how fine grained this must be will be so difficult that I would not wait for it. A cost model interface would nicely hide this within the end-point itself.</p> <p>With Virtuoso, we do not have a federated SPARQL scheme but we could have the ARQ-like service construct. We'd use our own cost model with explicit declarations of cardinalities of the remote data for guessing a join order. Still, this is a bit of work. We'll see.</p> <p>For practicality, the service construct coupled with join order hints is the best short term bet. Making this pretty enough for standardization is not self-evident, as it requires end-point description and/or cost model hooks for things to stay declarative.</p> </li> <li> <p> <b>End-point description</b> - This question has been around for a while; I have <a href="http://www.openlinksw.com/weblog/oerling/?id=1085" id="link-id101d3440">blogged about it earlier</a>, but we are not really at a point where there would be even rough consensus about an end-point ontology. We should probably do something on our own to demonstrate some application of this, as we host lots of <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x162d0de8">linked open data</a> sets.</p> </li> <li> <p> <b>SQL equivalence</b> - There were many requests for aggregation, some for subqueries and nesting, expressions in select, negation, existence and so on. I would call these all SQL equivalence. One use case was taking all the teams in the database and for all with over 5 members, add the big_team class and a property for member count.</p> <p>With Virtuoso, we could write this as -- </p> <blockquote> <code>construct { ?team a big_team . ?team member_count ?ct } from ... where {?team a team . { select ?team2 count (*) as ?ct where { ?m member_of ?team2 } . filter (?team = ?team2 and ? ct > 5) }}</code> </blockquote> <p>We have pretty much all the SQL equivalence features, as we have been working for some time at translating the <a href="http://dbpedia.org/resource/TPC-H" id="link-id0x11c870c8">TPC-H</a> workload into SPARQL.</p> <p>The usefulness of these things is uncontested but standardization could be hard as there are subtle questions about variable scope and the like.</p> </li> <li> <p> <b>Inference</b> - The SPARQL spec does not deal with transitivity or such matters because it is assumed that these are handled by an underlying inference layer. This is however most often not so. There was interest in more fine grained control of inference, for example declaring that just one property in a query would be transitive or that subclasses should be taken into account in only one triple pattern. As far as I am concerned, this is very reasonable, and we even offer extensions for this sort of thing in Virtuoso's SPARQL. This however only makes sense if the inference is done at query time and pattern by pattern. For instance, if forward chaining is used, this no longer makes sense. Specifying that some forward chaining ought to be done at query time is impractical, as the operation can be very large and time consuming and it is the DBA's task to determine what should be stored and for how long, how changes should be propagated, and so on. All these are application dependent and standardizing will be difficult.</p> <p>Support for RDF features like lists and bags would all fall into the functions an underlying inference layer should perform. These things are of special interest when querying <a href="http://dbpedia.org/resource/Web_Ontology_Language" id="link-id0x156f3830">OWL</a> models, for example.</p> </li> <li> <p> <b>Path expressions</b> - Path expressions were requested by a few people. We have implemented some, as in </p> <blockquote> <code>?product+?has_supplier+>s_name = "Gizmos, Inc.".</code> </blockquote> This means that one supplier of product has name "Gizmo, Inc.". This is a nice shorthand but we run into problems if we start supporting repetitive steps, optional steps, and the like.</li> <p>In conclusion, update, full text, and basic counting and grouping would seem straightforward at this point. Nesting queries, value subqueries, views, and the like should not be too hard if an agreement is reached on scope rules. Inference and federation will probably need more experimentation but a lot can be had already with very simple fine grained control of backward chaining, if such applies, or with explicit end-point references and explicit join order. These are practical but not pretty enough for committee consensus, would be my guess. Anyway, it will be a few months before anything formal will happen.</p>
Linked Data and Information Architecture
http://www.openlinksw.com/blog/vdb/blog/?date=2008-04-29#1350
2008-04-29T14:37:22Z
2008-04-29T17:18:21.000048-04:00
<div> <div style="display:none;">Linked Data and Information Architecture</div> <p>We had a workshop on <a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1437ac70">Linked Open Data</a> (<a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0x1315f788">LOD</a>) last week in <a href="http://www2008.org/" id="link-id0x13737468">Beijing</a>. You can see the papers in <a href="http://events.linkeddata.org/ldow2008/#program" id="link-id10651ab8">the program</a>. The event was a success with plenty of good talks and animated conversation. I will not go into every paper here but will comment a little on the conversation and draw some technology requirements going forward.</p> <p>Tim Berners-Lee showed a read-write version of <a href="http://dig.csail.mit.edu/2005/ajar/release/tabulator/0.8/tab.html" id="link-id0x15633520">Tabulator</a>. This raises the question of updating on the <a href="http://dbpedia.org/resource/Data" id="link-id0x1350a178">Data</a> Web. The consensus was that one could assert what one wanted in one's own space but that others' spaces would be read-only. What spaces one considered relevant would be the user's or developer's business, as in the document web.</p> <p>It seems to me that a significant use case of LOD is an open-web situation where the user picks a broad read-only "data wallpaper" or backdrop of assertions, and then uses this combined with a much smaller, local, writable data set. This is certainly the case when editing data for publishing, as in Tim's demo. This will also be the case when developing mesh-ups combining multiple distinct data sets bound together by sets of SameAs assertions, for example. Questions like, "What is the minimum subset of n data sets needed for deriving the result?" will be common. This will also be the case in applications using proprietary data combined with open data.</p> <p>This means that databases will have to deal with queries that specify large lists of included graphs, all graphs in the store or all graphs with an exclusion list. All this is quite possible but again should be considered when architecting systems for an open <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0xa27bae8">linked data</a> <a href="http://dbpedia.org/resource/Giant_Global_Graph" id="link-id0x155c3f18">web</a>.</p> <p>"There is data but what can we really do with it? How far can we trust it, and what can we confidently decide based on it?"</p> <p>As an answer to this question, <a href="http://zitgist.com/about/" id="link-id0xd447580">Zitgist</a> has compiled the <a href="http://umbel.org/about/" id="link-id0x14735008">UMBEL</a> taxonomy using <a href="http://dbpedia.org/resource/SKOS" id="link-id0x15ab1c48">SKOS</a>. This draws on Wikipedia, Open CYC, Wordnet, and <a href="http://www.mpi-inf.mpg.de/~suchanek/downloads/yago/" id="link-id0x15d5aa88">YAGO</a>, hence the acronym WOWY. UMBEL is both a taxononmy and a set of instance data, containing a large set of <a href="http://dbpedia.org/resource/Named_entity_recognition" id="link-id0x9fe45d98">named entities</a>, including persons, organizations, geopolitical entities, and so forth. By extracting references to this set of named entities from documents and correlating this to the taxonomy, one gets a good idea of what a document (or part thereof) is about.</p> <p>Kingsley presented this in the Zitgist demo. This is our answer to the criticism about <a href="http://dbpedia.org/resource/DBpedia" id="link-id0xa1920800">DBpedia</a> having errors in classification. DBpedia, as a bootstrap stage, is about giving names to all things. Subsequent efforts like UMBEL are about refining the relationships.</p> <p>"Should there be a global <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x12cd5290">URI</a> dictionary?"</p> <p>There was a talk by Paolo Bouquet about <a href="http://dbpedia.org/resource/Entity" id="link-id0x12d03400">Entity</a> Name System, a a sort of data DNS, with the purpose of associating some description and rough classification to URIs. This would allow discovering URIs for reuse. I'd say that this is good if it can cut down on the SameAs proliferation and if this can be widely distributed and replicated for resilience, <i>Ã la</i> DNS. On the other hand, it was pointed out that this was not quite in the LOD spirit, where parties would mint their own dereferenceable URIs, in their own domains. We'll see.</p> <p>"What to do when identity expires?"</p> <p>Giovanni of Sindice said that a document should be removed from search if it was no longer available. Kingsley pointed out that resilience of reference requires some way to recover data. The data web cannot be less resilient than the document web, and there is a point to having access to history. He recommended hooking up with the <a href="http://dbpedia.org/resource/Internet" id="link-id0x143e4130">Internet</a> Archive, since they make long term persistence their business. In this way, if an application depends on data, and the URIs on which it depends are no longer dereferenceable or or provide content from a new owner of the domain, those who need the old version can still get it and host it themselves.</p> <p>It is increasingly clear that OWL SameAs is both the blessing and bane of linked data. We can easily have tens of URIs for the same thing, especially with people. Still, these should be considered the same.</p> <p>Returning every synonym in a query answer hardly makes sense but accepting them as input seems almost necessary. This is what we do with <a href="http://virtuoso.openlinksw.com" id="link-id0x15a2a930">Virtuoso</a>'s SameAs support. Even so, this can easily double query times even when there are no synonyms.</p> <p>Be that as it may, SameAs is here to stay; just consider the mapping of DBpedia to Geonames, for example.</p> <p>Also, making aberrant SameAs statements can completely poison a data set and lead to absurd query results. Hence choosing which SameAs assertions from which source will be considered seems necessary. In an open web scenario, this leads inevitably to multi-graph queries that can be complex to write with regular <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x12bb8ce8">SPARQL</a>. By extension, it seems that a good query would also include the graphs actually used for deriving each result row. This is of course possible but has some implications on how databases should be organized.</p> <p>Yves Raymond gave a talk about deriving identity between Musicbrainz and Jamendo. I see the issue as a core question of linked data in general. The algorithm Yves presented started with attribute value similarities and then followed related entities. Artists would be the same if they had similar names and similar names of albums with similar song titles, for example. We can find the same basic question in any analysis, for example, looking at how news reporting differs between media, supposing there is adequate entity extraction.</p> <p>There is basic graph diffing in <a href="http://data.semanticweb.org/conference/iswc-aswc/2007/tracks/research/papers/533/html" id="link-id0x153c1fa8">RDFSync</a>, for example. But here we are expanding the context significantly. We will traverse references to some depth, allow similarity matches, SameAs, and so forth. Having presumed identity of two URIs, we can then look at the difference in their environment to produce a human readable summary. This could then be evaluated for purposes of analysis or of combining content.</p> <p>At first sight, these algorithms seem well parallelizable, as long as all threads have access to all data. For scaling, this means a probably message-bound distributed algorithm. This is something to look into for the next stage of linked data.</p> <p>Some inference is needed, but if everybody has their own choice of data sets to query, then everybody would also have their own entailed triples. This will make for an explosion of entailed graphs if forward chaining is used. Forward chaining is very nice because it keeps queries simple and easy to optimize. With Virtuoso, we still favor backward chaining since we expect a great diversity of graph combinations and near infinite volume in the open web scenario. With private repositories of slowly changing data put together for a special application, the situation is different.</p> <p>In conclusion, we have a real LOD movement with actual momentum and a good idea of what to do next. The next step is promoting this to the broader community, starting with <a href="http://www.linkeddataplanet.com/" id="link-id0x155d1d00">Linked Data Planet</a> in New York in June.</p> </div>
On Sem Web Search
http://www.openlinksw.com/blog/vdb/blog/?date=2008-04-29#1349
2008-04-29T14:37:21Z
2008-10-02T11:37:12.000008-04:00
<div> <div style="display:none;">On Sem Web Search</div> <p> <i>"I give the search keywords and you give me a <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1a603f18">SPARQL</a> end-point and a query that will get the <a href="http://dbpedia.org/resource/Data" id="link-id0x1bda5c00">data</a>."</i> </p> <p>Thus did one SPARQL user describe the task of a semantic/data web search engine.</p> <p>In <a href="http://www.openlinksw.com/weblog/oerling/?id=1336" id="link-idff98750">a previous post</a>, I suggested that if the data web were the size of the document web, we'd be looking at two orders of magnitude more search complexity. It just might be so.</p> <p>In the conversation, I pointed out that a search engine might have a copy of everything and even a capability to do SPARQL and full text on it all, yet still the users would be better off doing the queries against the SPARQL end-points of the data publishers. It is a bit like the fact that not all web browsing runs off Google's cache. With the data web, the point is even more pronounced, as serving a hit from Google's cache is a small operation but a complex query might be a very large one.</p> <p>Yet, the data web is about ad-hoc joining between data sets of different origins. Thus a search engine of the data web ought to be capable of joining also, even if large queries ought to be run against individual publishers' end-points or the user's own data warehouse.</p> <p>For ranking, the general consensus was that no single hit-ranking would be good for the data web. Thus word frequency-based hit-scores are OK for text hits but more is not obvious. I would think that some link analysis could apply but this will take some more experimentation.</p> <p>For search summaries, if we have splitting of data sets into small fragments <i>Ã la</i> <a href="http://sindice.com/" id="link-id0x1d2b7288">Sindice</a>, search summaries are pretty much the same as with just text search. If we store triples, then we can give text style summaries of text hits in literals and Fresnel lens views of the structured data around the literal. For showing a page of hits, the lenses must abbreviate heavily but this is still feasible. The engine would know about the most common ontologies and summarize instance data accordingly.</p> <p>Chris Bizer pointed out that trust and provenance are critical, especially if an answer is arrived at by joining multiple data sets. The trust of the conclusion is no greater than that of the weakest participating document. Different users will have different trusted sources.</p> <p>A mature data web search engine would combine a provenance/trust specification, a search condition consisting of SPARQL or full text or both, and a specification for hit rank. Again, most searches would use defaults, but these three components should in principle be orthogonally specifiable.</p> <p>Many places may host the same data set either for download or SPARQL access. The <a href="http://dbpedia.org/resource/Uniform_Resource_Identifier" id="link-id0x1b2317d0">URI</a> of the data set is not its <a href="http://dbpedia.org/resource/Uniform_Resource_Locator" id="link-id0x1c55dd68">URL</a>. Different places may further host multiple data sets on one end-point. Thus the search engine ought to return all end-points where the set is to be found. The end-points themselves ought to be able to say what data sets they contain, under what graph IRIs. Since there is no consensus about end-point self description, this too would be left to the search engine. In practice, this could be accomplished by extending Sindice's semantic site map specification. A possible query would be to find an end-point containing a set of named data sets. If none were found, the search engine itself could run a query joining all the sets since it at least would hold them all.</p> <p>Since many places will host sets like Wordnet or Uniprot, indexing these once for each copy hardly makes sense. Thus a site should identify its data by the data set's URI and not the copy's URL.</p> <p>It came up in the discussion that search engines should share a ping format so that a single message format would be enough to notify any engine about data being updated. This is already partly the case with Sindice and <a href="http://www.pingthesemanticweb.com/" id="link-id0xa405ebd0">PTSW</a> (<a href="http://www.pingthesemanticweb.com/" id="link-id0x1c051a00">PingTheSemanticWeb</a>) sharing a ping format. </p> <p>Further, since it is no trouble to publish a copy of the 45G Uniprot file but a fair amount of work to index it, search engines should be smart about processing requests to index things, since these can amount to a denial of service attack. </p> <p>Probably very large data sets should be indexed only in the form supplied by their publisher, and others hosting copies would just state that they hold a copy. If the claim to the copy proved false, users could complain and the search engine administrator would remove the listing. It seems that some manual curating cannot be avoided here. </p> <h2>On Data Web Search Business Model</h2> <p>It seems there can be an overlap between the data web search and the data web hosting businesses. For example, Talis rents space for hosting <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1a60c7e0">RDF</a> data with SPARQL access. A search engine should offer basic indexing of everything for free, but could charge either data publishers or end users for running SPARQL queries across data sets. These do not have the nicely anticipatable and fairly uniform resource consumption of text lookups. In this manner, a search provider could cost-justify the capacity for allowing arbitrary queries. </p> <p>The value of the data web consists of unexpected joining. Such joining takes place most efficiently if the sources are at least in some proximity, for example in the same data center. Thus the search provider could monetize functioning as the database provider for mesh-ups. In the document web, publishing pages is very simple and there is no great benefit from co-locating search and pages, rather the opposite. For the data web, the hosting with SPARQL and all is more complex and resembles providing search. Thus providing search can combine with providing SPARQL hosting, once we accept in principle that search should have arbitrary inter-document joining, even if it is at an extra premium.</p> <p>The present search business model is advertising. If the data web is to be accessed by automated agents such as mesh-up code, display of ads is not self-evident. This is quite separate from the fact that semantics can lead to better ad targeting.</p> <p>One model would be to do text lookups for free from a regular web page but show ads, just a la Google search ads. Using the service via web services for text or SPARQL would have a cost paid by the searching or publishing party and would not be financed by advertising.</p> <p>In the case of data used in value-add data products (mesh-ups) that have financial value to their users, the original publisher of the data could even be paid for keeping the data up-to-date. This would hold for any time-sensitive feeds like news or financial feeds. Thus the hosting/search provider would be a broker of data-use fees and the data producer would be in the position of an AdSense inventory owner, i.e., a web site which shows AdSense ads. Organizing this under a hub providing back-office functions similar to an ad network could make sense even if the actual processing were divided among many sites.</p> <p>Kingsley has repeatedly formulated the core value proposition of the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x3728a2f8">semantic web</a> in terms of dealing with <a href="http://dbpedia.org/resource/Information" id="link-id0x1bbcbeb8">information</a> overload: There is the real-time enterprise and the real-time individual and both are beasts of perception. Their image is won and lost in the <a href="http://dbpedia.org/resource/Internet" id="link-id0x1843b020">Internet</a> online conversation space. We know that allegations, even if later proven false, will stick if left unchallenged. The function of semantics on the web is to allow one to track and manage where one stands. In fact, Garlik has made a business of just this, but now from a privacy and security angle. The <a href="http://www.garlik.com/" id="link-id0x1aa76ab0">Garlik DataPatrol</a> harvests data from diverse sources and allows assessing vulnerability to identity theft, for example.</p> <p>If one is in the business of collating all the structured data in the world, as a data web search engine is, then providing custom alerts for both security or public image management is quite natural. This can be a very valuable service if it works well.</p> <p>At OpenLink, we will now experiment with the Sindice/<a href="http://zitgist.com/about/" id="link-id0x18800228">Zitgist</a>/PingTheSemanticWeb content. This is a regular part of the productization of <a href="http://virtuoso.openlinksw.com" id="link-id0x1adf39c8">Virtuoso</a>'s cluster edition. We expect to release some results in the next 4 weeks.</p> </div>
WWW 2008
http://www.openlinksw.com/blog/vdb/blog/?date=2008-04-29#1348
2008-04-29T14:37:20Z
2008-04-29T13:35:23-04:00
<div> <div style="display:none;">WWW 2008</div> <p>Following my return from WWW 2008 in <a href="http://www2008.org/" id="link-id0x9ff7d5d0">Beijing</a>, I will write a series of <a href="http://dbpedia.org/resource/Blog" id="link-id0x9e4a7650">blog</a> posts discussing diverse topics that were brought up in presentations and conversations during the week.</p> <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x9e7ae398">Linked data</a> was our main interest in the conference and there was a one day workshop on this, unfortunately overlapping with a day of W3C Advisory Committee meetings. Hence Tim Berners-Lee, one of the chairs of the workshop, could not attend for most of the day. Still, he was present to say that "<a href="http://community.linkeddata.org/dataspace/organization/lod#this" id="link-id0xa287d38">Linked open data</a> is the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x15372940">semantic web</a> and the web done as it ought to be done." <p>For my part, I will draw some architecture conclusions from the different talks and extrapolate about the requirements on database platforms for linked data.</p> <p>Chris Bizer predicted that 2008 would be the year of <a href="http://dbpedia.org/resource/Data" id="link-id0xa1454c58">data</a> web search, if 2007 was the year of <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xa0f73c50">SPARQL</a>. This may be the case, as linked data is now pretty much a reality and the questions of discovery become prevalent. There was a birds-of-a-feather session on this and I will make some comments on what we intend to explore in bridging between the text index based semantic web search engines and SPARQL.</p> <p>Andy Seaborne convened a birds-of-a-feather session on the future of SPARQL. Many of the already anticipated and implemented requirements were confirmed and a few were introduced. A separate blog post will discuss these further.</p> <p>From the various discussions held throughout the conference, we conclude that plug-and-play operation with the major semantic web frameworks of Jena, Sesame, and Redland, is our major immediate-term deliverable. Our efforts in this direction thus far are insufficient and we will next have these done with the right supervision and proper interop testing. The issues are fortunately simple but doing things totally right require some small server side support and some <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0xa5d4d5b8">JDBC</a>/<a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0x9dc28d10">ODBC</a> tweaks, so to the interested, we advise to wait for an update to be published on this blog.</p> <p>I further had a conversation with Andy Seaborne about using Jena reasoning capabilities with <a href="http://virtuoso.openlinksw.com" id="link-id0xa2754050">Virtuoso</a> and generally the issues of "impedance mismatch" between reasoning and typical database workloads. More on this later. </p> </div>
TPC H as Linked Data (Updated 2)
http://www.openlinksw.com/blog/vdb/blog/?date=2008-03-06#1322
2008-03-06T16:34:12Z
2008-08-28T11:26:03.000004-04:00
<div> <div style="display:none;">TPC H as Linked Data (Updated 2)</div> <p>We have a new demo online at <a href="http://demo.openlinksw.com/tpc-h" id="link-id1829c9a0">http://demo.openlinksw.com/tpc-h</a>. This takes the industry standard <a href="http://dbpedia.org/resource/TPC-H" id="link-id0xf0fff10">TPC-H</a> benchmark <a href="http://dbpedia.org/resource/Data" id="link-id0x1387cd90">data</a> and presents it as <a href="http://dbpedia.org/resource/Linked_Data" id="link-id0x152407c0">linked data</a> with a <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x11657940">SPARQL</a> end point and dereferenceable URIs. </p> <p>This is an example of using <a href="http://virtuoso.openlinksw.com" id="link-id0xe560628">Virtuoso</a>'s relational-to-<a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xa07e2588">RDF</a> mapping for publishing business data, for browsing using the linked data principles and opening it to analytics queries in SPARQL.</p> <p> As noted before, we have extended SPARQL with aggregation and nested queries, thus making it a viable <a href="http://dbpedia.org/resource/SQL" id="link-id0xa3666b00">SQL</a> substitute for decision support queries. </p> <p>The article at <a href="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSTPCHLinkedData" id="link-id10799d10">http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSTPCHLinkedData</a> gives details and the source code for the implementation.</p> <p> We are still working on some aspects of the more complex TPC-H queries, thus the demo is not complete with all the 22 queries. This is however enough to see a representative sample of how analytics queries work with SPARQL and Virtuoso's SQL-to-RDF mapping. The demo will be part of the next Virtuoso Open Source download, probably out next week.</p> </div>
LUBM results with Virtuoso 6.0
http://www.openlinksw.com/blog/vdb/blog/?date=2008-02-04#1309
2008-02-04T10:26:17Z
2008-08-28T12:06:11-04:00
<div> <div style="display:none;">LUBM results with Virtuoso 6.0</div> <p>We have now run the LUBM benchmark on <a href="http://virtuoso.openlinksw.com" id="link-id0x1c2197e8">Virtuoso</a> v6, with the same configuration <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1302" id="link-id107f0238">as discussed last Friday</a>.</p> <p>We had a database of 8000 universities, and we ran 8 clients on slices of 100, 1000 and 8000 universities — same <a href="http://dbpedia.org/resource/Data" id="link-id0x7d0e498">data</a> but different sizes of working set.</p> <blockquote> <pre> 100 universities: 35.3 qps 1000 universities: 26.3 qps 8000 universities: 13.1 qps</pre></blockquote> <p>The 100 universities slice is about the same as with v5.0.5 (35.3 vs 33.1 qps). <br />The 8000 universities set is almost 3x better (13.1 vs. 4.8 qps).</p> <p>This comes from the fact that the v6 database takes half of the space of the v5.0.5 one. Further, this is with 64-bit IDs for everything. If the 5.5 database were with 64-bit IDs, we'd have a difference of over 3x. This is worth something if it lets you get by with only 1 terabyte of RAM for the 100 billion triple application, instead of 3 TB.</p> <p> <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1358" id="link-id15fb4d38">In a few more days</a>, we'll give the results for Virtuoso v6 Cluster.</p> </div>
Latest LUBM Benchmark results for Virtuoso
http://www.openlinksw.com/blog/vdb/blog/?date=2008-02-01#1305
2008-02-01T15:16:59Z
2008-08-28T12:06:08-04:00
<div> <div style="display:none;">Latest LUBM Benchmark results for Virtuoso</div> <p>We have now taken a close look at the query side of the LUBM benchmark, <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1296" id="link-id10a98120">as promised a couple of blog posts ago.</a> </p> <p>We load 8000 universities and run a query mix consisting of the 14 LUBM queries with different numbers of clients against different portions of the database.</p> <p>When it is all in memory, we get 33 queries per second with 8 concurrent clients; when it is so I/O bound that 7.7 of 8 threads wait for disk, we get 5 qps. This was run in 8G RAM with 2 Xeon 5130.</p> <p>We adapted some of the queries so that they do not run over the whole database. In terms of retrieving triples per second, this would be about 330000 for the rate of 33 qps, with 4 cores at 2GHz. This is a combination of random access and linear scans and bitmap merge intersections; lookups for non-found triples are not counted. The rate of random lookups alone based on known G, S, P, O, without any query logic, is about 250000 random lookups per core per second.</p> <p>The article <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VOSArticleLUBMBenchmark" id="link-id10237708">LUBM and Virtuoso</a> gives the details.</p> <p>In the process of going through the workload we made some cost model adjustments and optimized the bitmap intersection join. In this way we can quickly determine which subjects are, for example, professors holding a degree from a given university. So the benchmark served us well in that it provided an incentive to further optimize some things.</p> <p>Now, what has been said about <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x12cba3d8">RDF</a> benchmarking previously still holds. What does it mean to do so many LUBM queries per second? What does this say about the capacity to run an online site off RDF <a href="http://dbpedia.org/resource/Data" id="link-id0x9e043f38">data</a>? Or about <a href="http://dbpedia.org/resource/Information" id="link-id0x10c1a4f8">information</a> integration? Not very much. But then this was not the aim of the authors either.</p> <p>So we still need to make a benchmark for online queries and search, and another for E-science and business intelligence. But we are getting there.</p> <p>In the immediate future, we have the general availability of <a href="http://virtuoso.openlinksw.com" id="link-id0x9ec5e620">Virtuoso</a> Open Source 5.0.5 early next week. This comes with a LUBM test driver and a test suite running against the LUBM qualification database.</p> <p>After this we will give some numbers for the cluster edition with LUBM and <a href="http://dbpedia.org/resource/TPC-H" id="link-id0xcd7ec48">TPC-H</a>.</p> </div>
RDBMS to RDF Mapping Workshop, and Benchmarks
http://www.openlinksw.com/blog/vdb/blog/?date=2007-11-21#1271
2007-11-21T13:07:03Z
2008-04-25T16:29:53-04:00
<div> <div style="display:none;">RDBMS to RDF Mapping Workshop, and Benchmarks</div> <p>I was recently in Boston for the <a href="http://www.w3.org/2007/03/RdfRDB/" id="link-id10f990b0">Mapping Relational Data to RDF workshop</a> of the W3C.</p> <p>The common feeling was that mapping everything to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1c343278">RDF</a> and querying it in terms of a generic domain ontology, mapped on demand into whatever line of business systems, would be very good if it only could be done. However, since this is not so easily done, the next best is to extract the <a href="http://dbpedia.org/resource/Data" id="link-id0xb6f01d0">data</a> and then warehouse it as RDF.</p> <p>The obstacles perceived were of the following types:</p> <ul> <li> <p>Lack of quality in the data. The different line of business systems do not in and of themselves hold enough semantics. If the meaning of data columns in relational tables were really known and explicit, these could be meaningfully used for joining across systems. But this is more complex than just mapping the metal <i>lead</i> to the chemical symbol <i>Pb</i> and back.</p> </li> <li> <p>Lack of performance in RDF storage. Data sets even in the tens-of-millions of triples do not run very well in some stores. Well, we had the Banff life sciences demo with 450M triples in a small server box running <a href="http://virtuoso.openlinksw.com" id="link-id0x1ca1c488">Virtuoso</a>, so this is not universal, plus of course we are coming up with a whole different order of magnitude, as often discussed on this <a href="http://dbpedia.org/resource/Blog" id="link-id0xb4dc850">blog</a>.</p> </li> <li> <p>Lack of functionality in mapping and possibly lack of pushing through enough of the query processing to the underlying data stores.</p> </li> </ul> <p>Personally, I am quite aware of what to do with regard to performance of mapping and storage, and see these as eminently solvable issues. After all, we have a great investment of talent in databases in general and it can be well deployed towards RDF, as we have been doing these past couple of years. So we talk about the promise of a 360-degree view of <a href="http://dbpedia.org/resource/Information" id="link-id0x1ae64448">information</a>, with RDF being the top layer. Everybody agrees that this is a nice concept. But this is a nice concept especially when it can do the things that are the most common baseline expectation of any regular DBMS, i.e., aggregation, grouping, sub-queries, VIEWs. Now, I would not go sell a DBMS that has no <code>COUNT</code> operator to a data warehousing shop.</p> <p>The fact that OpenLink and <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x1aa10fd8">Oracle</a> allow RDF inside <a href="http://dbpedia.org/resource/SQL" id="link-id0xa26d330">SQL</a>, and OpenLink even adds native aggregates and grouping to <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1d81d990">SPARQL</a>, fixes the problem with regard to specific products, but leaves the standardization issue open. Of course, any vendor will solve these questions one way or another because a database with no aggregation is a non-starter.</p> <p>I talked to Lee Feigenbaum, chair of the W3C DAWG, about the question of aggregates and general BI capabilities in SPARQL. He told me that, prior to his time with the DAWG, these were left out because they conflicted with the <a href="http://dbpedia.org/resource/Open_world_assumption" id="link-id0x1be3ab98">open-world</a> assumption around RDF: You cannot count a set because by definition you do not know that you have all the members, the world being open and all that.</p> <p>Say what? Talk about the road to hell being paved with good intentions. Now, this is in no way Lee's or the present day DAWG's fault; as a member myself, I can attest to the good work and would under no circumstances wish any delays or revisions to SPARQL at this point. I am just pointing out a matter that all implementations should address, as a sort of precondition of entry into the real world IS space. If this can be done interoperably, so much the better.</p> <p>Now, out of the deliberations at the Boston workshop arose at least two ideas for follow-up activity.</p> <p>The first was an incubator group for RDF store and mapping benchmarking. This is very appropriate in order to dispel the bad name RDF storage and querying performance has been saddled with. As a first step in this direction, I will outline a <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1269" id="link-id10306200">social web oriented benchmark</a> on this blog.</p> <p>The second activity was an <a href="http://www.w3.org/2005/Incubator/rdb2rdf/" id="link-id10150a58">incubator group for preparing standardization of mapping methodologies from relational schemas to RDF</a>. We will be active on this as well.</p> <p>The two offshoots appear logically separate but are not necessarily so in practice. A benchmark is after all something that is supposed to promote a technology to a user base. The user base seems to wish to put all online systems and data warehouses under a common top level RDF model and then query away, introducing no further replication of data or performance cost or ETL latencies.</p> <p>Updating would also be nice but even query only would be very good. Personally, I'd say the RDF strength is all on the query side. Transactions are taken care of well enough by what there already is, RDF stands out in integration and the ad-hoc and discovery side of the matter. Given this, we expect the value to be consumed in a heterogeneous, multi-database, federated environment. Thus a benchmark should measure this aspect of the use-case. With the right mapping and queries, we could probably demonstrate the added cost of RDF to be very low, as long as we could push all queries that can be answered by a single source to the responsible DBMS. For distributed joins, we are back at the question of optimizing distributed queries but this is a familiar one and RDF is not the principal cost factor.</p> <p>The subject does become quite complex at this point. We would have to take supposedly representative synthetic OLTP and BI data sets (like the ones in TPC-D, TPC-E, and <a href="http://dbpedia.org/resource/TPC-H" id="link-id0xb576e78">TPC-H</a>), and invent queries across them that would both make sense and be implementable in SPARQL extended with aggregates and sub-queries. Reliance on SPARQL extensions is simply unavoidable. Setting up the test systems would be non-trivial, even though there is a lot of industry experience in these matters on the database side.</p> <p>So, while this is probably the benchmark most relevant to the target audience, we may have to start with a simpler one. I will next <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1269" id="link-id10fa7a50">outline something to the effect</a>.</p> </div>
Virtuoso Cluster Stage 1
http://www.openlinksw.com/blog/vdb/blog/?date=2007-09-06#1251
2007-09-06T11:11:52Z
2008-04-25T15:57:25-04:00
<div> <div style="display:none;">Virtuoso Cluster Stage 1</div> <p>I recall a quote from a stock car racing movie.</p> <p>"What is the necessary prerequisite for winning a race?" asked the racing team boss.</p> <p>"Being the fastest," answered the hotshot driver, after yet another wrecked engine.</p> <p>"No. It is finishing the race."</p> <p>In the interest of finishing, we'll now leave optimizing the cluster traffic and scheduling and move to completing functionality. Our next stop is TPC-D. After this <a href="http://dbpedia.org/resource/TPC-C" id="link-id0x1c3d9cb8">TPC-C</a>, which adds the requirement of handling distributed deadlocks. After this we add <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1be00c10">RDF</a>-specific optimizations.</p> <p>This will be <a href="http://virtuoso.openlinksw.com" id="link-id0x1f91c098">Virtuoso</a> 6 with the first stage of clustering support. This is with fixed partitions, which is just like a single database, except it runs on multiple machines. The stage after this is Virtuoso Cloud, the database with all the space filling properties of foam, expanding and contracting to keep an even <a href="http://dbpedia.org/resource/Data" id="link-id0x1e1aeff8">data</a> density as load and resource availability change.</p> <p>Right now, we have a pretty good idea of the final form of evaluating loop joins in a cluster, which after all is the main function of the thing. It makes sense to tune this to a point before going further. You want the pipes and pumps and turbines to have known properties and fittings before building a power plant.</p> <p>To test this, we took a table of a million short rows and made one copy partitioned over 4 databases and one copy with all rows in one database. We ran all the instances in a 4 core Xeon box. We used Unix sockets for communication.</p> <p>We joined the table to itself, like <code><b>SELECT COUNT (*) FROM ct a, ct b WHERE b.row_no = a.row_no + 3</b></code>. The <code><b>+ 3</b></code> causes the joined rows never to be on the same partition.</p> <p>With cluster, the single operation takes 3s and with a single process it takes 4s. The overall CPU time for cluster is about 30% higher, some of which is inevitable since it must combine results, serialize them, and so forth. Some real time is gained by doing multiple iterations of the inner loop (getting the row for b) in parallel. This can be further optimized to maybe 2x better with cluster but this can wait a little.</p> <p>Then we make a stream of 10 such queries. The stream with cluster is 14s; with the single process, it is 22s. Then we run 4 streams in parallel. The time with cluster is 39s and with a single process 36s. With 16 streams in parallel, cluster gets 2m51 and single process 3m21.</p> <p>The conclusion is that clustering overhead is not significant in a CPU-bound situation. Note that all the runs were at 4 cores at 98-100%, except for the first, single-client run, which had one process at 98% and 3 at 32%.</p> <p>The SMP single process loses by having more contention for mutexes serializing index access. Each wait carries an entirely ridiculous penalty of up to 6µs or so, <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1229" id="link-id106dca10">as discussed earlier on this blog</a>. The cluster wins by less contention due to distributed data and loses due to having to process messages and remember larger intermediate results. These balance out, or close enough.</p> <p>For the case with a single client, we can cut down on the coordination overhead by simply optimizing the code some more. This is quite possible, so we could get one process at 100% and 3 at 50%.</p> <p>The numbers are only relevant as ballpark figures and the percentages will vary between different queries. The point is to prove that we actually win and do not jump from the frying pan into the fire by splitting queries across processes. As a point of comparison, running the query clustered just as one would run it locally took 53s.</p> <p>We will later look at <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1336" id="link-id108d9868">the effects of different networks</a>, as we get to revisit the theme with some real benchmarks.</p> </div>
Virtuoso and cluster capacity allocation
http://www.openlinksw.com/blog/vdb/blog/?date=2007-08-28#1248
2007-08-28T11:54:30Z
2008-04-30T15:08:20-04:00
<div> <div style="display:none;">Virtuoso and cluster capacity allocation</div> <p>I just read <a href="http://labs.google.com/papers/bigtable.html" id="link-id140a68b0">Google's Bigtable</a> paper. It is relevant here because it talks about keeping petabyte scale (1024TB) tables on a variable size cluster of machines.</p> <p>I have talked about partitioning versus distributed cache in the <a href="http://www.openlinksw.com/weblog/oerling/?id=1229" id="link-id13f70dc8">second to last post</a>. The problem in short is that you do not expect a DBA to really know how to partition things, and even if the indices are correctly partitioned initially, repartitioning them is so bad that doing it online can be a problem. And repartitioning is needed whenever adding machines, unless the size increment is a doubling, which it will never be.</p> <p>So <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0xa2957e38">Oracle</a> has really elegantly stepped around the whole problem by not partitioning for clustering in the first place. So incremental capacity change does not require repartitioning. Oracle has partitioning for other purposes but this is not tied to their cluster proposition.</p> <p>I did not go the cache fusion route because I could not figure a way to know with near certainty where to send a request for a given key value. In the case we are interested in, the job simply must go to the <a href="http://dbpedia.org/resource/Data" id="link-id0xa20ce8e0">data</a> and not the other way around. Besides, not being totally dependent on a microsecond latency interconnect and a SAN for performance enhances deployment options. Sending large batches of functions tolerates latency better than cache consistency messages which are a page at a time, unless of course you kill yourself with extra trickery for batching these too.</p> <p>So how to adapt to capacity change? Well, by making the unit of capacity allocation much smaller than a machine, of course.</p> <p>Google has done this in Bigtable by a scheme of dynamic range partitioning. The partition size is in the tens to hundreds of megabytes, something that can be moved around within reason. When the partition, called a tablet, gets too big, it splits. Just like a Btree index. The tree top must be common <a href="http://dbpedia.org/resource/Knowledge" id="link-id0xa20cebb0">knowledge</a>, as well as the allocation of partitions to servers but these can be cached here and there and do not change all the time.</p> <p>So how could we do something of the sort here? I know for an experiential fact that when people cannot change the server memory pool size, let alone correctly set up disk striping, they simply cannot be expected to deal with partitioning. Besides, even if you know exactly what you are doing and why, configuring and refilling large numbers of partitions by hand is error prone, tedious, time consuming, and will run out of disk and require restoring backups and all sorts of DBA activity that will have everything down for a long time, unless of course you have MIS staff such as is not easily found.</p> <p>The solution is not so complex. We start with a set number of machines and make a file group on each. A file group has a bunch of disk stripes and a log file and can be laid out on the local file system in the usual manner. The data goes into the file group, partitioned as defined. You still specify partitioning columns but not where each partition goes. The system will decide this by itself. When a server's file group gets too big, it splits. One half of each key's partition in the original stays where it was and the other half goes to the copy. The copies will hold rows that no longer belong there but these can be removed in the background. The new file group will be managed by the same server process and the partitioning <a href="http://dbpedia.org/resource/Information" id="link-id0xc803148">information</a> on all servers gets updated to reflect the existence of the new file group and the range of hash values that belong there.</p> <p>If a file group is kept at some reasonable size, under a few GB, these can be moved around between servers, even dynamically. </p> <p>If data is kept replicated, then the replicas have to split at the same time and the system will have to make sure that the replicas are kept on separate machines.</p> <p>So what happens to disk locality when file groups split? Nothing much. Firstly, partitioning will be set up so that consecutive values go to the same hash value, so that key compression is not ruined. Thus, consecutive numbers will be on the same page. Imagine an integer key partitioned two ways on bits 10-20. Values 0-1K go together, values 1K-2K go another way, values 2K-3K go the first way etc. </p> <p>Now let us suppose the first partition, the even K's splits. It could split so that multiples of 4 go one way and the rest another way. Now we'd have 0-1K in place, 2K-3K in the new partition, 4K-5K in place and so on. A sequential disk read, with some read ahead, would scan the partitions in parallel but the disk access would be made sequential by the read ahead logic â remember that these are controlled by the same server process.</p> <p>For purposes of sending functions, the file group would be the recipient, not the host, per se. The allocation of file groups to hosts could change. </p> <p>Now picture a transaction that touches multiple file groups. The requests going to collocated file groups can travel in the same batch and the recipient server process can run them sequentially or with a thread per file group, as may be convenient. Multiple threads per query on the same index make contention and needless thread switches. But since distinct file groups have their distinct mutexes there is less interference.</p> <p>For purposes of transactions, we might view a file group as deserving a its own branch. In this way we would not have to abort transactions if file groups moved. A file group split would probably have to kill all uncommitted transactions on it so as not to have to split one branch in two or deal with uncommitted data in the split. This is hardly a problem, the event being rare. For purposes of checkpoints, logging, log archival, recovery, and such, a file group is its own unit. The Bigtable paper had some ideas about combining transaction logs and such, all quite straightforward and intuitive.</p> <p>Writing the clustering logic with the file group, not the database process, as the main unit of location is a good idea and an entirely trivial change. This will make it possible to adjust capacity in almost real time without bringing everything to a halt by re-inserting terabytes of data in system wide repartitioning runs.</p> <p>Implementing this on the current <a href="http://virtuoso.openlinksw.com" id="link-id0xa21de0e0">Virtuoso</a> is not a real difficulty. There is already a concept of file group, although we use only two, one for the data and one for temp. Using multiple ones is not a big deal.</p> <p>Supporting capacity allocation at the file group level instead of the server level can be introduced towards the middle of the clustering effort and will not greatly impact timetables.</p> </div>
Virtuoso Cluster Preview
http://www.openlinksw.com/blog/vdb/blog/?date=2007-08-27#1245
2007-08-27T09:51:45Z
2008-04-25T11:59:31-04:00
<div> <div style="display:none;">Virtuoso Cluster Preview</div> <p> <b><i>I wrote the basics of the <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1229" id="link-id1383c310">Virtuoso clustering support</a> over the past three weeks. It can now manage connections, decide where things go, do two phase commits, insert and select <a href="http://dbpedia.org/resource/Data" id="link-id0xad603e0">data</a> from tables partitioned over multiple <a href="http://virtuoso.openlinksw.com" id="link-id0xbc64f48">Virtuoso</a> instances. It works about enough to be measured, of which I will <a href="http://dbpedia.org/resource/Blog" id="link-id0xb958e90">blog</a> more over the next two weeks.</i> </b> </p> <p> <b><i>I will in the following give a features preview of what will be in the Virtuoso clustering support when it is released in the fall of this year (2007).</i> </b> </p> <h3>Data Partitioning</h3> <p>A Virtuoso database consists of indices only, so that the row of a table is stored together with the primary key. Blobs are stored on separate pages when they do not fit inline within the row. With clustering, partitioning can be specified index by index. Partitioning means that values of specific columns are used for determining where the containing index entry will be stored. Virtuoso partitions by hash and allows specifying what parts of partitioning columns are used for the hash, for example bits 14-6 of an integer or the first 5 characters of a string. Like this, key compression gains are not lost by storing consecutive values on different partitions.</p> <p>Once the partitioning is specified, we specify which set of cluster nodes stores this index. Not every index has to be split evenly across all nodes. Also, all nodes do not have to have equal slices of the partitioned index, accommodating differences in capacity between cluster nodes.</p> <p>Each Virtuoso instance can manage up to 32TB of data. A cluster has no definite size limit.</p> <h3>Load Balancing and Fault Tolerance</h3> <p>When data is partitioned, an operation on the data goes where the data is.  This provides a certain natural parallelism but we will discuss this further below.</p> <p>Some data may be stored multiple times in the cluster, either for fail-over or for splitting read load. Some data, such as database schema, is replicated on all nodes. When specifying a set of nodes for storing the partitions of a key, it is possible to specify multiple nodes for the same partition. If this is the case, updates go to all nodes and reads go to a randomly picked node from the group.</p> <p>If one of the nodes in the group fails, operation can resume with the surviving node.  The failed node can be brought back online from the transaction logs of the surviving nodes. A few transactions may be rolled back at the time of failure and again at the time of the failed node rejoining the cluster but these are aborts as in the case of deadlock and lose no committed data.</p> <h3>Shared Nothing</h3> <p>The Virtuoso architecture does not require a SAN for disk sharing across nodes. This is reasonable since a few disks on a local controller can easily provide 300MB/s of read and passing this over an interconnect fabric that would also have to carry inter-node messages could saturate even a fast network. </p> <h3>Client View</h3> <p>A <a href="http://dbpedia.org/resource/SQL" id="link-id0xbb32b58">SQL</a> or <a href="http://dbpedia.org/resource/Hypertext_Transfer_Protocol" id="link-id0xaa4e2b8">HTTP</a> client can connect to any node of the cluster and get an identical view of all data with full transactional semantics. DDL operations like table creation and package installation are limited to one node, though.</p> <p>Applications such as <a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id0x1bc18300">ODS</a> will run unmodified. They are installed on all nodes with a single install command. After this, the data partitioning must be declared, which is a one time operation to be done cluster by cluster. The only application change is specifying the partitioning columns for each index. The gain is optional redundant storage and capacity not limited to a single machine. The penalty is that single operations may take a little longer when not all data is managed by the same process but then the parallel throughput is increased.  We note that the main ODS performance factor is web page logic and not database access.  Thus splitting the web server logic over multiple nodes gives basically linear scaling.</p> <h3>Parallel Query Execution</h3> <p>Message latency is the principal performance factor in a clustered database. Due to this, Virtuoso packs the maximum number of operations in a single message. For example, when doing a loop join that reads one table sequentially and retrieves a row of another table for each row of the outer table, a large number of the join of the inner loop are run in parallel. So, if there is a join of five tables that gets one row from each table and all rows are on different nodes, the time will be spent on message latency. If each step of the join gets 10 rows, for a total of 100000 results, the message latency is not a significant factor and the cluster will clearly outperform a single node.</p> <p>Also, if the workload consists of large numbers of concurrent short updates or queries, the message latencies will even out and throughput will scale up even if doing a single transaction were faster on a single node.</p> <h3>Parallel SQL</h3> <p>There are SQL extensions for stored procedures allowing parallelizing operations.  For example, if a procedure has a loop doing inserts, the inserted rows can be buffered until a sufficient number is available, at which point they are sent in batches to the nodes concerned.  Transactional semantics are kept but error detection is deferred to the actual execution.</p> <h3>Transactions</h3> <p>Each transaction is owned by one node of the cluster, the node to which the client is connected. When more than one node besides the owner of the transaction is updated, two phase commit is used. This is transparent to the application code. No external transaction monitor is required, the Virtuoso instances perform these functions internally. There is a distributed deadlock detection scheme based on the nodes periodically sharing transaction waiting <a href="http://dbpedia.org/resource/Information" id="link-id0xb78b5c0">information</a>.</p> <p>Since read transactions can operate without locks, reading the last committed state of uncommitted updated rows, waiting for locks is not very common.</p> <h3>Interconnect and Threading</h3> <p>Virtuoso uses TCP to connect between instances. A single instance can have multiple listeners at different network interfaces for cluster activity. The interfaces will be used in a round-robin fashion by the peers, spreading the load over all network interfaces. A separate thread is created for monitoring each interface. Long messages, such as transfers of blobs are done on a separate thread, thus allowing normal service on the cluster node while the transfer is proceeding.</p> <p>We will have to test the performance of TCP over <i>Infiniband</i> to see if there is clear gain in going to a lower level interface like <i>MPI</i>. The Virtuoso architecture is based on streams connecting cluster nodes point to point. The design does not per se gain from remote DMA or other features provided by MPI. Typically, messages are quite short, under 100K.  Flow control for transfer of blobs is however nice to have but can be written at the application level if needed. We will get real data on the performance of different interconnects in the next weeks. </p> <h3>Deployment and Management</h3> <p>Configuring is quite simple, with each process sharing a copy of the same configuration file.  One line in the file differs from host to host, telling it which one it is. Otherwise the database configuration files are individual per host, accommodating different file system layouts etc.  Setting up a node requires copying the executable and two configuration files, no more.  All functionality is contained in a single process. There are no installers to be run or such.</p> <p>Changing the number or network interface of cluster nodes requires a cluster restart. Changing data partitioning requires copying the data into a new table and renaming this over the old one. This is time consuming and does not mix well with updates. Splitting an existing cluster node requires no copying with repartitioning but shifting data between partitions does.</p> <p>A consolidated status report shows the general state and level of intra-cluster traffic as count of messages and count of bytes.</p> <p>Start, shutdown, backup, and package installation commands can only be issued from a single master node. Otherwise all is symmetrical.</p> <h3>Present State and Next Developments</h3> <p>The basics are now in place. Some code remains to be written for such things as distributed deadlock detection, 2-phase commit recovery cycle, management functions, etc. Some SQL operations like text index, statistics sampling, and index intersection need special support, yet to be written.</p> <p>The <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xbca4e90">RDF</a> capabilities are not specifically affected by clustering except in a couple of places. Loading will be slightly revised to use larger batches of rows to minimize latency, for example.</p> <p>There is a pretty much infinite world of SQL optimizations for splitting aggregates, taking advantage of co-located joins etc. These will be added gradually. These are however not really central to the first application of RDF storage but are quite important for business intelligence, for example.</p> <p>We will run some benchmarks for comparing single host and clustered Virtuoso instances over the next weeks. Some of this will be with real data, giving an estimate on when we can move some of the RDF data we presently host to the new platform. We will benchmark against <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0xa9cc1b8">Oracle</a> and <a href="http://dbpedia.org/resource/IBM_DB2" id="link-id0x1be5abb0">DB2</a> later but first we get things to work and compare against ourselves.</p> <p>We roughly expect a halving in space consumption and a significant increase in single query performance and linearly scaling parallel throughput through addition of cluster nodes.</p> <p> <i>The <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1246" id="link-id106de430">next update</a> will be on this blog within two weeks.</i> </p> </div>
RDF, Clustered Databases, and Partitioning Vs. Cache Fusion
http://www.openlinksw.com/blog/vdb/blog/?date=2007-07-19#1230
2007-07-19T12:29:12Z
2008-04-25T11:59:26-04:00
<div> <div style="display:none;">RDF, Clustered Databases, and Partitioning Vs. Cache Fusion</div> <p>I recently read <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0xb279258">Oracle</a>'s papers about RAC, Real Application Clusters. This is relevant as we are presently working on the <a href="http://virtuoso.openlinksw.com" id="link-id0xba1d4e0">Virtuoso</a> equivalent.</p> <p>Caveat: The following is quite technical and not the final word on the matter.</p> <p>Oracle's claim is roughly as follows: Take a number of machines with access to a shared pool of disks and get scalability in processing power and memory without having to explicitly partition the <a href="http://dbpedia.org/resource/Data" id="link-id0x1a9a7ab8">data</a> or perform other complicated configuration.</p> <p>This works through implementing a cache consistency protocol between the participating boxes and by parallelizing queries just as one would do on a shared memory SMP box. Each disk page has a box assigned to keep track of it and the responsibility migrates so that the box most often needing the page gets to be the page's guardian, so as not to have to ask anybody else for permission to write the page.</p> <p>This is a compelling proposition. Surely, it must be unrealistic to expect people to manually partition databases. This would require some understanding of first principles which is scarce out there.</p> <p>So, should we implement clustering a la Oracle?</p> <p>Let's look at some basics. If we have an OLTP workload like <a href="http://dbpedia.org/resource/TPC-C" id="link-id0xbaa89d8">TPC-C</a>, we usually have affinity between clients and the data they access. This will make each client's pages migrate to be managed by the box the client is connected to. This will work pretty well, no worse than with a single box. If two clients are updating the same data but are connected to two different boxes, this is quite bad since the box that does not have responsibility for the page must ask the other box for write access. This is a round trip, at least tens of microseconds (µs). Consider in comparison that finding a row out of a million takes some 3µs.</p> <p>Would it not be better to have each partition in a known place and leave all processing to that place? The write contention would be resolved in the box owning the partition and there would be a message but now for requesting the update, not dealing with cache consistency. At what level should one communicate between cluster nodes? Talk about disk pages or about logical operations? If there is complete affinity between boxes and data, the RAC style shared cache needs no messages, each box ends up managing the pages of its clients and all works just as with a local situation. If on the other hand any client will update any page at random, most updates must request the write permission from another node. I will here presume that index tree tops get eventually cached on all nodes. If this were not so, even index lookups would have to most often request each index page from a remote box. Never forget that with a tree index, it takes about 1µs to descend one level and 50µs for a message round trip between processes, not counting any transport latency.</p> <p>What of <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xbe1bd98">RDF</a> query workloads? After all, we are in the first instance concerned with winning the RDF storage race and after this the TPC ones. We design for both but do RDF first since this is our chosen specialty.</p> <p>The disadvantage of having to specify partitioning is less weighty with RDF since there are only a few big tables and they will be at default settings, pretty much always. We do not expect the application developer to ever change these settings although it is in principle possible.</p> <p>What about queries? The RDF workload is mostly random access and loop joins. How would these run on RAC? For now, let's make a thought experiment and compare cache fusion to a hash partitioned cluster. In the following, I do not describe how Oracle actually works but will just describe how I would do things if I implemented a RAC style clustering. With a RAC style cluster, I'd split the outer loop into equal partitions and run them in parallel on different boxes. Each would build a working set for its part of the query and pages that were needed by more than one box would be read once from disk and a second time from the node that had them in cache. The top nodes of index trees would end up cached on all boxes. It would seem that all boxes would fill their cache with the same data. Now it may be that RAC makes it so that a page is cached only on one box and other boxes wanting the page must go to that box to get access to the page. But this would be a disaster in index lookups. It is less than a microsecond per local index tree level, but if there is a round trip, it would be at least 50µs per level of the index tree. I don't know for sure about Oracle but if I did RAC, I'd have to allow duplicate read content in caches. This would have the effect that the aggregate cache size would be closer to the single cache size than to the sum of the cache sizes. A physically partitioned database would not ship pages so caches would not overlap and the aggregate cache would indeed be the sum of the sizes. Now this is good only insofar all boxes participate but with evenly distributed data this is a good possibility.</p> <p>Of course, if RAC knew to split queries so that data and nodes had real affinity then the problem would be less. For indices one would need a map of key values to boxes. A little like a top level index shared among all nodes. The key value would give the node, just like with partitioning.</p> <p>This would make partitioning on the fly. Joins that were made the most frequently would cause migration that would make these joins co-located.</p> <p>We must optimize the number of messages needed to execute long series of loop joins. For parallelizing single queries, the most obvious approach would be to partition the first/outermost loop that is more than one iteration into equal size chunks. With RDF data, the join keys will mostly begin GS or GO with a possible P appended. If GO or GS specify the partition, partitioning by hash will yield the node that will provide the result.</p> <p>The number of messages can be reduced to a minimum of the number of join steps times number of boxes minus one if the loops are short enough and multiple operations are carried by one message.</p> <p>With RAC style clustering, each index lookup would have to be sent to the node most likely to hold the answer. If pages have to be fetched from other nodes, we have disastrous performance, at least 50µs for each non-local page. If there are two non-local pages in a lookup, the overhead will exceed the overhead of delegating the single lookup. Index page access in lookups cannot be easily batched, the way index lookups going to the same node can be batched. Batching multiple hopefully long operations into a single message is the only way to defeat the extreme cost of sending messages. An index lookup does not know what page it will need until it needs it. A way of batching these would be to run multiple lookups in parallel and to combine remote page requests grouping them by destination. This would not be impossible, simply we would have to run 100 index lookups in parallel on a thread, 100 first levels, 100 second levels and so forth. Suppose an outer loop that gives 100 rows and then an inner loop that retrieves 1 row for each. A query to get the email address of Mary would do this, supposing 100 Marys in the db. <code>{?person firstName "Mary" . ?person email ?m .}</code>. Suppose a cluster of 10 nodes. The first node gets the 100 rows of the outer loop, splits these into 10x10, 10 on each node and then each node does 10 lookups in parallel, meaning 10 first levels, 10 second levels, 10 third levels. The index tree would be 4 deep, branching 300 ways. Running the query a second time would find all data in memory and run with only 18 messages after getting the 100 rows of the first loop. The first run would send lots of messages, almost two per page, for about 800 messages after getting the 100 rows of the first loop.</p> <p>With partitioning, the situation would be sending 18 messages constant. 9 batches of 10 index lookups and their replies. The latency is 50µs and the lookup is 4µs. We would in fact gain in real time, counting 50µs for messages and 4µs per lookup, the time through the whole exercise of 10x10 random lookups would be 90µs.</p> <p>If I did RAC style clustering, I'd have to allow replicating the tops of index trees to all caches, and I'd have to batch page request messages from index lookups, effectively doing the lookup vector processing style, meaning 100 first levels, 100 second levels etc. Given a key beginning, I'd have to know what node to send this to, meaning pretty much doing the first levels of the lookup before deciding where to send the lookup, only to have the lookup redone by the box ending up with the lookup. Doing things this way would make Oracle RAC style clustering work with the use case.</p> <p>Given this, it appears that hash partitioning is easier to implement. Cache fusion clustering without the above mentioned gimmicks would be easiest of all but it would have a disastrous number of messages or it would fill all the caches with the same data. Avoiding this is possible but hard, as described above.</p> <p>We will have to experiment with Oracle RAC itself a bit farther down the road. Deciding to use partitioning instead of cache fusion does bring along conversion cost and a very high cost for repartitioning.</p> <p>Now let us look at the issue of co-location of joins. In a loop join this means that the node that holds the row from the outer loop also holds the row in the inner loop. For example, if order and order line are partitioned on order id, joining them on order id will be a co-located join. Such joins do not involve necessary messages in partitioned clusters. In RAC, they do not involve messages if the pages have migrated to be managed by the node doing the join, otherwise they do, up to 20 or so for the worst case.</p> <p>Do we get any benefit from co-location with RDF? Supposing joins that go from S to O to S (e.g., population of the city where Mary lives), we do not get much guarantee of co-location.</p> <p>Suppose the indices GSPO partitioned of GS and OGPS on OG, we know the box with Marys and then we'd know the box where the residence of each Mary was, based on GS. given the city as S, we would again know the box that had the population. All the three triples could be on different boxes. This cannot be helped at design time. At run time this can be helped by batching messages that go to the same node. Let's see how this fans out. 100 Marys from the first node. To get the city, we get 10 batches of 10 messages. We get the 100 cities and then we get their populations, again 10 batches of 10. In this scenario, the scheduling is centrally done by one thread. Suppose it were done by the 10 batches of 10 for getting the city of each Mary. For 10 cities, we'd get 10 lookups for population, each potentially to a different node. For this case, managing the execution by one thread instead of several makes bigger batches and less messages, as one would expect.</p> <p>It seems that with the RDF case, one may as well forget co-location. In the relational case, one must take advantage of it when there is co-location and when not, try to compensate with longer batches of function shipping.</p> <p>Excellent as some of the RAC claims are, it still seems that making it work well for an RDF workload would take such magical heuristics of location choice that implementing them would be hard and the result not altogether certain. I could get it to work eventually but hash partitioning seems by far the more predictable route. Also hash partitioning will work in shared nothing scenarios whereas RAC requires shared disks. Shared nothing will not require a SAN, which may make it somewhat lower cost. Also, if messages are grouped in large batches, the performance of the interconnect is not so critical, meaning that maybe even gigabit ethernet might do in cases. RAC style cache maintenance is more sensitive to interconnect latency than batched function shipping. Batched cache consistency is conceivable as discussed above but tough to do.</p> <p>For recovery and hot software updates, things can be arranged if there is non-local disk access or if partitions are mirrored. A RAC type cluster could use a SAN with internal mirroring. A hash partitioned system could mirror partitions to more than one box with local disk, thus using no mirrored disks. Repartitioning remains the bane of partitioning and not much can be done about that, it seems. The only easy repartitioning is doubling the cluster size. So it seems.</p> </div>
Storage News
http://www.openlinksw.com/blog/vdb/blog/?date=2007-07-12#1226
2007-07-12T14:29:26Z
2008-04-24T13:22:37-04:00
<div> <div style="display:none;">Storage News</div> <p>I have been away from the world for a few weeks, concentrating on technology.</p> <p>We have now implemented an entirely new storage layout. With <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1c53b838">RDF</a> <a href="http://dbpedia.org/resource/Data" id="link-id0x1cc7af28">data</a>, we have now successfully doubled the working set.</p> <p>This means that the number of triples that will fit in memory is doubled for any configuration. For any database in the hundreds of millions of triples, this is very significant. For LUBM data, we go from 75b to 35b per triple with the default indices.</p> <p>This is obtained without using gzip or some other stream compression. Thus no decompression is needed at read time. Random access speeds are within 5% of those of <a href="http://virtuoso.openlinksw.com" id="link-id0x1cf61820">Virtuoso</a> v5.0.1, but the space requirement is halved and you can still locate a random triple in cache in a few microseconds.</p> <p>What is better still, when using 8-byte IDs for IRIs instead of 4-byte ones, the space consumption stays almost the same since unique values are stored only once per page.</p> <p>When applying gzip to the new storage layout, we usually get 3x compression. This means that 99% of 8K pages fit in 3K after compression. This is no real surprise since an index is repetitive pretty much by definition, even if the repeated sections are now shorter than in v5.0.1.</p> <p>Gzip applied to pages does nothing for the working set since a page must remain random accessible for fast search but will cut disk usage to between half and a third. We will make this an option later. There are other tricks to be done with compression, like using a separate dictionary for non key text columns in relational applications. This would improve the working set in <a href="http://dbpedia.org/resource/TPC-C" id="link-id0x1c019500">TPC-C</a> and TPC-D quite a bit so we may do this also while on the subject.</p> <p>Right now we are writing the clustering support, revising all internal APIs to run with batches of rows instead of single rows. We will most likely release clustering and the new storage layout together, towards the end of summer, at least in internal deployments.</p> <p>I will <a href="http://dbpedia.org/resource/Blog" id="link-id0x1d383588">blog</a> about results as and when they are obtained, over the next few weeks.</p> </div>
Virtuoso Feature Update
http://www.openlinksw.com/blog/vdb/blog/?date=2007-05-23#1199
2007-05-23T14:09:37Z
2008-04-18T16:58:50-04:00
<div> <div style="display:none;">Virtuoso Feature Update</div> <p>We have a few new features that we did for the <a href="http://www2007.org/" id="link-id12603130">WWW 2007</a> conference that we will be shortly adding to the open source release.</p> <ul> <li>Optimization for <a href="http://dbpedia.org/resource/SQL" id="link-id0xe67fed8">SQL</a> <code>IN</code> predicate. The IN predicate with a list of values will now use an index if available. This is useful for <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x18791f70">SPARQL</a> queries with multiple <code>FROM</code> graphs, for example.</li> <li> <a href="http://docs.openlinksw.com/virtuoso/fn_key_estimate.html" id="link-idffb5400">API for index population estimates</a>. There is an API for getting an approximate count of matches given one or more leading key parts of an index.</li> <li> <a href="http://docs.openlinksw.com/virtuoso/coredbengine.html#RowbyRowAutoCommit" id="link-id1097c420">Row-level autocommit mode</a> – If one updates a huge table and the application does not require transaction isolation, it is possible to do this with an automatic commit after each row. This saves the server from having to keep rollback <a href="http://dbpedia.org/resource/Information" id="link-id0x18e3c340">information</a> on millions and billions of rows and saves it from temporary rollbacks of the uncommitted <a href="http://dbpedia.org/resource/Data" id="link-id0xde26fe8">data</a> for checkpoints etc. These things can completely hang a server if there are a few tens of millions of uncommitted inserts/deletes/updates.</li> <li>64-bit IDs for IRIs and <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1896f518">RDF</a> objects, 64-bit integer data type. With the growth of some RDF databases to the tens of billions of triples, we run out of the 32-bit range for IDs of distinct IRIs. To accommodate this before actually running out, we introduce a longer ID.</li> <li>Some cost model adjustments.</li> <li>SQL extension for producing multiple result set rows from a single table row. This is useful for mapping SPARQL queries like <code>SELECT * FROM graph WHERE {?s ?p ?o}</code> into a <code>UNION</code> of <code>SELECT *</code>’s from multiple tables of different width. Each term of the <code>UNION</code> will simply produce multiple 3 column result rows for each actual row while not having to run through the tables multiple times. Together with this, we have also fixed a number of things with the relational-to-RDF mapping. We have been testing this extensively with the <a href="http://blog.musicbrainz.org/" id="link-id12109cb0">Musicbrainz</a> mapping by <a href="http://fgiasson.com/blog/" id="link-idffd52f0">Fred Giasson</a>. </li> </ul> <p>These changes are small and to be released shortly.</p> <p>There are also some larger things in the works, to be released during this summer, <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1197" id="link-id10cafde8">the next post</a> gives an overview of these.</p> </div>
Virtuoso Cluster
http://www.openlinksw.com/blog/vdb/blog/?date=2007-05-23#1201
2007-05-23T14:09:37Z
2008-04-24T09:52:23.000003-04:00
<div> <div style="display:none;">Virtuoso Cluster</div> <p>We often get questions on clustering support, especially around <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1e53d008">RDF</a>, where databases quickly get rather large. So we will answer them here.</p> <p>But first on some support technology. We have an entire new disk allocation and IO system. It is basically operational but needs some further tuning. It offers much better locality and much better sequential access speeds.</p> <p>Specially for dealing with large RDF databases, we will introduce <a href="http://dbpedia.org/resource/Data" id="link-id0x1e042690">data</a> compression. We have over the years looked at different key compression possibilities but have never been very excited by them since thy complicate random access to index pages and make for longer execution paths, require scraping data for one logical thing from many places, and so on. Anyway, now we will compress pages before writing them to disk, so the cache is in machine byte order and alignment and disk is compressed. Since multiple processors are commonplace on servers, they can well be used for compression, that being such a nicely local operation, all in cache and requiring no serialization with other things.</p> <p>Of course, what was fixed length now becomes variable length, but if the compression ratio is fairly constant, we reserve space for the expected compressed size, and deal with the rare overflows separately. So no complicated shifting data around when something grows.</p> <p>Once we are done with this, this could well be a separate intermediate release.</p> <p>Now about clusters. We have for a long time had various plans for clusters but have not seen the immediate need for execution. With the rapid growth in the Linking Open Data movement and questions on web scale <a href="http://dbpedia.org/resource/Knowledge" id="link-id0x1e7714f0">knowledge</a> systems, it is time to get going.</p> <p>How will it work? <a href="http://virtuoso.openlinksw.com" id="link-id0x1e3caea8">Virtuoso</a> remains a generic DBMS, thus the clustering support is an across the board feature, not something for RDF only. So we can join <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x1ac67648">Oracle</a>, IBM <a href="http://dbpedia.org/resource/IBM_DB2" id="link-id0x1c2267d0">DB2</a>, and others at the multi-terabyte TPC races.</p> <p>We introduce hash partitioning at the index level and allow for redundancy, where multiple nodes can serve the same partition, allowing for load balancing read and replacement of failing nodes and growth of cluster without interruption of service.</p> <p>The <a href="http://dbpedia.org/resource/SQL" id="link-id0x1daea638">SQL</a> compiler, <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1ddb8c50">SPARQL</a>, and database engine all stay the same. There is a little change in the SQL run time, not so different from what we do with remote databases at present in the context of our <a href="http://dbpedia.org/resource/Virtual_Database" id="link-id0x1e13a880">virtual database</a> federation. There is a little extra complexity for distributed deadlock detection and sometimes multiple threads per transaction. We remember that one RPC round trip Is 3-4 index lookups, so we pipeline things so as to move requests in batches, a few dozen at a time.</p> <p>The cluster support will be in the same executable and will be enabled by configuration file settings. Administration is limited to one node, but Web and SQL clients can connect to any node and see the same data. There is no balancing between storage and control nodes because clients can simply be allocated round robin for statistically even usage. In relational applications, as exemplified by <a href="http://dbpedia.org/resource/TPC-C" id="link-id0x1c236bb0">TPC-C</a>, if one partitions by fields with an application meaning (such as warehouse ID), and if clients have an affinity to a particular chunk of data, they will of course preferentially connect to nodes hosting this data. With RDF, such affinity is unlikely, so nodes are basically interchangeable.</p> <p>In practice, we develop in June and July. Then we can rent a supercomputer maybe from Amazon EC2 and experiment away.</p> <p>We should just come up with a name for this. Maybe something astronomical, like star cluster. Big, bright but in this case not far away.</p> </div>
WWW 2007
http://www.openlinksw.com/blog/vdb/blog/?date=2007-05-23#1195
2007-05-23T13:31:38Z
2008-04-18T11:08:12.000003-04:00
<div> <div style="display:none;">WWW 2007</div> <p>We were at the <a href="http://www2007.org/" id="link-id10a0aa08">WWW 2007</a> conference in Banff, Canada week before last. <a href="http://virtuoso.openlinksw.com" id="link-id0x1963b400">Virtuoso</a> was a part of <a href="http://network.nature.com/profile/U1ACB1466" id="link-id1071a250">Alan Ruttenberg</a>’s <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0xd98ab38">semantic web</a> in health care and life sciences presentation. Alan had a database of 350M triples extracted from different biology and publication databases running on Virtuoso. We will also be experimenting on other biomedical datasets, both with real <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xe91aa88">RDF</a> and relational <a href="http://dbpedia.org/resource/Data" id="link-id0x181143b0">data</a> mapped to RDF on demand.</p> <p>Linking Open Data was a big thing at WWW 2007. There is quite a bit of momentum gathering around publishing publicly available data as RDF and making these data sets mutually joinable. <a href="http://www.bizer.de/" id="link-id10142a40">Chris Bizer</a> of the <a href="http://www.fu-berlin.de/en/" id="link-id109d4f40">Free University of Berlin</a> will be demonstrating <a href="http://dbpedia.org/resource/DBpedia" id="link-id0xe9ab408">Dbpedia</a> linked with a number of other data sets such as <a href="http://www.geonames.org/" id="link-id100c46d0">Geonames</a> and <a href="http://blog.musicbrainz.org/" id="link-id106e1980">Musicbrainz</a> and others at <a href="http://www.eswc2007.org/" id="link-id10f3c060">ESWC 2007</a> in a couple of weeks, also running on Virtuoso.</p> <p>The last month or so has been spent mostly on the conference preparation and follow up, not to mention taking part in two EU project proposals. But now we are returning to normal operations and can do some technology for a change. More on this in the next post.</p> </div>
Announcing Virtuoso Open-Source Edition v5.0.0
http://www.openlinksw.com/blog/vdb/blog/?date=2007-04-12#1184
2007-04-12T13:48:34Z
2007-04-12T09:50:10-04:00
All, OpenLink Software are pleased to announce a new release of <a href="http://virtuoso.openlinksw.com/wiki/main/">Virtuoso</a>, Open-Source Edition, version 5.0.0. This version includes: <ul> <li>Significant rewrite of database engine resulting in 50%-100% improvement on single CPU and in some cases up to 300% on multiprocessor CPUs by decreasing resource-contention between threads and other optimizations.</li> <li>Radical expansion of RDF support including</li> </ul> <ul> <li>In-built middleware (called the Sponger) for transforming non-RDF into RDF "on the fly" (e.g. producing Triples from Microformats, REST-style Web Services, and (X)HTML etc.)</li> <li>Full Text Indexing of Literal Objects in Triple Patterns (via Filter or magic bif:contains predicate applied to Literal Objects)</li> <li>Basic Inferencing (Subclass and Subproperty Support)</li> <li>SPARQL Aggregate Functions</li> <li>SPARQL Update Language Support (Updates, Inserts, Deletions in SPARQL)</li> <li>Improved Support of XML Schema Type System (including the use of XML Schema Complex Types as Objects of bif:xcontains predicate)</li> </ul> <ul> <li>Enhancements to the in-built SPARQL to SQL Compiler's Cost Optimizer</li> <li>Performance Optimizations to RDF VIEWs (SQL to RDF Mapping)</li> <li>Various bug-fixes</li> </ul> NOTE: Databases created with earlier versions of Virtuoso will be automatically upgraded to Virtuoso 5.0 but after upgrade will not be readable with older Virtuoso versions. For more information please see: Virtuoso Open Source Edition: Home Page: <a href="http://virtuoso.openlinksw.com/wiki/main/">http://virtuoso.openlinksw.com/wiki/main/</a> Download Page: <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VOSDownload">http://virtuoso.openlinksw.com/wiki/main/Main/VOSDownload</a> OpenLink Data Spaces: Home Page: <a href="http://virtuoso.openlinksw.com/wiki/main/Main/OdsIndex">http://virtuoso.openlinksw.com/wiki/main/Main/OdsIndex</a> SPARQL Usage Examples (re. SIOC, FOAF, AtomOWL, SKOS): <a href="http://virtuoso.openlinksw.com/wiki/main/Main/ODSSIOCRef">http://virtuoso.openlinksw.com/wiki/main/Main/ODSSIOCRef</a> Interactive SPARQL Demo: <a href="http://demo.openlinksw.com/isparql/">http://demo.openlinksw.com/isparql/</a> OpenLink AJAX Toolkit (OAT): Project Page: <a href="http://sourceforge.net/projects/oat">http://sourceforge.net/projects/oat</a> Live Demonstration: <a href="http://demo.openlinksw.com/DAV/JS/oat/index.html">http://demo.openlinksw.com/DAV/JS/oat/index.html</a> <p style="text-align:right;font-size:10px;">Technorati Tags: <a href="http://www.technorati.com/tag/database" rel="tag">database</a>, <a href="http://www.technorati.com/tag/databases" rel="tag">databases</a>, <a href="http://www.technorati.com/tag/open-source" rel="tag">open-source</a>, <a href="http://www.technorati.com/tag/OpenLink" rel="tag">OpenLink</a>, <a href="http://www.technorati.com/tag/RDBMS" rel="tag">RDBMS</a>, <a href="http://www.technorati.com/tag/RDF" rel="tag">RDF</a>, <a href="http://www.technorati.com/tag/semantic web" rel="tag">semantic web</a>, <a href="http://www.technorati.com/tag/Semantic Web" rel="tag">Semantic Web</a>, <a href="http://www.technorati.com/tag/SPARQL" rel="tag">SPARQL</a>, <a href="http://www.technorati.com/tag/virtuoso" rel="tag">virtuoso</a> </p>
Comparison of Open Source Databases with TPC D Queries
http://www.openlinksw.com/blog/vdb/blog/?date=2007-02-05#1132
2007-02-05T11:44:34Z
2008-04-17T21:04:40-04:00
<div> <div style="display:none;">Comparison of Open Source Databases with TPC D Queries</div> <p> <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1116" id="link-id10598cc0">Last time</a> we talked about database engine and transactions. Now we have come to the realm of query processing in our revisiting of the DBMS side of <a href="http://virtuoso.openlinksw.com" id="link-id0xd3ecd30">Virtuoso</a>.</p> <p>Now the well established, respectable standard benchmark for the basics of query processing is TPC D with its derivatives H and R. So we have, for testing how different <a href="http://dbpedia.org/resource/SQL" id="link-id0x15cf6938">SQL</a> optimizers manage the 22 queries, run a mini version of the D queries with a 1% scale database, some 30M of <a href="http://dbpedia.org/resource/Data" id="link-id0x15ce3928">data</a>, all in memory. This basically catches whether SQL implementations miss some of the expected tricks and how efficient in memory loop and hash joins and aggregation are.</p> <p>When we get to our next stop, high volume I/O, we will run the same with D databases in the 10G ballpark.</p> <p>The databases were tested on the same machine, with warm cache, taking the best run of 3. All had full statistics and were running with read committed isolation, where applicable. The data was generated using the procedures from the Virtuoso test suite. The Virtuoso version tested was 5.0, to be released shortly. The <a href="http://dbpedia.org/resource/MySQL" id="link-id0xc952f58">MySQL</a> was 5.0.27, the PostgreSQL was 8.1.6. </p> <table style="width: 334px; height: 556px; " border="1"> <tbody> <tr> <th rowspan="2">Query</th> <th colspan="4">Query Times in Milliseconds</th> </tr> <tr> <th> Virtuoso </th> <th> PostgreSQL </th> <th> MySQL </th> <th> MySQL with InnoDB </th> </tr> <tr> <td> Q1 </td> <td align="right"> <b>206</b> </td> <td align="right"> 763 </td> <td align="right"> 312 </td> <td align="right"> 198 </td> </tr> <tr> <td> Q2 </td> <td align="right"> 4 </td> <td align="right"> 6 </td> <td align="right"> <b>3</b> </td> <td align="right"> <b>3</b> </td> </tr> <tr> <td> Q3 </td> <td align="right"> <b>13</b> </td> <td align="right"> 51 </td> <td align="right"> 254 </td> <td align="right"> 64 </td> </tr> <tr> <td> Q4 </td> <td align="right"> <b>4</b> </td> <td align="right"> 16 </td> <td align="right"> 24 </td> <td align="right"> 60 </td> </tr> <tr> <td> Q5 </td> <td align="right"> <b>15</b> </td> <td align="right"> 22 </td> <td align="right"> 64 </td> <td align="right"> 68 </td> </tr> <tr> <td> Q6 </td> <td align="right"> <b>9</b> </td> <td align="right"> 70 </td> <td align="right"> 189 </td> <td align="right"> 65 </td> </tr> <tr> <td> Q7 </td> <td align="right"> <b>52</b> </td> <td align="right"> 143 </td> <td align="right"> 211 </td> <td align="right"> 84 </td> </tr> <tr> <td> Q8 </td> <td align="right"> 29 </td> <td align="right"> 31 </td> <td align="right"> 13 </td> <td align="right"> <b>11</b> </td> </tr> <tr> <td> Q9 </td> <td align="right"> <b>36</b> </td> <td align="right"> 114 </td> <td align="right"> 97 </td> <td align="right"> 61 </td> </tr> <tr> <td> Q10 </td> <td align="right"> <b>32</b> </td> <td align="right"> 51 </td> <td align="right"> 117 </td> <td align="right"> 57 </td> </tr> <tr> <td> Q11 </td> <td align="right"> 16 </td> <td align="right"> <b>9</b> </td> <td align="right"> 12 </td> <td align="right"> 10 </td> </tr> <tr> <td> Q12 </td> <td align="right"> <b>8</b> </td> <td align="right"> 21 </td> <td align="right"> 18 </td> <td align="right"> 130 </td> </tr> <tr> <td> Q13 </td> <td align="right"> <b>18</b> </td> <td align="right"> 74 </td> <td align="right"> - </td> <td align="right"> - </td> </tr> <tr> <td> Q14 </td> <td align="right"> <b>7</b> </td> <td align="right"> 21 </td> <td align="right"> 418 </td> <td align="right"> 1425 </td> </tr> <tr> <td> Q15 </td> <td align="right"> <b>14</b> </td> <td align="right"> 43 </td> <td align="right"> 389 </td> <td align="right"> 122 </td> </tr> <tr> <td> Q16 </td> <td align="right"> <b>16</b> </td> <td align="right"> 22 </td> <td align="right"> 18 </td> <td align="right"> 25 </td> </tr> <tr> <td> Q17 </td> <td align="right"> <b>1</b> </td> <td align="right"> 54 </td> <td align="right"> 26 </td> <td align="right"> 10 </td> </tr> <tr> <td> Q18 </td> <td align="right"> <b>82</b> </td> <td align="right"> 120 </td> <td align="right"> - </td> <td align="right"> - </td> </tr> <tr> <td> Q19 </td> <td align="right"> 19 </td> <td align="right"> 8 </td> <td align="right"> <b>2</b> </td> <td align="right"> 17 </td> </tr> <tr> <td> Q20 </td> <td align="right"> <b>7<b> </b></b></td> <td align="right"> 15 </td> <td align="right"> 66 </td> <td align="right"> 52 </td> </tr> <tr> <td> Q21 </td> <td align="right"> <b>34</b> </td> <td align="right"> 86 </td> <td align="right"> 524 </td> <td align="right"> 278 </td> </tr> <tr> <td> Q22 </td> <td align="right"> <b>4</b> </td> <td align="right"> 323 </td> <td align="right"> 3311</td> <td align="right"> 805 </td> </tr> <tr> <td>Total (msec)</td> <td align="right"><b>626</b></td> <td align="right">2063</td> <td align="right">6068</td> <td align="right">3545</td> </tr> </tbody> </table> <p>We lead by a fair margin but MySQL is hampered by obviously getting some execution plans wrong and not doing Q13 and Q18 at all, at least not under several tens of seconds; so we left these out of the table in the interest of having comparable totals.</p> <p>As usual, we also ran the workload on <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x167807c8">Oracle</a> 10g R2. Since Oracle does not like their numbers being published without explicit approval, we will just say that we are even with them within the parameters described above. Oracle has a more efficient decimal type so it wins where that is central, as on Q1. Also it seems to notice that the <code>GROUP BY</code>s of Q18 are produced in order of grouping columns, so it needs no intermediate table for storing the aggregates. If we addressed these matters, we'd lead by some 15% whereas now we are even. A faster decimal arithmetic implementation may be in the release after next.</p> <p>In the next posts, we will look at IO and disk allocation, and also return to <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xbb63c20">RDF</a> and LUBM.</p> </div>
Virtuoso 5.0 Preview
http://www.openlinksw.com/blog/vdb/blog/?date=2007-01-10#1117
2007-01-10T14:58:29Z
2008-04-17T21:04:37-04:00
<div> <div style="display:none;">Virtuoso 5.0 Preview</div> <p>As <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1108" id="link-id10c66e68">previously said</a>, we have a <a href="http://virtuoso.openlinksw.com" id="link-id0x15b8d948">Virtuoso</a> with brand new engine multithreading. It is now complete and passes its regular test suite. This is the basis for Virtuoso 5.0, to be available as the open source and commercial cuts as before.</p> <p>As one benchmark, we used the <a href="http://dbpedia.org/resource/TPC-C" id="link-id0x15fc4380">TPC-C</a> test driver that has always been bundled with Virtuoso. We ran 100000 new orders worth of the TPC-C transaction mix first with one client and then with 4 clients, each client going to its own warehouse, so there was not much lock contention. We did this on a 4 core Intel, the working set in RAM. With the old one, 1 client took 1m43 and 4 clients took 3m47. With the new one, one client took 1m30 and 4 clients took 2m37. So, 400000 new orders in 2m37, for 152820 new orders per minute as opposed to 105720 per minute previously. Do not confuse with the official tpmC metric, that one involves a whole bunch of further rules.</p> <p>TPC-C has activity spread over a few different tables. With tests dealing with fewer tables, improvements in parallelism are far greater.</p> <p>Aside from better parallelism, we have other features. One of them is a change in the read committed isolation, so that we now return the previous committed state for uncommitted changed rows instead of waiting for the updating transaction to terminate. This is similar to what <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0xe8cc528">Oracle</a> does for read committed. Also we now do log checkpoints without having to abort pending write transactions.</p> <p>When we have faster inserts, we actually see the <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xe0e5ff8">RDF</a> bulk loader run slower. This is really backwards. The reason is that while one thread parses, other threads insert and if the inserting threads are done they go to wait on a semaphore and this whole business of context switching absolutely kills performance. With slower inserts, the parser keeps ahead so there is less context switching, hence better overall throughput. I still do not get it how the OS can spend between 1.5 and 6 microseconds, several thousand instructions, deciding what to do next when there are only 3-4 eligible threads and all the rest is background which goes with a few dozen slices per second. Solaris is a little better than Linux at this but not dramatically so. Mac OS X is way worse.</p> <p>As said, we use Oracle 10G2 on the same platform (Linux FC5 64 bit) for sparring. It is really a very good piece of software. We have written the TPC C transactions in <a href="http://dbpedia.org/resource/SQL" id="link-id0x183aac30">SQL</a>/PL. What is surprising is that these procedures run amazingly slowly, even with a single client. Otherwise the Oracle engine is very fast. Well, as I recall, the official TPC C runs with Oracle use an OCI client and no stored procedures. Strange. While Virtuoso for example fills the initial TPC C state a little faster than Oracle, the procedures run 5-10 times slower with Oracle than with Virtuoso, all <a href="http://dbpedia.org/resource/Data" id="link-id0x1659fd20">data</a> in warm cache and a single client. While some parts of Oracle are really well optimized, all basic joins and aggregates etc, we are surprised at how they could have neglected such a central piece as the PL.</p> <p>Also, we have looked at transaction semantics. Serializable is mostly serializable with Oracle but does not always keep a steady count. Also it does not prevent inserts into a space that has been found empty by a serializable transaction. True, it will not show these inserts to the serializable transaction, so in this it follows the rules. Also, to make a read really repeatable, it seems that the read has to be FOR UPDATE. Otherwise one can not implement a reliable resource transaction, like changing the balance of an account.</p> <p>Anyway, the Virtuoso engine overhaul is now mostly complete. This is of course an open ended topic but the present batch is nearing completion. We have gone through as many as 3 implementations of hash joins, some things have yet to be finished there. Oracle has very good hash joins. The only way we could match that was to do it all in memory, dropping any persistent storage of the hash. This is of course OK if the hash is not very large and anyway hash joins go sour if the hash does not fit in working set.</p> <p>As next topics, we have more RDF and the LUBM benchmark to finish. Also we should revisit TPC-D.</p> <p>Databases are really quite complicated and extensive pieces of software. Much more so than the casual observer might think.</p> </div>
Season's Greetings from Virtuoso Development
http://www.openlinksw.com/blog/vdb/blog/?date=2007-01-09#1113
2007-01-09T07:05:07Z
2008-04-17T21:04:33.000001-04:00
<div> <div style="display:none;">Season's Greetings from Virtuoso Development</div> <p>It's been a long and very busy time since <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1085" id="link-id104d7ac0">the last blog post</a>.</p> <p>Now and then, circumstances call for a return to the contemplation of first principles. I have lately beheld the Platonic ideal of database-ness and translated it into engineering elegance. No quest is static and no objective is permanently achieved.</p> <p>Accordingly, I have redone all <a href="http://virtuoso.openlinksw.com" id="link-id0xe7f8d18">Virtuoso</a> core engine structures for control of parallel execution. As we now routinely get multiple cores per chip, this is more important than before. Aside from dramatic improvements in multiprocessor performance, there is also quite a bit of optimization for basic relational operations.</p> <p>Of course, this is not for the pure pleasure of geek-craft; it serves a very practical purpose. <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xde83868">RDF</a> opens a new database frontier, where these things make a significant difference. In application scenarios involving either federated/<a href="http://dbpedia.org/resource/Virtual_Database" id="link-id0x17791fa0">virtual database</a> or running typical web applications, the core concurrency of the DBMS is not really the determining factor. However, with RDF, we get a small number of very large tables and most processing goes to these tables. This is also often so with business intelligence but it is still more so with RDF. Thus the parallelism within a single index becomes essential.</p> <p>We have also made a point by point comparison of Virtuoso and <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x17a3aeb0">Oracle</a> 10g for basic relational operations. Oracle is very good, certainly in the basic relational operations like table scans and different kinds of joins. As a matter of principle, we will at the minimum match Oracle in all these things, in single and multiprocessor environments. The Virtuoso cut forthcoming in January will have all this inside. We are also considering making and publishing a basic <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x181c4438">RDBMS</a> performance checklist, aimed at comparing specific aspects of relational engine performance. While the TPC tests give a good aggregate figure, it is sometimes interesting to look at a finer level of detail. We may not be allowed to give out numbers in all cases due to license terms but we can certainly make the test available and publish numbers for those who do not object to this.</p> <p>Of course, RDF is the direct beneficiary of all these efforts, since RDF loading and querying basically rests on the performance of very relational things, such as diverse types of indices and joins.</p> <p> More <a href="http://dbpedia.org/resource/Information" id="link-id0x1aa37c80">information</a> will be forthcoming in January.</p> <p>Merry Christmas and productive new year to all.</p> </div>
Virtuoso TPCC and Multiprocessor Linux and Mac
http://www.openlinksw.com/blog/vdb/blog/?date=2007-01-09#1110
2007-01-09T06:35:06Z
2008-04-16T16:53:38.000001-04:00
<div> <div style="display:none;">Virtuoso TPCC and Multiprocessor Linux and Mac</div> <p>We have <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VOSScale" rel="sql" id="link-id10650ec0">updated our article</a> on <a href="http://virtuoso.openlinksw.com" id="link-id0xc41ba10">Virtuoso</a> scalability with two new platforms: A 2 x dual core Intel Xeon and a Mac Mini with an Intel Core Duo.</p> <p>We have more than quadrupled the best result so far.</p> <p>The best score so far is 83K transactions per minute with a 40 warehouse (about 4G) database. This is attributable to the process running in mostly memory, with 3 out of 4 cores busy on the database server. But even when doubling the database size and number of 3 clients, we stay at 49K transactions per minute, now with a little under 2 cores busy and am average of 20 disk reads pending at all times, split over 4 SATA disks. The measurement is the count of completed transactions during a 1h run. With the 80 warehouse database, it took about 18 minutes for the system to reach steady state, with a warm working set, hence the actual steady rate is somewhat higher than 49K, as the warm up period was included in the measurement.</p> <p>The metric on the Mac Mini was 2.7K with 2G RAM and one disk. The CPU usage was about one third of one core. Since we have had rates of over 10K with 2G RAM, we attribute the low result to running on a single disk which is not very fast at that.</p> <p>We have run tests in 64 and 32 bit modes but have found little difference as long as actual memory does not exceed 4g. If anything, 32 bit binaries should have an advantage in cache hit rate since most <a href="http://dbpedia.org/resource/Data" id="link-id0xdf23240">data</a> structures take less space there. After the process size exceeds the 32 bit limit, there is a notable difference in favor of 64 bit. Having more than 4G of database buffers produces a marked advantage over letting the OS use the space for file system cache. So, 64 bit is worthwhile but only if there is enough memory. As for X86 having more registers in 64 bit mode, we have not specifically measured what effect that might have.</p> <p>We also note that Linux has improved a great deal with respect to multiprocessor configurations. We use a very simple test with a number of threads acquiring and then immediately freeing the same mutex. On single CPU systems, the real time has pretty much increased linearly with the number of threads. On multiprocessor systems, we used to get very non-linear behavior, with 2 threads competing for the same mutex taking tens of times the real time as opposed to one thread. At last measurement, with a 64 bit FC 5, we saw 2 threads take 7x the real time when competing for the same mutex. This is in the same ballpark as Solaris 10 on a similar system. Mac OS X 10.4 Tiger on a 2x dual core Xeon Mac Pro did the worst so far, with two threads taking over 70x the time of one. With a Mac Mini with a single Core Duo, the factor between one thread and two was 73.</p> <p>Also the proportion of system CPU on Tiger was consistently higher than on Solaris or Linux when running the same benchmarks. Of course for most applications this test is not significant but it is relevant for database servers, as there are many very short critical sections involved in multithreaded processing of indices and the like.</p> </div>
Recent Virtuoso Developments
http://www.openlinksw.com/blog/vdb/blog/?date=2007-01-09#1109
2007-01-09T06:35:05Z
2008-04-16T16:53:37.000013-04:00
<div> <div style="display:none;">Recent Virtuoso Developments</div> <p>We have been extensively working on <a href="http://dbpedia.org/resource/Virtual_Database" id="link-id0x19aebcf8">virtual database</a> refinements. There are many <a href="http://dbpedia.org/resource/SQL" id="link-id0xa217cd28">SQL</a> cost model adjustments to better model distributed queries and we now support direct access to <a href="http://dbpedia.org/resource/Oracle_Database" id="link-id0x1751b990">Oracle</a> and <a href="http://dbpedia.org/resource/IBM_Informix" id="link-id0x17393938">Informix</a> statistics system tables. Thus, when you attach a table from one or the other, you automatically getup to date statistics. This helps <a href="http://virtuoso.openlinksw.com" id="link-id0x19fb24f0">Virtuoso</a> optimize distributed queries. Also the documentation is updated as concerns these, with a new section on distributed query optimization.</p> <p>On the applications side, we have been keeping up with the SIOC <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xd13efd8">RDF</a> ontology developments. All <a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id0x170b1630">ODS</a> applications now make their <a href="http://dbpedia.org/resource/Data" id="link-id0xdc517d0">data</a> available as SIOC graphs for download and <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x19fec088">SPARQL</a> query access.</p> <p>What is most exciting however is our advance in mapping relational data into RDF. We now have a mapping language that makes arbitrary legacy data in Virtuoso or elsewhere in the relational world RDF query-able. We will put out a white paper on this in a few days.</p> <p>Also we have some innovations in mind for optimizing the physical storage of RDF triples. We keep experimenting, now with our sights set to the high end of triple storage, towards billion triple data sets. We are experimenting with a new more space efficient index structure for better working set behavior. Next week will yield the first results.</p> </div>
More RDF scalability tests
http://www.openlinksw.com/blog/vdb/blog/?date=2006-11-01#1075
2006-11-01T20:36:17Z
2008-04-16T16:53:39.000004-04:00
<div> <div style="display:none;">More RDF scalability tests</div> <p>We have lately been busy with <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1aeda730">RDF</a> scalability. We work with the 8000 university LUBM <a href="http://dbpedia.org/resource/Data" id="link-id0x185e6e00">data</a> set, a little over a billion triples. We can load it in 23h 46m on a box with 8G RAM. With 16G we probably could get it in 16h.</p> <p>The resulting database is 75G, 74 bytes per triple which is not bad. It will shrink a little more if explicitly compacted by merging adjacent partly filled pages. See <a href="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSBitmapIndexing" id="link-id105e5cf8">Advances in Virtuoso RDF Triple Storage</a> for an in-depth treatment of the subject.</p> <p>The real question of RDF scalability is finding a way of having more than one CPU on the same index tree without them hitting the prohibitive penalty of waiting for a mutex. The sure solution is partitioning, would probably have to be by range of the whole key. but before we go to so much trouble, we'll look at dropping a couple of critical sections from index random access. Also some kernel parameters may be adjustable, like a spin count before calling the scheduler when trying to get an occupied mutex. Still we should not waste too much time on platform specifics. We'll see.</p> <p>We just updated the <a href="http://virtuoso.openlinksw.com" id="link-id0x17b07998">Virtuoso</a> Open Source cut. The latest RDF refinements are not in, so maybe the cut will have to be refreshed shortly.</p> <p>We are also now applying the relational to RDF mapping discussed in <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VOSSQLRDF" id="link-id10677bb8">Declarative SQL Schema to RDF Ontology Mapping</a> to the <a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id0x1732fa20">ODS</a> applications.</p> <p>There is a form of the mapping in the VOS cut on the net but it is not quite ready yet. We must first finish testing it through mapping all the relational schemas of the ODS apps before we can really recommend it. This is another reason for a VOS update in the near future.</p> <p>We will be looking at the query side of LUBM after the ISWC 2006 conference. So far, we find queries compile OK for many SIOC use cases with the cost model that there is now. A more systematic review of the cost model for <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xc56b868">SPARQL</a> will come when we get to the queries.</p> <p>We put some ideas about inferencing in the Advances in Triple Storage paper. The question is whether we should forward chain such things as class subsumption and subproperties. If we build these into the <a href="http://dbpedia.org/resource/SQL" id="link-id0xa25c7a70">SQL</a> engine used for running SPARQL, we probably can do these as unions at run time with good performance and better working set due to not storing trivial entailed triples. Some more thought and experimentation needs to go into this.</p> </div>
Virtuoso and ODS Update
http://www.openlinksw.com/blog/vdb/blog/?date=2006-08-10#1025
2006-08-10T11:55:26Z
2008-04-16T16:53:34.000008-04:00
<div> <div style="display:none;">Virtuoso and ODS Update</div> <p>We have released an update of <a href="http://virtuoso.openlinksw.com" id="link-id0x1b0d5100">Virtuoso</a> Open Source Edition and the <a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id0x1770ad30">OpenLink Data Spaces</a> suite.</p> <p>This marks the coming of age of our <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x1a1c6800">RDF</a> and <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1779b790">SPARQL</a> efforts. We have the new <a href="http://dbpedia.org/resource/SQL" id="link-id0x170db778">SQL</a> cost model with SPARQL awareness, we have applications which present much of their <a href="http://dbpedia.org/resource/Data" id="link-id0x18ab4600">data</a> as SIOC, FOAF, ATOM OWL and other formats.</p> <p>We continue refining these technologies. Our next roadmap item is mapping relational data into RDF and offering SPARQL access to relational data without data duplication. Expect a white paper about this soon.</p> </div>
New RDF Store White Paper
http://www.openlinksw.com/blog/vdb/blog/?date=2006-07-31#1022
2006-07-31T15:49:49Z
2008-04-16T16:53:31.000012-04:00
<div> <div style="display:none;">New RDF Store White Paper</div> <p>There is a new paper <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VOSRDFWP" id="link-id104e1698">Implementing an RDF Triple Store using an ORDBMS</a> at the <a href="http://virtuoso.openlinksw.com" id="link-id0xc358098">Virtuoso</a> wiki.</p> <p>This paper summarizes how we have extended Virtuoso's <a href="http://dbpedia.org/resource/SQL" id="link-id0x1ab48290">SQL</a> and database engine to better accommodate storing <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xccf6878">RDF</a> triples and optimizing queries of RDF <a href="http://dbpedia.org/resource/Data" id="link-id0x19f1f538">data</a>. This is the first of a series. The next will concern mapping relational databases onto RDF ontologies for <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x19fd9cf8">SPARQL</a> access.</p> <p>This paper concerns the next Virtuoso Open Source release, to be available for download a few days from this posting.</p> </div>
More Thoughts on ORDBMS Clients, .NET and RDF
http://www.openlinksw.com/blog/vdb/blog/?date=2006-07-17#1008
2006-07-17T12:16:02Z
2008-04-16T16:13:30.000001-04:00
<div> <div style="display:none;">More Thoughts on ORDBMS Clients, .NET and RDF</div> <p>Continuing on from <a href="http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/1002" id="link-id1064f0c8">the previous post</a>... If Microsoft opens the right interfaces for independent developers, we see many exciting possibilities for using <a href="http://msdn2.microsoft.com/en-us/data/aa937699.aspx" id="link-id10f3ab60">ADO.NET</a> 3.0 with <a href="http://virtuoso.openlinksw.com" id="link-id0x171ad660">Virtuoso</a>.</p> <p>Microsoft quite explicitly states that their thrust is to decouple the client side representation of <a href="http://dbpedia.org/resource/Data" id="link-id0xdaf01b0">data</a> as .NET objects from the relational schema on the database. This is a worthy goal.</p> <p>But we can also see other possible applications of the technology when we move away from strictly relational back ends. This can go in two directions: Towards object oriented database (OODBMS) and towards making applications for the <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x175fa2f0">semantic web</a>.</p> <p>In the OODBMS direction, we could equate Virtuoso table hierarchies with .NET classes and create a tighter coupling between client and database, going as it were in the other direction from Microsoft's intended decoupling. For example, we could do typical OODBMS tricks such as pre-fetch of objects based on storage clustering. The simplest case of this is like virtual memory, where the request for one byte brings in the whole page or group of pages. The basic idea is that what is created together probably gets used together and if all objects are modeled as subclasses of (sub-tables) of a common superclass, then, regardless of instance type, what is created together (has consecutive IDs) will indeed tend to cluster on the same page. These tricks can deliver good results in very navigational applications like GIS or CAD. But these are rather specialized things and we do not see OODBMS making any great comeback.</p> <p>But what is more interesting and more topical in the present times is making clients for the <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xc58f9f8">RDF</a> world. There, the OWL ontology could be used to make the .NET classes and the DBMS could, when returning URIs serving as subjects of triple include specified predicates on these subjects, enough to allow instantiating .NET instances as "proxies" of these RDF objects. Of course, only predicates for which the client has a representation are relevant, thus some client-server handshake is needed at the start. What data could be pre-fetched is like the intersection of a concise bounded description and what the client has classes for. The rest of the mapping would be very simple, with IRIs becoming pointers, multi-valued predicates lists, and so on. IRIs for which the RDF type is not known or inferable could be left out or represented as a special class with name-value pairs for its attributes, same with blank nodes.</p> <p>In this way, .NET's considerable UI capabilities could directly be exploited for visualizing RDF data, only given that the data complies reasonably well with a known ontology.</p> <p>If a <a href="http://dbpedia.org/resource/SPARQL" id="link-id0xc5d8728">SPARQL</a> query returned a result-set, IRI type columns would be returned as .NET instances and the server would pre-fetch enough data for filling them in. For a CONSTRUCT, a collection object could be returned with the objects materialized inside. If the interfaces allow passing an <a href="http://dbpedia.org/resource/Entity" id="link-id0x19a434e8">Entity</a> <a href="http://dbpedia.org/resource/SQL" id="link-id0x1a146d30">SQL</a> string, these could possibly be specialized to allow for a SPARQL string instead. LINQ might have to be extended to allow for SPARQL type queries, though.</p> <p>Many of these questions will be better answerable as we get more details on Microsoft's forthcoming <a href="http://dbpedia.org/resource/ADO.NET" id="link-id0x985bc50">ADO</a> .NET release. We hope that sufficient latitude exists for exploring all these interesting avenues of development.</p> </div>
Object Relational Rediscovered?
http://www.openlinksw.com/blog/vdb/blog/?date=2006-07-13#1003
2006-07-13T12:33:32Z
2008-04-16T16:13:26-04:00
<div> <div style="display:none;">Object Relational Rediscovered?</div> <p>I have recently read some of Microsoft's <a href="http://dbpedia.org/resource/ADO.NET" id="link-id0x173cea20">ADO</a> .NET 3 papers. I am reminded of the distant past when I designed Kubl, which later became OpenLink <a href="http://virtuoso.openlinksw.com" id="link-id0x18bdfe68">Virtuoso</a>. So I will reminisce and speculate a little.</p> <p>So now is the time when polymorphic queries and mixing relational style joins and object style navigation become politically acceptable and even recommended and there finally is a workable solution to having a foreign key in the database and a pointer or set of pointers in the client application. Not to mention change tracking so as to be able to update in-memory <a href="http://dbpedia.org/resource/Data" id="link-id0xd6f0ae0">data</a> structures and commit a delta against the database without explicit update statements.</p> <p>All these questions existed already in the mid 90s and earlier. Since I was coming from OO and LISP into the database world, I even felt these questions to be important. The solution in the earliest Kubl was to have inheritance between tables, what became the <a href="http://dbpedia.org/resource/SQL" id="link-id0xddcdac0">SQL</a> 2K <code>UNDER</code> clause, and a virtual column called <code>_ROW</code> that would select a serialization of the primary key entry. Then there was the function <code>row_key()</code>, which when applied to a <code>_ROW</code> virtual column would return a database-wide unique identifier of the row, containing the key info and the key part values plus which subtable of the table was at hand. Then there was a function for dereferencing a <code>row_key</code> for getting the <code>_ROW</code>. And one could store <code>row_keys</code> into columns and dereference these in queries. Within SQL, one could use the <code>row_column</code> function to extract individual column values from a <code>row_key</code> or <code>_ROW</code>.</p> <p>This was all fine server side. But we also had a client for Franz Inc.'s Allegro Common Lisp that talked to Kubl's <a href="http://dbpedia.org/resource/Open_Database_Connectivity" id="link-id0xde2c348">ODBC</a> listener. This client had the basic statements and prepared statements and result sets, parameters and array parameters, a little like <a href="http://dbpedia.org/resource/Java_Database_Connectivity" id="link-id0x156409f8">JDBC</a> does now. But the extra was that we could do a mapping between a Lisp struct or object and a database key, so the <code>_ROW</code> would automatically materialize into the Lisp struct or class instance. And the mapping between these materializations and the <code>row_keys</code> identifying them in the database were kept in a thread environment called object space. Updates could be relational-style <code>UPDATEs</code> or consist of putting a <code>_ROW</code> serialization in database format back into the Kubl store with a single SQL function.</p> <p>This was different from just storing object serializations into LOB columns, as is often done, insofar as the object classes and data members were really database tables and columns, thus native to the DBMS, not just opaque data to be processed client-side only.</p> <p>So it was then possible to program a little like is shown in the ADO .NET 3 demos today, some ten years later.</p> <p>Some of these functions still exist in Virtuoso, albeit in a deprecated state, and there is no client that can use these to any advantage. Indeed, we dropped this line of work when Kubl became Virtuoso, mostly because there was no standard and no client applications that would use such features. Instead, we concentrated on virtual <a href="http://dbpedia.org/resource/Relational_database_management_system" id="link-id0x175a7b10">RDBMS</a>, transparently accessing any third party data via ODBC.</p> <p>Now however, as objects, both native SQL and Java and .NET, have become mainstream citizens of relational databases in general, Virtuoso and otherwise, and as Microsoft has legitimized accessing whole objects and not only scalar columns in result sets as part of ADO .NET 3, these things might be worth a second look.</p> </div>
Virtuoso/PL
http://www.openlinksw.com/blog/vdb/blog/?date=2006-07-13#1001
2006-07-13T11:23:26Z
2008-04-16T16:13:24-04:00
<div> <div style="display:none;">RDF Bulk Load and <a href="http://docs.openlinksw.com/virtuoso/PLREF.html" id="link-id0x16395e78">Virtuoso/PL</a> Parallelism</div> <p>We have been playing with the Wikipedia3 <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x16341340">RDF</a> <a href="http://dbpedia.org/resource/Data" id="link-id0xcc5b1d0">data</a> set, 48 million triples or so. We have for a long time foreseen the need for a special bulk loader for RDF but this brought this into immediate relevance.</p> <p>So I wrote a generic parallel extension to <a href="http://docs.openlinksw.com/virtuoso/PLREF.html" id="link-id105ca910">Virtuoso/PL</a> and <a href="http://dbpedia.org/resource/SQL" id="link-id0x161633c8">SQL</a>. This consists of a function for creating a queue that will feed async requests to be served on a thread pool of configurable size. Each of the worker threads has its own transaction and the owner of the thread pool can look at or block for return states of individual request . This is a generic means for delegating work to async threads from <a href="http://virtuoso.openlinksw.com" id="link-id0xdc6b680">Virtuoso</a>/PL. Of course this can also be used at a lower level for parallelizing single SQL queries, for example aggregation of a large table or creating an index on a large table. Many applications, such as the <a href="http://dbpedia.org/resource/OpenLink_Data_Spaces" id="link-id0x191da620">ODS</a> <a href="http://virtuoso.openlinksw.com/wiki/main/Main/OdsFeedManager" id="link-id106793c0">Feed Manager</a> will also benefit, since this makes it more convenient to schedule parallel downloads from news sources and the like. This extension will make its way into the release after next.</p> <p>But back to RDF. We presently have the primary key of the triple store as GSPO and a second index as PGOS. Using this mechanism, we will experiment with different multithreaded loading configurations. One thread translates from the IRI text representation to the IRI IDs, one thread may insert into the GSPO index, which is typically local and a few threads will share the inserting into the PGOS key. The latter key is inserted in random order, whereas the former is inserted mainly in ascending order when loading new data. In this way, we should be able to keep full load on several CPUs and even more disks.</p> <p>It turns out that the new async queue plus thread pool construct is very handy for any pipeline or symmetric parallelization. When this is well tested, I will update the documents and maybe do a technical article about this.</p> <p>Transactionality is not an issue in the bulk load situation. The graph being loaded will anyway be incomplete until it is loaded, other graphs will not be affected and no significant amount of locks will be held at any time by the bulk loader threads.</p> <p>Also later, when looking at within-query and other parallelization, we have many interesting possibilities. For example, we may measure the CPU and IO load and adjust the size of the shareable thread pool accordingly. All SQL or web requests get their thread just as they now do, and extra threads may be made available for opportunistic parallelization up until we have full CPU and IO utilization. Still, this will not lead to long queries preempting short ones, since all get at least one thread. I may post some results of parallel RDF loading later on this <a href="http://dbpedia.org/resource/Blog" id="link-id0x1b8b6ca8">blog</a>.</p> </div>
Virtuoso SQL, SPARQL and Dynamic Statistics
http://www.openlinksw.com/blog/vdb/blog/?date=2006-07-11#999
2006-07-11T19:39:29Z
2008-04-16T16:13:22-04:00
<div> <div style="display:none;">Virtuoso SQL, SPARQL and Dynamic Statistics</div> <p>The last couple of weeks have been very busy, dealing with updates to the <a href="http://virtuoso.openlinksw.com" id="link-id0xd616b98">Virtuoso</a> <a href="http://dbpedia.org/resource/SQL" id="link-id0x1a3033d0">SQL</a> and <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1a12d9f8">SPARQL</a> compiler cost model. </p> <p>The new SQL compiler takes samples of index population on demand, thus always works with up-to-date statistics. Further, when there are constant leading key parts, it can get an estimate of the selectivity of the constant criteria with a single lookup. </p> <p>This is especially important for processing <a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0xd547cb0">RDF</a>. Since all triples go to one table unless otherwise declared, normal SQL statistics are not very useful for determining the join order for a SPARQL query. However, nearly always, SPARQL queries have a constant graph, constant predicate, sometimes constant subjects and objects. For example, using the index P, G, O, S, the compiler can know how many triples will have a given predicate within a given graph. This is done with a single lookup, without needing to count the actual triples, which would defeat the purpose. Also there is no need to do periodic statistics collection runs or to maintain counts of distinct combinations for multiple key parts. This makes for virtual certainty of getting reasonable join orders even for recently inserted or fast changing <a href="http://dbpedia.org/resource/Data" id="link-id0x163b39d8">data</a> sets. </p> <p>This will be part of the next Virtuoso Open Source release, probably in the next couple of weeks. There will also be a technical article with examples of how the dynamic statistics feature helps with RDF queries. </p> </div>
Virtuoso Open Source Edition now for Windows
http://www.openlinksw.com/blog/vdb/blog/?date=2006-04-27#964
2006-04-27T08:56:00Z
2008-04-16T16:13:37.000002-04:00
<div> <div style="display:none;">Virtuoso Open Source Edition now for Windows</div> <p> <a href="http://virtuoso.openlinksw.com/wiki/main/Main" id="link-id10726730">Virtuoso Open Source Edition</a> has been updated to version 4.5.2. We have added a binary distribution for Windows and the source distribution comes with Visual Studio project files for building on Win32. The new source distribution is available from <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VOSUsageWindows" id="link-id106bbbc8">the Virtuoso wiki</a>. Make sure that you get the distribution with version 4.5.2. Win64 support will be in the next update. </p> <p>This release also enhances the <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1be84360">SPARQL</a> support with better inlining of SPARQL in <a href="http://dbpedia.org/resource/SQL" id="link-id0x1770e2f0">SQL</a> and other features and fixes.</p> </div>
Virtuoso and Database Scalability
http://www.openlinksw.com/blog/vdb/blog/?date=2006-04-24#962
2006-04-24T16:06:00Z
2008-04-16T16:13:34.000004-04:00
<div> <div style="display:none;">Virtuoso and Database Scalability</div> <p>We have a new <a href="http://virtuoso.openlinksw.com/wiki/main/Main/VOSScale" id="link-id1068c3f8">technical article</a>, benchmarking <a href="http://virtuoso.openlinksw.com" id="link-id0xd413030">Virtuoso</a> on different hardware configurations.</p> <p>This is useful reading for anyone interested in using Virtuoso as a database back end for online applications or simply anyone interested in relational database scalability, no matter what specific DBMS.</p> <p>We use an adaptation of the well known <a href="http://dbpedia.org/resource/TPC-C" id="link-id0x1aed8170">TPC-C</a> benchmark to see what hardware configuration will give the best price/performance. We also explain how to tune Virtuoso and how and why different parameters affect the throughput.</p> </div>
Introducing Virtuoso Open Source Edition
http://www.openlinksw.com/blog/vdb/blog/?date=2006-04-11#950
2006-04-11T16:33:07Z
2008-04-16T16:13:32-04:00
<div> <div style="display:none;">Introducing Virtuoso Open Source Edition</div> <p>I am Orri Erling, program manager for <a href="http://virtuoso.openlinksw.com" id="link-id0xd7d9bc0">Virtuoso</a> at <a href="http://www.openlinksw.com/dataspace/organization/openlink#this" id="link-id0xd9951b0">OpenLink Software</a>. This <a href="http://dbpedia.org/resource/Blog" id="link-id0x1775bac0">blog</a> is about any and all aspects of technology that have to do with Virtuoso.</p> <p>The launch of <a href="http://virtuoso.openlinksw.com/wiki/main/Main" id="link-id10b0c208">Virtuoso Open Source Edition (VOS)</a> marks a new period in our participation in the database world. We will henceforth be much more active, publish much more material, have a faster release cycle and actively reach out to the various areas of the open source community.</p> <p>We have years worth of demos, white papers, articles, a suite of Virtuoso based applications, and much more that we will be unveiling over the following months.</p> <p>We will track different aspects of Virtuoso work on this and related blogs. In the middle term, we will talk about the following:</p> <ul> <li> <b><a href="http://dbpedia.org/resource/Resource_Description_Framework" id="link-id0x171d1158">RDF</a>, <a href="http://dbpedia.org/resource/SPARQL" id="link-id0x1a5dbd50">SPARQL</a> and <a href="http://dbpedia.org/resource/Semantic_Web" id="link-id0x17106870">semantic web</a> work</b> - The initial VOS release has SPARQL support and this will continue to be refined and optimized. We will introduce SPARQL benchmark suites and the like as these become ready.</li> <li> <b>Relational database</b> - Virtuoso's extensible <a href="http://dbpedia.org/resource/SQL" id="link-id0xd3a91b0">SQL</a> and relational storage engine is the platform on which all the rest stands. Thus this continues to be improved, ranging from low level database engine work to SQL optimizations to various developer convenience features. A database-only configuration of Virtuoso is another possibility.</li> <li> <b>DAV and web services</b> - Web services are the main entry point for all Virtuoso's features. These may eventually become more significant than the traditional SQL client interfaces, of which Virtuoso supports several.</li> </ul> <p>There is a whole suite of next generation file server features to be unveiled. These include items such as automatic metadata extraction and logical views on content based on its metadata, permissions etc.</p> <p>In the immediate future, we will:</p> <ul> <li>Keep enhancing the VOS wiki and edit the existing base of unpublished material to be ready for publication on this platform.</li> <li>Keep adding to technical notes and FAQ's on compiling and running on different platforms and using the different run time hosting options of Virtuoso.</li> </ul> <p>The VOS development CVS will be updated at high frequency, in some areas even weekly. Stable snapshots will be made available 3 or 4 times a year.</p> <p>We will have a very exciting spring, with radically more participation in the database and open source worlds than ever. Look for frequent updates on this blog.</p> </div>