http://www.openlinksw.com:443/weblog/oerling/?id=1822Scalability, specifically linear scalability, means that twice the data takes twice as long to process, or that double the gear processes the same data in half the time. This is only literally true for "embarrassingly parallel" workloads. There are parts of TPC-H which have an embarrassingly parallel nature, like Q1 and Q7. There are parts that are almost as easy, like Q14, Q17, Q19, and Q21, where there is a big scan and a selective hash join with a hash table small enough to replicate everywhere. The scan scales linearly; building the hash does not, since it is done at single-server speed (once in each process). Some queries like Q9 and Q13 end up doing a big cross-partition join which runs into communication overheads. This is our first look at how performance behaves with bigger data and a larger platform. The results shown here are interesting but are not final. I bet I can do better; by how much is what we'll find out soon enough. We will here compare a 1000G setup on my desktop, and a 3000G setup at the CWI's Scilens cluster. The former is 2 boxes of dual Xeon E5 2630, and the latter is 8 boxes of dual Xeon E5 2650v2. All things run from memory and both have QDR IB interconnect. Counting cores and clock, the CWI cluster is 6x larger. As a rough approximation, for the worst queries, 6x the gear runs 3x the data in the same amount of real time. The 1000G setup has near full platform utilization and the 3000G setup has about half platform utilization. In both cases, running two instances of the same query at the same time takes twice as long. We use Q9 for this study. The plan makes a hash table of part with 1/14 of all parts, replicating to all processes. Then there is a hash table of partsupp with a key of ps_partkey, ps_suppkey, and a dependent of ps_supplycost. This is much larger than the part hash table and is therefore partitioned on ps_partkey. The build is for 1/14th of partsupp. Then there is a scan of lineitem filtered by the part hash table; then a cross-partition join to the partsupp hash table; then a cross partition join to orders, this time by index; then a hash join on a replicated hash table of supplier; then nation; then aggregation. The aggregation is done in each slice; then the slices are added up at the end. The plan could be made better by one fewer partition crossing. Now there is a crossing from l_orderkey to l_partkey and back to o_orderkey. This would not be so if the cost model knew that the partsupp always hits. The cost model thinks it hits 1/14 of the time, because it does not know that the selection on the build is exactly the same as on the probe. For the present purposes, the extra crossing just serves to make the matter of interest more visible. So, for the 1000G setup, we have 43.6 seconds (s) and Cluster 4 nodes, 44 s. 459 m/s 119788 KB/s 3120% cpu 0% read 19% clw threads 1r 0w 0i buffers 17622126 68 d 0 w 0 pfs For the 3000G setup, we have 49.9 s and Cluster 16 nodes, 50 s. 49389 m/s 1801815 KB/s 7283% cpu 0% read 18% clw threads 1r 0w 0i buffers 135122893 15895255 d 0 w 17 pfs The platform utilization on the small system is better, at 31/48 (running/total threads); the large one has 73/256. The large case is clearly network bound. If this were for CPU only, it should be done in half the time it takes the small system to do 1000G. We confirm this by looking at write wait: 3940 seconds of thread time blocked on write over 50s of real time. The figures on the small one are 3.9s of thread time blocked for 39s of real time. The data transfer on the large one is 93 GB. How to block less? One idea would be to write less. So we try compression; there is a Google snappy-based message compression option in Virtuoso. We now get 39.6 s and Cluster 16 nodes, 40 s. 65161 m/s 1239922 KB/s 10201% cpu 0% read 21% clw threads 1r 0w 0i buffers 52828440 172 d 0 w 0 pfs The write block time is 397 s of thread time over 39 s of real time, 10x better. The data transfer is 50.9 GB after compression. Snappy is somewhat effective for compression and very fast; in CPU profile, it is under 3% of Q9 on the small system. Gains on the small system are less, though, since blocking is not a big issue to start with. This is still not full platform. But if the data transfer is further cut in half by a better plan, the situation will be quite good. Now we have 102/256 threads running, meaning that there could be another 40-50% of throughput to be added. The last 128 threads are second threads of a core, so count for roughly 30% of a real core. The main cluster-specific operation is a send from one to many. This is now done by formulating the message to each recipient in a chain of string buffers; then, after all the messages are prepared, these are optionally compressed and sent to their recipient. This is needlessly simple: Compressing can proceed if ever there is a would-block situation on writing. If all the compression is done, then a blocked write should switch to another recipient, and only after all recipients have a would-block situation, then the thread can call-select with all descriptors and block on them collectively. There is a piece of code to this effect, but is not now being used. It has been seen to add no value in small cases, but could be useful here. The IB fabric has been seen to do 1.8 GB/s bidirectionally on multiple independent point-to-point TCP links. This is about half the nominal 4 GB/s (40 Gbit/s with 10/8 encoding). So the aggregate throughputs that we see here are nowhere near the nominal spec of the network. Lower level interfaces and the occasional busy wait on the reading end could be tried to some advantage. We have not tried 10GbE either; but if that works at nominal speed, then 10GbE should also be good enough. We will try this at Amazon in due time. In the meantime, there is a 3000G test made at the CWI cluster without message compression. The score is about 4x that of the single server at 300G using the same hardware. The run is with approximately half platform utilization. There are three runs of power plus throughput, the first run being cold. Run Power Throughput Composite Run 1 305,881.5 1,072,411.9 572,739.8 Run 2 1,292,085.1 1,179,391.6 1,234,453.1 Run 3 1,178,534.1 1,092,936.2 1,134,928.4 The numerical quantities summaries follow. One problem of the run is a high peak of query memory consumption leading to slowdown. Some parts should probably be done in multiple passes to keep the peak lower and not run into swapping. The details will have to be sorted out. This is a demonstration of capability; the perfected accomplishment is to follow. 3000G Run 1 Virt-H Executive Summary Report Date September 29, 2014 Database Scale Factor 3000 Query Streams for Throughput Test 8 Virt-H Power 305,881.5 Virt-H Throughput 1,072,411.9 Virt-H Composite Query-per-Hour Metric (Qph@100GB) 572,739.8 Measurement Interval in Throughput Test (Ts) 1,772.554000 seconds Duration of stream execution Start Date/Time End Date/Time Duration Stream 0 09/29/2014 12:54:52 09/29/2014 13:31:17 0:36:25 Stream 1 09/29/2014 13:31:24 09/29/2014 13:59:24 0:28:00 Stream 2 09/29/2014 13:31:24 09/29/2014 13:58:59 0:27:35 Stream 3 09/29/2014 13:31:24 09/29/2014 13:58:29 0:27:05 Stream 4 09/29/2014 13:31:24 09/29/2014 13:58:52 0:27:28 Stream 5 09/29/2014 13:31:24 09/29/2014 14:00:06 0:28:42 Stream 6 09/29/2014 13:31:24 09/29/2014 13:58:18 0:26:54 Stream 7 09/29/2014 13:31:24 09/29/2014 13:59:25 0:28:01 Stream 8 09/29/2014 13:31:24 09/29/2014 13:58:50 0:27:26 Refresh 0 09/29/2014 12:54:52 09/29/2014 12:56:59 0:02:07 09/29/2014 13:31:17 09/29/2014 13:31:23 0:00:06 Refresh 1 09/29/2014 14:00:38 09/29/2014 14:01:11 0:00:33 Refresh 2 09/29/2014 13:31:25 09/29/2014 13:36:57 0:05:32 Refresh 3 09/29/2014 13:36:56 09/29/2014 13:47:02 0:10:06 Refresh 4 09/29/2014 13:47:03 09/29/2014 13:51:40 0:04:37 Refresh 5 09/29/2014 13:51:42 09/29/2014 13:56:40 0:04:58 Refresh 6 09/29/2014 13:56:40 09/29/2014 13:59:25 0:02:45 Refresh 7 09/29/2014 13:59:25 09/29/2014 14:00:10 0:00:45 Refresh 8 09/29/2014 14:00:11 09/29/2014 14:00:37 0:00:26 Numerical Quantities Summary Timing Intervals in Seconds: Query Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Stream 0 601.576975 90.803782 108.725110 177.112667 171.995572 2.098138 15.768311 152.511444 Stream 1 13.310341 32.722946 125.551415 1.912836 46.041675 13.294214 85.345068 165.424288 Stream 2 19.425885 9.248670 150.855556 7.085737 88.445566 10.490432 49.318554 322.500839 Stream 3 30.534391 14.273478 100.987791 59.341763 46.442443 9.613795 64.186196 146.324186 Stream 4 28.211213 37.134522 64.189335 10.931513 100.610673 9.929866 112.270530 108.489951 Stream 5 29.226411 18.132589 95.245160 63.100068 115.663908 6.151231 46.251309 127.742471 Stream 6 30.750930 20.888658 108.894177 55.168565 82.016828 69.451493 65.161517 103.697733 Stream 7 13.462570 18.033847 32.065492 78.910373 202.998301 10.688279 47.167022 139.601948 Stream 8 24.354314 16.711503 112.008551 8.307098 126.849630 7.127605 51.083118 98.648077 Min Qi 13.310341 9.248670 32.065492 1.912836 46.041675 6.151231 46.251309 98.648077 Max Qi 30.750930 37.134522 150.855556 78.910373 202.998301 69.451493 112.270530 322.500839 Avg Qi 23.659507 20.893277 98.724685 35.594744 101.133628 17.093364 65.097914 151.553687 Query Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Stream 0 92.991259 5.175922 42.238393 29.239879 367.805534 3.604910 15.557396 11.650267 Stream 1 149.502128 30.197806 50.786184 217.190836 283.545905 11.653171 73.321150 116.860455 Stream 2 245.783668 22.278841 50.578731 36.301810 181.405269 32.236754 57.631764 61.540533 Stream 3 377.782738 24.129319 84.097657 10.959661 171.698669 8.973519 54.532180 45.527142 Stream 4 341.148908 74.358770 85.782399 43.116347 151.146233 22.870727 74.439693 51.871535 Stream 5 72.259919 11.424035 79.310504 9.833135 562.871920 14.961209 127.861874 55.377721 Stream 6 373.301225 41.379753 81.983260 9.373200 95.039317 19.071346 76.159452 48.324504 Stream 7 449.871952 16.099152 48.047940 8.559784 211.094730 10.569071 26.710228 72.571454 Stream 8 395.771006 33.537585 54.850876 141.526389 153.763316 12.997092 127.961975 57.100346 Min Qi 72.259919 11.424035 48.047940 8.559784 95.039317 8.973519 26.710228 45.527142 Max Qi 449.871952 74.358770 85.782399 217.190836 562.871920 32.236754 127.961975 116.860455 Avg Qi 300.677693 31.675658 66.929694 59.607645 226.320670 16.666611 77.327289 63.646711 Query Q17 Q18 Q19 Q20 Q21 Q22 RF1 RF2 Stream 0 12.230334 70.991261 33.092797 17.517230 15.798438 19.743562 127.494687 5.893471 Stream 1 27.550293 14.970857 16.442806 111.138612 68.214095 7.884782 27.109441 6.087067 Stream 2 43.277918 12.748690 22.681844 92.835566 84.416610 14.661934 151.094498 153.285076 Stream 3 129.696125 13.435663 14.674499 129.179966 39.176513 6.286296 181.596838 416.052710 Stream 4 110.348816 7.080225 21.051910 85.758973 65.130356 7.292999 123.386514 151.000786 Stream 5 43.365006 9.847612 32.881770 94.752284 67.788314 9.035439 72.539334 223.967821 Stream 6 34.534280 36.347298 27.849276 122.736244 51.447492 25.051058 80.452175 84.519426 Stream 7 48.021860 30.594474 22.522426 99.245893 73.076698 7.260729 38.585852 5.697277 Stream 8 29.484201 12.368769 40.344043 84.137820 30.813313 4.856991 22.196547 4.600057 Min Qi 27.550293 7.080225 14.674499 84.137820 30.813313 4.856991 22.196547 4.600057 Max Qi 129.696125 36.347298 40.344043 129.179966 84.416610 25.051058 181.596838 416.052710 Avg Qi 58.284812 17.174198 24.806072 102.473170 60.007924 10.291279 87.120150 130.651277 3000G Run 2 Virt-H Executive Summary Report Date September 29, 2014 Database Scale Factor 3000 Query Streams for Throughput Test 8 Virt-H Power 1292085.1 Virt-H Throughput 1179391.6 Virt-H Composite Query-per-Hour Metric (Qph@100GB) 1234453.1 Measurement Interval in Throughput Test (Ts) 1611.779000 seconds Duration of stream execution Start Date/Time End Date/Time Duration Stream 0 09/29/2014 14:01:15 09/29/2014 14:06:48 0:05:33 Stream 1 09/29/2014 14:06:53 09/29/2014 14:30:22 0:23:29 Stream 2 09/29/2014 14:06:53 09/29/2014 14:32:30 0:25:37 Stream 3 09/29/2014 14:06:53 09/29/2014 14:31:23 0:24:30 Stream 4 09/29/2014 14:06:53 09/29/2014 14:31:34 0:24:41 Stream 5 09/29/2014 14:06:53 09/29/2014 14:32:53 0:26:00 Stream 6 09/29/2014 14:06:53 09/29/2014 14:29:51 0:22:58 Stream 7 09/29/2014 14:06:53 09/29/2014 14:31:34 0:24:41 Stream 8 09/29/2014 14:06:53 09/29/2014 14:30:35 0:23:42 Refresh 0 09/29/2014 14:01:15 09/29/2014 14:01:35 0:00:20 09/29/2014 14:06:49 09/29/2014 14:06:53 0:00:04 Refresh 1 09/29/2014 14:33:16 09/29/2014 14:33:45 0:00:29 Refresh 2 09/29/2014 14:06:55 09/29/2014 14:12:28 0:05:33 Refresh 3 09/29/2014 14:12:29 09/29/2014 14:21:55 0:09:26 Refresh 4 09/29/2014 14:21:55 09/29/2014 14:27:40 0:05:45 Refresh 5 09/29/2014 14:27:43 09/29/2014 14:31:14 0:03:31 Refresh 6 09/29/2014 14:31:14 09/29/2014 14:31:51 0:00:37 Refresh 7 09/29/2014 14:31:51 09/29/2014 14:32:52 0:01:01 Refresh 8 09/29/2014 14:32:52 09/29/2014 14:33:16 0:00:24 Numerical Quantities Summary Timing Intervals in Seconds: Query Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Stream 0 9.451169 3.644118 18.419151 1.404395 15.740525 2.085038 15.171847 25.400834 Stream 1 19.558041 6.607300 85.774410 4.503525 81.448472 11.976129 92.140470 145.743853 Stream 2 31.042019 7.877299 71.958033 8.862111 142.452144 18.489193 81.003310 85.856529 Stream 3 38.833612 12.440326 86.063103 7.165120 84.707025 16.931531 100.442710 122.411252 Stream 4 15.751913 33.026762 50.457193 7.064220 114.130257 5.992556 66.035959 84.596973 Stream 5 18.462884 28.047942 110.690543 16.566547 104.403789 5.303453 72.552640 402.383383 Stream 6 17.858339 33.988800 110.431091 7.238431 72.229953 16.850955 68.231546 180.601000 Stream 7 23.055572 17.044813 96.105520 8.941132 171.130879 8.423100 70.634541 147.261648 Stream 8 19.840798 13.860740 74.961175 16.171566 56.165875 5.904921 47.646217 125.991819 Min Qi 15.751913 6.607300 50.457193 4.503525 56.165875 5.303453 47.646217 84.596973 Max Qi 38.833612 33.988800 110.690543 16.566547 171.130879 18.489193 100.442710 402.383383 Avg Qi 23.050397 19.111748 85.805134 9.564082 103.333549 11.233980 74.835924 161.855807 Query Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Stream 0 54.766945 5.551163 29.216632 3.035008 52.816902 3.346243 15.767022 10.066112 Stream 1 130.666380 9.658277 49.332720 103.036705 194.520370 12.166344 65.144599 97.158571 Stream 2 254.754936 22.605298 38.102466 21.121168 300.467330 12.262318 108.203491 50.696657 Stream 3 283.761567 19.327164 73.414574 7.431651 183.121904 12.573854 73.814766 46.802493 Stream 4 290.341947 57.452026 58.354221 13.066162 189.263163 18.998781 121.269774 54.831406 Stream 5 81.787025 8.410538 79.822552 16.005077 190.730342 21.697136 100.456487 46.744884 Stream 6 202.558515 39.360009 74.519981 15.960756 137.321631 26.583824 57.537668 60.758997 Stream 7 226.790801 44.175536 73.992368 7.561897 182.853851 17.597471 31.128055 44.389893 Stream 8 275.423934 21.980040 60.538239 39.736622 173.574795 58.786316 95.124912 25.564108 Min Qi 81.787025 8.410538 38.102466 7.431651 137.321631 12.166344 31.128055 25.564108 Max Qi 290.341947 57.452026 79.822552 103.036705 300.467330 58.786316 121.269774 97.158571 Avg Qi 218.260638 27.871111 63.509640 27.990005 193.981673 22.583255 81.584969 53.368376 Query Q17 Q18 Q19 Q20 Q21 Q22 RF1 RF2 Stream 0 13.620157 2.288504 4.166807 16.468447 9.991810 1.101775 20.152227 4.294680 Stream 1 44.026143 31.720525 25.684461 134.254716 30.797008 9.568594 24.328205 4.319533 Stream 2 40.283148 9.970277 29.731019 133.083785 29.322194 8.859556 73.251098 249.850045 Stream 3 44.288244 18.914661 38.162762 144.458624 22.556235 6.184842 117.267234 445.700238 Stream 4 67.147744 6.649451 27.876825 59.226248 69.373248 44.478703 61.381724 282.608075 Stream 5 36.403227 12.226129 21.997683 95.912670 44.219799 21.117974 106.473817 97.896971 Stream 6 42.114038 30.805969 25.929027 51.658733 26.475662 34.816500 31.309953 5.608395 Stream 7 48.601889 18.708127 18.893532 132.558026 50.476383 12.309402 22.661371 37.610815 Stream 8 34.413417 34.709883 37.058335 121.710608 44.676485 9.449332 19.311945 4.420232 Min Qi 34.413417 6.649451 18.893532 51.658733 22.556235 6.184842 19.311945 4.319533 Max Qi 67.147744 34.709883 38.162762 144.458624 69.373248 44.478703 117.267234 445.700238 Avg Qi 44.659731 20.463128 28.166705 109.107926 39.737127 18.348113 56.998168 141.001788 3000G Run 3 Virt-H Executive Summary Report Date September 29, 2014 Database Scale Factor 3000 Query Streams for Throughput Test 8 Virt-H Power 1178534.1 Virt-H Throughput 1092936.2 Virt-H Composite Query-per-Hour Metric (Qph@100GB) 1134928.4 Measurement Interval in Throughput Test (Ts) 1739.269000 seconds Duration of stream execution Start Date/Time End Date/Time Duration Stream 0 09/29/2014 14:33:48 09/29/2014 14:40:59 0:07:11 Stream 1 09/29/2014 14:41:04 09/29/2014 15:10:02 0:28:58 Stream 2 09/29/2014 14:41:04 09/29/2014 15:09:07 0:28:03 Stream 3 09/29/2014 14:41:04 09/29/2014 15:09:17 0:28:13 Stream 4 09/29/2014 14:41:04 09/29/2014 15:09:55 0:28:51 Stream 5 09/29/2014 14:41:04 09/29/2014 15:09:39 0:28:35 Stream 6 09/29/2014 14:41:04 09/29/2014 15:09:46 0:28:42 Stream 7 09/29/2014 14:41:04 09/29/2014 15:09:58 0:28:54 Stream 8 09/29/2014 14:41:04 09/29/2014 15:08:58 0:27:54 Refresh 0 09/29/2014 14:33:48 09/29/2014 14:34:07 0:00:19 09/29/2014 14:40:59 09/29/2014 14:41:04 0:00:05 Refresh 1 09/29/2014 15:06:57 09/29/2014 15:09:49 0:02:52 Refresh 2 09/29/2014 14:41:05 09/29/2014 14:47:39 0:06:34 Refresh 3 09/29/2014 14:47:40 09/29/2014 14:56:46 0:09:06 Refresh 4 09/29/2014 14:56:49 09/29/2014 15:03:19 0:06:30 Refresh 5 09/29/2014 15:03:24 09/29/2014 15:06:45 0:03:21 Refresh 6 09/29/2014 15:06:46 09/29/2014 15:06:49 0:00:03 Refresh 7 09/29/2014 15:06:50 09/29/2014 15:06:53 0:00:03 Refresh 8 09/29/2014 15:06:53 09/29/2014 15:10:04 0:03:11 Numerical Quantities Summary Timing Intervals in Seconds: Query Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Stream 0 9.393632 5.001910 17.053567 1.427500 17.813839 2.230451 13.884490 25.610995 Stream 1 12.971454 9.383520 94.257760 1.603106 127.940946 20.791892 78.869819 138.521273 Stream 2 21.428177 31.431513 96.366083 5.611843 58.394596 11.279502 47.114473 407.135077 Stream 3 23.377920 37.474814 83.640621 9.152178 71.186158 11.001543 46.763758 110.015662 Stream 4 49.580860 31.979940 87.662950 8.983661 68.052295 14.367631 59.266063 301.788652 Stream 5 13.483836 20.203772 391.980128 12.505446 77.966993 10.487869 52.989448 226.837637 Stream 6 38.104903 21.271630 84.689348 8.626460 86.620802 11.981171 69.182098 111.810485 Stream 7 20.243617 12.298692 99.547203 6.020951 151.584400 17.528287 62.037348 101.023802 Stream 8 22.808294 17.583072 59.180595 5.618565 123.108771 11.477376 42.485363 92.035709 Min Qi 12.971454 9.383520 59.180595 1.603106 58.394596 10.487869 42.485363 92.035709 Max Qi 49.580860 37.474814 391.980128 12.505446 151.584400 20.791892 78.869819 407.135077 Avg Qi 25.249883 22.703369 124.665586 7.265276 95.606870 13.614409 57.338546 186.146037 Query Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Stream 0 146.487681 6.798942 29.834475 3.177879 55.067866 4.503738 17.215591 9.333281 Stream 1 177.581204 44.178095 69.746005 12.306166 215.602727 30.443709 64.276384 45.266949 Stream 2 211.311651 27.403143 61.412478 12.173058 216.879170 18.272234 96.753886 35.587072 Stream 3 482.581456 68.663026 60.354163 13.408513 187.921639 17.469237 62.337222 31.706120 Stream 4 178.297373 23.711312 67.129677 15.216904 328.149575 20.258853 78.891201 84.852368 Stream 5 209.496498 28.346366 55.584081 9.644075 131.622351 24.171156 80.046801 43.625932 Stream 6 521.691639 24.126176 72.964805 15.311409 146.152570 34.748843 71.957130 58.470644 Stream 7 580.320149 17.054563 56.172396 7.530832 200.100326 12.444021 25.910599 75.653693 Stream 8 472.231674 15.064398 89.875570 42.394675 166.589234 12.831209 81.697881 73.821769 Min Qi 177.581204 15.064398 55.584081 7.530832 131.622351 12.444021 25.910599 31.706120 Max Qi 580.320149 68.663026 89.875570 42.394675 328.149575 34.748843 96.753886 84.852368 Avg Qi 354.188955 31.068385 66.654897 15.998204 199.127199 21.329908 70.233888 56.123068 Query Q17 Q18 Q19 Q20 Q21 Q22 RF1 RF2 Stream 0 12.252670 2.593733 4.115862 16.895672 10.183350 1.240096 18.679685 4.876067 Stream 1 356.740980 21.197870 30.422216 81.779038 65.468650 3.947503 63.933750 107.563796 Stream 2 54.087768 10.152604 34.940701 113.510640 70.908809 12.316233 109.091578 283.076004 Stream 3 52.807104 18.525982 13.740089 212.364908 16.413964 17.998809 58.653503 483.718271 Stream 4 42.389062 36.157809 28.909260 86.427025 21.605419 7.608729 54.910853 331.074114 Stream 5 48.214794 15.778893 20.681799 130.560005 43.846752 33.905533 54.536966 139.563667 Stream 6 84.061840 26.224851 16.546432 117.265210 34.766856 39.037423 0.710642 1.645351 Stream 7 63.034890 15.966686 31.666488 112.689765 28.661943 12.828171 1.274731 1.780452 Stream 8 43.879104 8.596666 32.585746 177.928730 26.763334 6.112333 1.187693 0.533668 Min Qi 42.389062 8.596666 13.740089 81.779038 16.413964 3.947503 0.710642 0.533668 Max Qi 356.740980 36.157809 34.940701 212.364908 70.908809 39.037423 109.091578 483.718271 Avg Qi 93.151943 19.075170 26.186591 129.065665 38.554466 16.719342 43.037465 168.619415 To be continued... In Hoc Signo Vinces (TPC-H) Series In Hoc Signo Vinces (part 1): Virtuoso meets TPC-H In Hoc Signo Vinces (part 2): TPC-H Schema Choices In Hoc Signo Vinces (part 3): Benchmark Configuration Settings In Hoc Signo Vinces (part 4): Bulk Load and Refresh In Hoc Signo Vinces (part 5): The Return of SQL Federation In Hoc Signo Vinces (part 6): TPC-H Q1 and Q3: An Introduction to Query Plans In Hoc Signo Vinces (part 7): TPC-H Q13: The Good and the Bad Plans In Hoc Signo Vinces (part 8): TPC-H: INs, Expressions, ORs In Hoc Signo Vinces (part 9): TPC-H Q18, Ordered Aggregation, and Top K In Hoc Signo Vinces (part 10): TPC-H Q9, Q17, Q20 - Predicate Games In Hoc Signo Vinces (part 11): TPC-H Q2, Q10 - Late Projection In Hoc Signo Vinces (part 12): TPC-H: Result Preview In Hoc Signo Vinces (part 13): Virtuoso TPC-H Kit Now on V7 Fast Track In Hoc Signo Vinces (part 14): Virtuoso TPC-H Implementation Analysis In Hoc Signo Vinces (part 15): TPC-H and the Science of Hash In Hoc Signo Vinces (part 16): Introduction to Scale-Out In Hoc Signo Vinces (part 17): 100G and 300G Runs on Dual Xeon E5 2650v2 In Hoc Signo Vinces (part 18): Cluster Dynamics In Hoc Signo Vinces (part 19): Scalability, 1000G, and 3000G (this post) In Hoc Signo Vinces (part 20): 100G and 1000G With Cluster; When is Cluster Worthwhile; Effects of I/O In Hoc Signo Vinces (part 21): Running TPC-H on Virtuoso Cluster on Amazon EC2 Tue, 30 Sep 2014 20:33:29 GMTVirtuoso Universal Server 08.03.3334Orri Erlinghttp://www.openlinksw.com:443/weblog/public/images/vbloglogo.gifhttp://www.openlinksw.com:443/weblog/oerling/?id=18228831