<div dir="ltr">I second that comment !<div><br></div><div>Boy, I was chasing ghosts until very recently, regarding disk configurations (physical RAID).</div><div><br></div><div>I had a 4 x 1 TB SAS disks, configured as RAID5 on a Dell PERC 6, and thought that would be good enough. </div>
<div><br></div><div>My network throughput would not go over 280 Mbps and I was blaming Linux. </div><div><br></div><div>Then I finally decided to test disk speed with:</div><div><br></div><div>dd if=/dev/zero of=/localvolume bs=512k count=17000</div>
<div><br></div><div>Much to my surprise it reported something like 35-40 Mbytes/s. </div><div><br></div><div>I then changed my layout to RAID 10 and the same test bumped results to 400 Mbytes/sec. Average network throughput now is around 600 mbps, with spikes of 750 mbps.</div>
<div><br></div><div>Ah... You might want to make sure your network is set to use jumbo frames.</div><div><br></div><div>Cheers, </div><div><br></div><div>Carlos<br><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Thu, Apr 3, 2014 at 3:52 PM, Josh Boon <span dir="ltr"><<a href="mailto:gluster@joshboon.com" target="_blank">gluster@joshboon.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div style="font-size:10pt;font-family:Arial"><div>Hey David,</div><div><br></div><div>Can you provide the qemu command to run each of them? What's your gluster/disk/network layout look like?</div><div><br></div>
<div>Depending on your disk and network setup you may be hitting a bottleneck there that would prevent gfapi from performing at capacity. Lots of options here that could impact things.</div><div><br></div><div><br></div>
<hr><div style="font-size:12pt;font-style:normal;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal"><b>From: </b>"Dave Christianson" <<a href="mailto:davidchristianson3@gmail.com" target="_blank">davidchristianson3@gmail.com</a>><br>
<b>To: </b><a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br><b>Sent: </b>Thursday, April 3, 2014 6:05:51 AM<br><b>Subject: </b>[Gluster-users] No performance difference using libgfapi?<div>
<div class="h5"><br><div><br></div><div dir="ltr">Good Morning,<div><br></div><div>In my earlier experience invoking a VM using qemu/libgfapi, I reported that it was noticeably faster than the same VM invoked from libvirt using a FUSE mount; however, this was erroneous as the qemu/libgfapi-invoked image was created using 2x the RAM and cpu's...</div>
<div><br></div><div>So, invoking the image using both methods using consistent specs of 2GB RAM and 2 cpu's, I attempted to check drive performance using the following commands:</div><div><br></div><div>(For regular FUSE mount I have the gluster volume mounted at /var/lib/libvirt/images.)</div>
<div><br></div><div>(For libgfapi I specify -disk file=gluster://gfs-00/gfsvol/tester1/img.)</div><div><br></div><div>Using libvirt/FUSE mount:</div><div>[root@tester1 ~]# hdparm -Tt /dev/vda1</div><div>/dev/vda1:</div><div>
Timing cached reads: 11346 MB in 2.00 seconds = 5681.63 MB/sec</div><div> Timing buffered disk reads: 36 MB in 3.05 seconds = 11.80 MB/sec</div><div>[root@tester1 ~]# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output<br>
</div><div>10240+0 records in</div><div>10240+0 records out</div><div>41943040 bytes (42MB) copied, 0.0646241 s, 649 MB/sec</div><div><br></div><div>Using qemu/libgfapi:<br></div><div><div>[root@tester1 ~]# hdparm -Tt /dev/vda1</div>
<div>/dev/vda1:</div><div> Timing cached reads: 11998 MB in 2.00 seconds = 6008.57 MB/sec</div><div> Timing buffered disk reads: 36 MB in 3.03 seconds = 11.89 MB/sec</div><div>[root@tester1 ~]# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output<br>
</div><div>10240+0 records in</div><div>10240+0 records out</div><div>41943040 bytes (42MB) copied, 0.0621412 s, 675 MB/sec</div></div><div><br></div><div>Should I be seeing a bigger difference, or am I doing something wrong? </div>
<div><br></div><div>I'm also curious whether people have found that the performance difference is greater as the size of the gluster cluster scales up.</div><div><br></div><div>Thanks,</div><div>David</div><div><br></div>
</div><br></div></div>_______________________________________________<br>Gluster-users mailing list<br><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></div>
<div><br></div></div></div><br>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>