Hello again,<div><br></div><div>For testing purposes, I have deleted my original distributed/replicated volume that was seeing write's at around 10 MB/s and created one on the same cluster that is only distributed. </div>
<div><br></div><div>Now, I'm seeing the kind of performance I was hoping to see from distributed/replicated:</div><div><br></div><div><div>[root@vm-container-0-0 ~]# gluster volume info pifs</div><div><br></div><div>Volume Name: pifs</div>
<div>Type: Distribute</div><div>Status: Started</div><div>Number of Bricks: 4</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: vm-container-0-0:/gluster</div><div>Brick2: vm-container-0-1:/gluster</div><div>
Brick3: vm-container-0-2:/gluster</div><div>Brick4: vm-container-0-3:/gluster</div><div><br></div><div><div>[root@vm-container-0-0 ~]# mount | grep pifs</div><div>glusterfs#127.0.0.1:pifs on /pifs type fuse (rw,allow_other,default_permissions,max_read=131072)</div>
</div><div><br></div><div>[root@vm-container-0-0 ~]# dd if=/dev/zero of=/pifs/dd_test.img bs=1M count=2000</div><div>2000+0 records in</div><div>2000+0 records out</div><div>2097152000 bytes (2.1 GB) copied, 22.2793 seconds, 94.1 MB/s</div>
<div><br></div><div><br></div><div>An order of magnitude slower with replication. What's going on I wonder? Thanks for any suggestions.</div><div><br></div><div> --joey</div><div><br></div><div><br></div><br><div class="gmail_quote">
On Wed, Aug 10, 2011 at 10:05 AM, Joey McDonald <span dir="ltr"><<a href="mailto:joey@scare.org">joey@scare.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br><br><div class="gmail_quote"><div class="im">On Tue, Aug 9, 2011 at 11:19 PM, Mohit Anchlia <span dir="ltr"><<a href="mailto:mohitanchlia@gmail.com" target="_blank">mohitanchlia@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
And can you also give the mount options of gluster fs?<br>
<div><div></div><div><br></div></div></blockquote><div><br></div></div><div>Sure:</div><div><br></div><div><div>[root@vm-container-0-0 ~]# tail -2 /etc/fstab </div><div>/dev/sdb1 /gluster ext3 defaults 0 1</div>
<div>127.0.0.1:pifs /pifs glusterfs defaults,_netdev 0 0</div></div><div><br></div><div>I'm not running VM's off the gluster share but that's the general idea of where I'd like to get with this platform. That's obviously going to require some different mount options but I'd like to get this performance issue resolved before I start in on that.</div>
<div><br></div><font color="#888888"><div><br></div><div> --joey</div></font><div><div></div><div class="h5"><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div>
On Tue, Aug 9, 2011 at 4:41 PM, Joey McDonald <<a href="mailto:joey@scare.org" target="_blank">joey@scare.org</a>> wrote:<br>
><br>
><br>
> On Tue, Aug 9, 2011 at 5:40 PM, Joey McDonald <<a href="mailto:joey@scare.org" target="_blank">joey@scare.org</a>> wrote:<br>
>><br>
>> Hi Pavan,<br>
>> Thanks for your quick reply, comments inline:<br>
>><br>
>>><br>
>>> 1. Are these baremetal systems or are they Virtual machines ?<br>
>><br>
>> Bare metal systems.<br>
>><br>
>>><br>
>>> 2. What is the amount of RAM of each of these systems ?<br>
>><br>
>> They all have 4194304 kB of memory.<br>
>><br>
>>><br>
>>> 3. How many CPUs do they have ?<br>
>><br>
>> They each have 8 procs.<br>
>><br>
>>><br>
>>> 4. Can you also perform the dd on /gluster as opposed to /root to check<br>
>>> the backend performance ?<br>
>><br>
>> Sure, here is that output:<br>
>><br>
>> [root@vm-container-0-0 ~]# dd if=/dev/zero of=/gluster/dd_test.img bs=1M<br>
>> count=2000<br>
>> 2000+0 records in<br>
>> 2000+0 records out<br>
>> <a href="tel:2097152000" value="+12097152000" target="_blank">2097152000</a> bytes (2.1 GB) copied, 6.65193 seconds, 315 MB/s<br>
>><br>
>>><br>
>>> 5. What is your disk backend ? Is it direct attached or is it an array ?<br>
>><br>
>> Direct attached, /gluster is /dev/sdb1, 1TB SATA drive (as is /dev/sda):<br>
>> [root@vm-container-0-0 ~]# hdparm -i /dev/sdb<br>
>> /dev/sdb:<br>
>> Model=WDC WD1002FBYS-02A6B0 , FwRev=03.00C06, SerialNo=<br>
>> WD-WMATV5311442<br>
>> Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq<br>
>> }<br>
>> RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=50<br>
>> BuffType=unknown, BuffSize=32767kB, MaxMultSect=16, MultSect=?0?<br>
>> CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=268435455<br>
>> IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}<br>
>> PIO modes: pio0 pio3 pio4<br>
>> DMA modes: mdma0 mdma1 mdma2<br>
>> UDMA modes: udma0 udma1 udma2<br>
>> AdvancedPM=no WriteCache=enabled<br>
>> Drive conforms to: Unspecified: ATA/ATAPI-1 ATA/ATAPI-2 ATA/ATAPI-3<br>
>> ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7<br>
>> * signifies the current active mode<br>
>><br>
>>><br>
>>> 6. What is the backend filesystem ?<br>
>><br>
>> ext3<br>
>><br>
>>><br>
>>> 7. Can you run a simple scp of about 10M between any two of these systems<br>
>>> and report the speed ?<br>
>><br>
>> Sure, output:<br>
>> [root@vm-container-0-1 ~]# scp vm-container-0-0:/gluster/dd_test.img .<br>
>> Warning: Permanently added 'vm-container-0-0' (RSA) to the list of known<br>
>> hosts.<br>
>> root@vm-container-0-0's password:<br>
>> dd_test.img<br>
>> 100% 2000MB<br>
>> 39.2MB/s 00:51<br>
>><br>
>> --joey<br>
>><br>
>><br>
><br>
><br>
</div></div><div><div></div><div>> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> <a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
><br>
><br>
</div></div></blockquote></div></div></div><br>
</blockquote></div><br></div>