<div dir="ltr"><div class="gmail_extra">On Fri, Jun 7, 2013 at 5:00 AM, <span dir="ltr"><<a href="mailto:gluster-users-request@gluster.org" target="_blank">gluster-users-request@gluster.org</a>></span> wrote:<br><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
Message: 2<br>
Date: Thu, 6 Jun 2013 16:07:08 +0200<br>
From: Stephan von Krawczynski <<a href="mailto:skraw@ithnet.com" target="_blank">skraw@ithnet.com</a>><br>
To: Pablo <<a href="mailto:paa.listas@gmail.com" target="_blank">paa.listas@gmail.com</a>><br>
Cc: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] GlusterFS (3.3.1) - performance issues -<br>
large number of LOOKUP calls & high CPU usage<br>
Message-ID: <<a href="mailto:20130606160708.898f5049.skraw@ithnet.com" target="_blank">20130606160708.898f5049.skraw@ithnet.com</a>><br>
Content-Type: text/plain; charset=US-ASCII<br>
<br>
On Thu, 06 Jun 2013 10:39:21 -0300<br>
Pablo <<a href="mailto:paa.listas@gmail.com" target="_blank">paa.listas@gmail.com</a>> wrote:<br>
<br>
> I have never try this (In fact I'm just learning a bit more how to<br>
> administer a Gluster server.), buy you may find it useful.<br>
><br>
> <a href="http://download.gluster.org/pub/gluster/glusterfs/doc/HA%20and%20Load%20Balancing%20for%20NFS%20and%20SMB.html" target="_blank">http://download.gluster.org/pub/gluster/glusterfs/doc/HA%20and%20Load%20Balancing%20for%20NFS%20and%20SMB.html</a><br>
><br>
> Pablo.<br>
<br>
The thing with this way of failover is though, that you will likely corrupt a<br>
currently written file. If your NFS-server (gluster) node dies while you write<br>
your file will be corrupt. If you use native glusterfs mounts it will not<br>
(should not). This is why I consider the NFS server feature nothing more than<br>
a bad hack. It does not deliver the safety that glusterfs promises, even if<br>
you solve the failover problem somehow.<br>
<br>
--<br>
Regards,<br>
Stephan<br>
<br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
<br>
End of Gluster-users Digest, Vol 62, Issue 17<br>
*********************************************<br>
</blockquote></div><div class="gmail_extra"><br></div>Yes - and in fact, we'd rather NOT use NFS if it can be avoided.</div><div class="gmail_extra"><br></div><div class="gmail_extra">As another point of interest, the GlusterFS FUSE native appears to be due to the large number of LOOKUP calls as initially indicated (69000s for FUSE vs. 592 for NFS). We also see a tremendous increase in network overhead using the FUSE client which I suppose is related to the LOOKUP calls (about 300-400k sustained over 60s for FUSE, 30-50k over about 10s for NFS).</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">Any idea as to what would be causing this overhead?<br>
<br>===FUSE===<br clear="all"><div><div>Interval 10 Stats:</div><div> Block Size: 1b+ 2b+ 4b+</div><div> No. of Reads: 0 0 0</div>
<div>No. of Writes: 15 12 20</div><div><br></div><div> Block Size: 8b+ 16b+ 32b+</div><div> No. of Reads: 0 0 0</div>
<div>No. of Writes: 27 17 59</div><div><br></div><div> Block Size: 64b+ 128b+ 256b+</div><div> No. of Reads: 1 0 0</div>
<div>No. of Writes: 68 27 24</div><div><br></div><div> Block Size: 512b+ 1024b+ 2048b+</div><div> No. of Reads: 5 20 5</div>
<div>No. of Writes: 60 0 10</div><div><br></div><div> Block Size: 4096b+</div><div> No. of Reads: 0</div><div>No. of Writes: 30</div>
<div> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop</div><div> --------- ----------- ----------- ----------- ------------ ----</div><div> 0.00 0.00 us 0.00 us 0.00 us 58 FORGET</div>
<div> 0.00 0.00 us 0.00 us 0.00 us 671 RELEASE</div><div> 0.00 0.00 us 0.00 us 0.00 us 114 RELEASEDIR</div><div> 0.01 31.20 us 21.00 us 62.00 us 10 LK</div>
<div> 0.02 38.45 us 32.00 us 56.00 us 31 READ</div><div> 0.03 210.17 us 187.00 us 271.00 us 6 STATFS</div><div> 0.03 287.20 us 264.00 us 313.00 us 5 MKDIR</div>
<div> 0.11 62.73 us 48.00 us 168.00 us 82 TRUNCATE</div><div> 0.11 38.25 us 23.00 us 620.00 us 135 STAT</div><div> 0.13 52.50 us 37.00 us 268.00 us 114 OPENDIR</div>
<div> 0.15 156.36 us 129.00 us 247.00 us 45 UNLINK</div><div> 0.21 123.66 us 51.00 us 2014.00 us 82 SETATTR</div><div> 0.23 132.70 us 90.00 us 254.00 us 82 RENAME</div>
<div> 0.44 56.75 us 33.00 us 1096.00 us 369 WRITE</div><div> 0.44 45.49 us 24.00 us 3014.00 us 461 FSTAT</div><div> 0.69 292.22 us 216.00 us 1777.00 us 112 CREATE</div>
<div> 0.75 64.18 us 36.00 us 3763.00 us 559 OPEN</div><div> 0.94 46.40 us 10.00 us 2253.00 us 963 FLUSH</div><div> 3.55 669.45 us 83.00 us 3746.00 us 253 READDIRP</div>
<div> 4.10 2385.21 us 762.00 us 5424.00 us 82 FSYNC</div><div> 88.09 60.94 us 40.00 us 21474.00 us 69004 LOOKUP</div><div><br></div><div> Duration: 113 seconds</div>
<div> Data Read: 47519 bytes</div><div>Data Written: 224013 bytes</div></div><div><br></div><div><br></div><div style>===NFS===</div><div><div>Interval 13 Stats:</div><div> Block Size: 8b+ 16b+ 64b+ </div>
<div> No. of Reads: 0 0 0 </div><div>No. of Writes: 5 10 11 </div><div> </div><div> Block Size: 128b+ 256b+ 512b+ </div>
<div> No. of Reads: 0 0 0 </div><div>No. of Writes: 6 10 23 </div><div> </div><div> Block Size: 1024b+ 2048b+ 16384b+ </div>
<div> No. of Reads: 0 0 0 </div><div>No. of Writes: 18 12 5 </div><div> </div><div> %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop</div>
<div> --------- ----------- ----------- ----------- ------------ ----</div><div> 0.00 0.00 us 0.00 us 0.00 us 57 FORGET</div><div> 0.00 0.00 us 0.00 us 0.00 us 113 RELEASE</div>
<div> 0.07 225.00 us 199.00 us 251.00 us 2 STATFS</div><div> 0.09 58.22 us 43.00 us 137.00 us 9 OPEN</div><div> 0.12 36.65 us 20.00 us 139.00 us 20 LK</div>
<div> 0.24 292.20 us 277.00 us 297.00 us 5 MKDIR</div><div> 1.01 150.39 us 102.00 us 278.00 us 41 FSTAT</div><div> 1.09 66.31 us 18.00 us 194.00 us 100 FLUSH</div>
<div> 1.36 83.16 us 46.00 us 424.00 us 100 WRITE</div><div> 1.94 289.24 us 189.00 us 599.00 us 41 READDIRP</div><div> 2.28 29.13 us 14.00 us 181.00 us 478 ACCESS</div>
<div> 2.61 39.76 us 18.00 us 376.00 us 401 FINODELK</div><div> 2.98 81.78 us 52.00 us 383.00 us 222 SETATTR</div><div> 2.98 40.78 us 18.00 us 531.00 us 446 INODELK</div>
<div> 3.92 322.81 us 95.00 us 3197.00 us 74 RENAME</div><div> 4.71 143.64 us 83.00 us 303.00 us 200 FXATTROP</div><div> 4.91 287.68 us 218.00 us 679.00 us 104 CREATE</div>
<div> 5.48 54.41 us 16.00 us 10294.00 us 614 ENTRYLK</div><div> 6.10 35.55 us 23.00 us 299.00 us 1046 STAT</div><div> 10.15 104.55 us 42.00 us 2412.00 us 592 LOOKUP</div>
<div> 13.87 1692.02 us 782.00 us 4956.00 us 50 UNLINK</div><div> 34.09 196.52 us 91.00 us 14576.00 us 1058 XATTROP</div><div> </div><div> Duration: 39 seconds</div>
<div> Data Read: 0 bytes</div><div>Data Written: 222075 bytes</div></div><div><br></div><div dir="ltr"><div></div></div>
</div></div>