<br><br><div class="gmail_quote">On Wed, Jul 6, 2011 at 6:58 AM, Tomasz Chmielewski <span dir="ltr"><<a href="mailto:mangoo@wpkg.org">mangoo@wpkg.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
One of the peers is behaving very slow - load is constantly around 6-15, and generally all connected clients work very slow.<br>
<br>
As soon as I shut that peer down, clients are relatively fast.<br>
<br>
<br>
What I noticed, that on one gluster peer, I only get around 100 files opened by the gluster process:<br>
<br>
# lsof -n|grep -c gluster<br>
96<br>
<br>
<br>
On the slow peer, it's hundreds, tens of thousands, and growing constantly:<br>
<br>
<br>
# while true; do lsof -n|grep -c gluster; sleep 10; done<br>
169934<br>
170181<br>
170363<br>
170655<br>
<br>
<br>
If I restart glusterd on that peer, it starts to grow again.<br>
<br>
<br>
<br>
On both peers, I have a big number of such entries:<br>
<br>
[2011-07-06 03:22:00.20873] W [server-resolve.c:556:server_<u></u>resolve] 0-gluster-data-server: pure path resolution for /some/path/.../file (LOOKUP)<br>
<br>
<br>
Where the path is a symbolic link to the gluster mount.<br></blockquote><div><br></div><div><br></div><div>Can you describe this a little more? What do you mean by those paths are symbolic links to the gluster mount? They are supposed to be files within the gluster mount. Do you have a gluster mount on that peer as well? If so, does unmounting have any effect?</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
The unaffected peer has these entries in the log as well:<br>
<br>
[2011-07-06 03:23:30.117245] E [posix.c:3672:do_xattrop] 0-gluster-data-posix: getxattr failed on /some/path/.../file while doing xattrop: Numerical result out of range<br>
</blockquote><div><br></div><div><br></div><div>This is strange. There is almost no reason how you can be seeing this error unless the extended attributes were modified outside gluster. Can you get a dump of those attributes on those files with 'getfattr -d -m . -e hex /some/path/../file' and post it here?</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"> </blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Why is gluster on one of the peers opening so many files?<br><font color="#888888">
<br>
<br></font></blockquote><div><br></div><div>It is certainly strange that it's opening more files on only one peer. Please give as much info about the setup as possible.</div><div><br></div><div>Avati </div></div>