<div dir="ltr">Hi Niels,<div><br></div><div><div>We have tested the patch for some days. It works well when the gluster peer status </div><div>change to disconnected. However, if we retore the network just before the peer </div>
<div>status change to disconnected status, we found out that glusterfsd will still </div><div>open a new fd, and leave the old one not released even stop the file process. </div><div><br></div><div>Why does glusterfsd open a new fd instead of reusing the original reopened fd?</div>
</div><div>Does glusterfsd have any kind of mechanism retrieve such fds?</div><div><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-08-20 21:54 GMT+08:00 Niels de Vos <span dir="ltr"><<a href="mailto:ndevos@redhat.com" target="_blank">ndevos@redhat.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On Wed, Aug 20, 2014 at 07:16:16PM +0800, Jaden Liang wrote:<br>
> Hi gluster-devel team,<br>
><br>
> We are running a 2 replica volume in 2 servers. One of our service daemon<br>
> open a file with 'flock' in the volume. We can see every glusterfsd daemon<br>
> open the replica files in its own server(in /proc/pid/fd). When we pull off<br>
> the cable of one server about 10 minutes then re-plug in. We found that the<br>
> glusterfsd open a 'NEW' file descriptor while still holding the old one<br>
> which is opened in the first file access.<br>
><br>
> Then we stop our service daemon, but the glusterfsd(the re-plug cable one)<br>
> only closes the new fd, leave the old fd open, we think that may be a fd<br>
> leak issue. And we restart our service daemon. It flocked the same file,<br>
> and get a flock failure. The errno is Resource Temporary Unavailable.<br>
><br>
> However, this situation is not replay every time but often come out. We are<br>
> still looking into the source code of glusterfsd, but it is not a easy job.<br>
> So we want to look for some help in here. Here are our questions:<br>
><br>
> 1. Has this issue been solved? Or is it a known issue?<br>
> 2. Does anyone know the file descriptor maintenance logic in<br>
> glusterfsd(server-side)? When the fd will be closed or held?<br>
<br>
</div></div>I think you are hitting bug 1129787:<br>
- <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1129787" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1129787</a><br>
file locks are not released within an acceptable time when<br>
a fuse-client uncleanly disconnects<br>
<br>
There has been a (short) discussion about this earlier, see<br>
<a href="http://supercolony.gluster.org/pipermail/gluster-devel/2014-May/040748.html" target="_blank">http://supercolony.gluster.org/pipermail/gluster-devel/2014-May/040748.html</a><br>
<br>
Updating the proposed change is on my TODO list, in the end, the<br>
network.ping-timeout option should be used to define the timeout towards<br>
storage servers (like it is now) and the timeout from storage server to<br>
GlusterFS-client.<br>
<br>
You can try out the patch at <a href="http://review.gluster.org/8065" target="_blank">http://review.gluster.org/8065</a> and see if<br>
the network.tcp-timeout option works for you. Just remember that the<br>
option will get fold into the network.ping-timeout one later on. If you<br>
are interested in sending an updated patch, let me know :)<br>
<br>
Cheers,<br>
Niels<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div><font face="'courier new', monospace">Best regards,</font></div><font face="'courier new', monospace">Jaden Liang</font></div>
</div></div>