<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Thanks for the response. I have been having this problem for a while
but now that I test again I cant reproduce it... If it comes backup
I will run these commands thanks again..<br>
<br>
<div class="moz-cite-prefix">On 2/11/14, 9:22 PM, Venky Shankar
wrote:<br>
</div>
<blockquote
cite="mid:CAO48DR6PEqhsM-=5e7oVePNPOQANPh_44voMLJvzyng91p0p4Q@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">Could you
provide this information from the server (also the
client/server logs):</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">
<br>
</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">cat
/proc/<pid>/task/*/stack</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">ls -l
/proc/<pid>/task/*/fd</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><pid> ==
process ID of glusterfsd</div>
</div>
<div class="gmail_extra">
<br>
<br>
<div class="gmail_quote">On Fri, Feb 7, 2014 at 6:14 PM, Barry
Stetler <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:barry@hivelocity.net" target="_blank">barry@hivelocity.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
We started using gluster in a custom solution recently.
Basically we setup a gluster volume that we mount at /home.
Then we give users their own home directory with a gluster
quota. They have FTP, SFTP and SSHMOUNT access. I first used
vsftpd and notice that ftp connections will not close and
the load goes up ever couple of days the funny thing is the
load does not seem to slow down the server. The load went up
to 20 in the last two months but the server was not slow at
all... At first I thought it was vsftpd so I changed to
proftpd then the issue came back. If I reboot the server it
goes away for a few days depending on how many ftp
connections. I noticed in the dmesg log this making me think
the issue was some kind of leak in gluster-fuse<br>
<br>
This is on CentOS 6.5 64 bit.<br>
<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
INFO: task proftpd:7831 blocked for more than 120 seconds.<br>
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.<br>
proftpd D 0000000000000000 0 7831 4384
0x00000004<br>
ffff88031f3bde38 0000000000000082 0000000000000000
ffffffff81051439<br>
ffff88031f3bddc8 0000000300000000 ffff88031f3bddd8
ffff880335936040<br>
ffff8803365cfab8 ffff88031f3bdfd8 000000000000fb88
ffff8803365cfab8<br>
Call Trace:<br>
[<ffffffff81051439>] ? __wake_up_common+0x59/0x90<br>
[<ffffffffa0213075>] fuse_request_send+0xe5/0x290
[fuse]<br>
[<ffffffff81096da0>] ? autoremove_wake_function+0x0/0x40<br>
[<ffffffffa0218b36>] fuse_flush+0x106/0x140 [fuse]<br>
[<ffffffff8117e0bc>] filp_close+0x3c/0x90<br>
[<ffffffff8117e1b5>] sys_close+0xa5/0x100<br>
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b<br>
<br>
<br>
I appreciate any help I can get on this...<br>
<br>
Thank you<span class="HOEnZb"><font color="#888888"><br>
<br>
-- <br>
Barry Stetler<br>
Hivelocity Hosting<br>
<a moz-do-not-send="true" href="http://hivelocity.net"
target="_blank">http://hivelocity.net</a><br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org"
target="_blank">Gluster-users@gluster.org</a><br>
<a moz-do-not-send="true"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</font></span></blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<div class="moz-signature">-- <br>
Barry Stetler<br>
Hivelocity Hosting<br>
<a href="http://hivelocity.net">http://hivelocity.net</a></div>
</body>
</html>