We applied the patch mentioned the thread, and use fixed thread count in the server config. Unfortunately, we got the same error:<br><br>[2009-06-03 04:57:36] W [fuse-bridge.c:2284:fuse_setlk_cbk] glusterfs-fuse: 22347008: ERR => -1 (Resource temporarily unavailable)<br>
[2009-06-03 07:55:04] W [fuse-bridge.c:2284:fuse_setlk_cbk] glusterfs-fuse: 23431094: ERR => -1 (Resource temporarily unavailable)<br>[2009-06-03 15:58:25] E [client-protocol.c:292:call_bail] brick1: bailing out frame LOOKUP(32) frame sent = 2009-06-03 15:28:23. frame-timeout = 1800<br>
<br>John<br><br><br><div class="gmail_quote">On Tue, Jun 2, 2009 at 12:25 AM, Shehjar Tikoo <span dir="ltr"><<a href="mailto:shehjart@gluster.com">shehjart@gluster.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Hi<div class="im"><br>
<br>
><br>
> Also, avoid using autoscaling in io-threads for now.<br>
><br>
> -Shehjar<br>
><br>
><br>
<br></div>
-Shehjar<br>
<br>
Alpha Electronics wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="h5">
Thanks for looking into this. We do use io-threads. Here is the server config:<br>
: volume brick1-posix<br>
2: type storage/posix<br>
3: option directory /mnt/brick1<br>
4: end-volume<br>
5:<br>
6: volume brick2-posix<br>
7: type storage/posix<br>
8: option directory /mnt/brick2<br>
9: end-volume<br>
10:<br>
11:<br>
12: volume brick1-locks<br>
13: type features/locks<br>
14: subvolumes brick1-posix<br>
15: end-volume<br>
16:<br>
17: volume brick2-locks<br>
18: type features/locks<br>
19: subvolumes brick2-posix<br>
20: end-volume<br>
21:<br>
22: volume brick1<br>
23: type performance/io-threads<br>
24: option min-threads 16<br>
25: option autoscaling on<br>
26: subvolumes brick1-locks<br>
27: end-volume<br>
28:<br>
29: volume brick2<br>
30: type performance/io-threads<br>
31: option min-threads 16<br>
32: option autoscaling on<br>
33: subvolumes brick2-locks<br>
34: end-volume<br>
35:<br>
36: volume server<br>
37: type protocol/server<br>
38: option transport-type tcp<br>
40: option auth.addr.brick1.allow *<br>
41: option auth.addr.brick2.allow *<br>
42: subvolumes brick1 brick2<br>
43: end-volume<br>
44:<br>
<br>
<br>
<br></div></div><div><div></div><div class="h5">
On Sun, May 31, 2009 at 11:44 PM, Shehjar Tikoo <<a href="mailto:shehjart@gluster.com" target="_blank">shehjart@gluster.com</a> <mailto:<a href="mailto:shehjart@gluster.com" target="_blank">shehjart@gluster.com</a>>> wrote:<br>
<br>
Alpha Electronics wrote:<br>
<br>
We are testing the glusterfs before recommending them to<br>
enterprise clients. We found that the file system always hang<br>
after running for about 2 days. after killing the server side<br>
process and then restart, everything goes back to normal.<br>
<br>
<br>
What is the server config?<br>
If you're not using io-threads on the server, I suggest you do,<br>
because it does basic load-balancing to avoid timeouts.<br>
<br>
Also, avoid using autoscaling in io-threads for now.<br>
<br>
-Shehjar<br>
<br>
<br>
Here is the spec and error logged:<br>
GlusterFS version: v2.0.1<br>
<br>
Client volume:<br>
volume brick_1<br>
type protocol/client<br>
option transport-type tcp/client<br>
option remote-port 7777 # Non-default port<br>
option remote-host server1<br>
option remote-subvolume brick<br>
end-volume<br>
<br>
volume brick_2<br>
type protocol/client<br>
option transport-type tcp/client<br>
option remote-port 7777 # Non-default port<br>
option remote-host server2<br>
option remote-subvolume brick<br>
end-volume<br>
<br>
volume bricks<br>
type cluster/distribute<br>
subvolumes brick_1 brick_2<br>
end-volume<br>
<br>
Error logged on client side through /var/log/glusterfs.log<br>
[2009-05-29 14:58:55] E [client-protocol.c:292:call_bail]<br>
brick_1: bailing out frame LK(28) frame sent = 2009-05-29<br>
14:28:54. frame-timeout = 1800<br>
[2009-05-29 14:58:55] W [fuse-bridge.c:2284:fuse_setlk_cbk]<br>
glusterfs-fuse: 106850788: ERR => -1 (Transport endpoint is not<br>
connected)<br>
error logged on server<br>
[2009-05-29 14:59:15] E [client-protocol.c:292:call_bail]<br>
brick_2: bailing out frame LK(28) frame sent = 2009-05-29<br>
14:29:05. frame-timeout = 1800<br>
[2009-05-29 14:59:15] W [fuse-bridge.c:2284:fuse_setlk_cbk]<br>
glusterfs-fuse: 106850860: ERR => -1 (Transport endpoint is not<br>
connected)<br>
<br>
There is error message logged on server side after 1 hour in<br>
/var/log/messages:<br>
May 29 16:04:16 server2 winbindd[3649]: [2009/05/29 16:05:16, 0]<br>
lib/util_sock.c:write_data(564)<br>
May 29 16:04:16 server2 winbindd[3649]: write_data: write<br>
failure. Error = Connection reset by peer<br>
May 29 16:04:16 server2 winbindd[3649]: [2009/05/29 16:05:16, 0]<br>
libsmb/clientgen.c:write_socket(158)<br>
May 29 16:04:16 server2 winbindd[3649]: write_socket: Error<br>
writing 104 bytes to socket 18: ERRNO = Connection reset by peer<br>
May 29 16:04:16 server2 winbindd[3649]: [2009/05/29 16:05:16, 0]<br>
libsmb/clientgen.c:cli_send_smb(188)<br>
May 29 16:04:16 server2 winbindd[3649]: Error writing 104<br>
bytes to client. -1 (Connection reset by peer)<br>
May 29 16:04:16 server2 winbindd[3649]: [2009/05/29 16:05:16, 0]<br>
libsmb/cliconnect.c:cli_session_setup_spnego(859)<br>
May 29 16:04:16 server2 winbindd[3649]: Kinit failed: Cannot<br>
contact any KDC for requested realm<br>
<br>
<br>
------------------------------------------------------------------------<br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br></div></div>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a> <mailto:<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a>><div class="im"><br>
<a href="http://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br>
<br>
<br>
<br>
</div></blockquote>
<br>
</blockquote></div><br><br clear="all"><br>