<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>Hi,<o:p></o:p></p>
<p class=MsoNormal>I have a setup using server-side afr (glusterfs-1.3.11). Is
it possible when one of the server went down, it will result in the other
server also clashing a few seconds later? Below are some of the log just before
the second server starting to crash after the first server went down. Could
someone also explain what " forced unwinding frame type(1) op(35)
reply=@0xb4091b58" and " frame : type(0) op(0)" means? Tks in
advance.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>2009-01-05 18:56:08 E
[client-protocol.c:4834:client_protocol_cleanup] gfs-storage1-ds: forced
unwinding frame type(1) op(35) reply=@0xb4091b58<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:08 E
[client-protocol.c:4834:client_protocol_cleanup] gfs-storage1-ds: forced
unwinding frame type(1) op(35) reply=@0xb4091b58<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:08 E
[client-protocol.c:4834:client_protocol_cleanup] gfs-storage1-ds: forced
unwinding frame type(1) op(35) reply=@0xb4091b58<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:08 E
[client-protocol.c:4834:client_protocol_cleanup] gfs-storage1-ds: forced
unwinding frame type(1) op(35) reply=@0xb4091b58<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:08 E
[client-protocol.c:4834:client_protocol_cleanup] gfs-storage1-ds: forced
unwinding frame type(1) op(35) reply=@0xb4091b58<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:08 E [client-protocol.c:4834:client_protocol_cleanup]
gfs-storage1-ds: forced unwinding frame type(1) op(35) reply=@0xb4091b58<o:p></o:p></p>
<p class=MsoNormal>.<o:p></o:p></p>
<p class=MsoNormal>.<o:p></o:p></p>
<p class=MsoNormal>.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E
[client-protocol.c:4430:client_lookup_cbk] gfs-storage1-ds: no proper reply
from server, returning ENOTCONN<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E
[client-protocol.c:4834:client_protocol_cleanup] gfs-storage1-ds: forced
unwinding frame type(1) op(34) reply=@0xb4091b58<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E
[client-protocol.c:4430:client_lookup_cbk] gfs-storage1-ds: no proper reply
from server, returning ENOTCONN<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp.c:87:tcp_disconnect]
gfs-storage1-ds: connection disconnected<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [afr.c:6025:notify] gfs-ds-afr:
GF_EVENT_CHILD_DOWN from gfs-storage1-ds<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp-server.c:145:tcp_server_notify]
server: Registering socket (6) for new transport object of 172.18.229.31<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [inode.c:321:__destroy_inode]
gfs/inode: destroy inode(123814120) [@0xb40afdb0]<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 W [client-protocol.c:1296:client_close]
gfs-storage1-ds: no valid fd found, returning<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [ip.c:120:gf_auth] gfs: allowed =
"*", received ip addr = "172.18.229.31"<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [server-protocol.c:5674:mop_setvolume]
server: accepted client from 172.18.229.31:1020<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp-server.c:145:tcp_server_notify]
server: Registering socket (42) for new transport object of 172.18.229.41<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [ip.c:120:gf_auth] gfs: allowed =
"*", received ip addr = "172.18.229.41"<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [server-protocol.c:5674:mop_setvolume]
server: accepted client from 172.18.229.41:1019<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp-server.c:145:tcp_server_notify]
server: Registering socket (43) for new transport object of 172.18.229.33<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 W [client-protocol.c:1296:client_close]
gfs-storage1-ds: no valid fd found, returning<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E
[client-protocol.c:325:client_protocol_xfer] gfs-storage1-ds: transport_submit
failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 W [client-protocol.c:1296:client_close]
gfs-storage1-ds: no valid fd found, returning<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp-client.c:77:tcp_connect]
gfs-storage1-ds: socket fd = 41<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [inode.c:321:__destroy_inode]
gfs/inode: destroy inode(113328382) [@0xb3ac0348]<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp-client.c:107:tcp_connect]
gfs-storage1-ds: finalized on port `1023'<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp-client.c:128:tcp_connect]
gfs-storage1-ds: defaulting remote-port to 6996<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [common-utils.c:179:gf_resolve_ip]
resolver: DNS cache not present, freshly probing hostname: cyvmvqb3.world.net<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D
[server-protocol.c:6269:server_protocol_cleanup] server: cleaned up transport
state for client 172.18.229.33:1023<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D
[server-protocol.c:6269:server_protocol_cleanup] server: cleaned up transport
state for client 172.18.229.33:1021<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D
[server-protocol.c:6269:server_protocol_cleanup] server: cleaned up transport
state for client 172.18.229.31:1021<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [common-utils.c:204:gf_resolve_ip]
resolver: returning IP:172.18.228.156[0] for hostname: cyvmvqb3.world.net<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [common-utils.c:212:gf_resolve_ip]
resolver: flushing DNS cache<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [tcp-client.c:161:tcp_connect]
gfs-storage1-ds: connect on 41 in progress (non-blocking)<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D
[server-protocol.c:6269:server_protocol_cleanup] server: cleaned up transport
state for client 172.18.229.41:1020<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D
[server-protocol.c:6269:server_protocol_cleanup] server: cleaned up transport
state for client 172.18.229.35:1023<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 E [server-protocol.c:186:generic_reply]
server: transport_writev failed<o:p></o:p></p>
<p class=MsoNormal>2009-01-05 18:56:10 D [server-protocol.c:6269:server_protocol_cleanup]
server: cleaned up transport state for client 172.18.229.40:1021<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>TLA Repo Revision: glusterfs--mainline--2.5--patch-795<o:p></o:p></p>
<p class=MsoNormal>Time : 2009-01-05 18:56:10<o:p></o:p></p>
<p class=MsoNormal>Signal Number : 11<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>/usr/sbin/glusterfsd -f /etc/glusterfs/glusterfs-server.vol
-l /var/log/glusterfsd.log -L DEBUG<o:p></o:p></p>
<p class=MsoNormal>volume server<o:p></o:p></p>
<p class=MsoNormal> type protocol/server<o:p></o:p></p>
<p class=MsoNormal> option auth.ip.gfs.allow *<o:p></o:p></p>
<p class=MsoNormal> option auth.ip.gfs-ds-locks.allow *,127.0.0.1<o:p></o:p></p>
<p class=MsoNormal> option transport-type tcp/server<o:p></o:p></p>
<p class=MsoNormal> subvolumes gfs<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume gfs<o:p></o:p></p>
<p class=MsoNormal> type performance/io-threads<o:p></o:p></p>
<p class=MsoNormal> option cache-size 96MB<o:p></o:p></p>
<p class=MsoNormal> option thread-count 4<o:p></o:p></p>
<p class=MsoNormal> subvolumes gfs-ds-afr<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume gfs-ds-afr<o:p></o:p></p>
<p class=MsoNormal> type cluster/afr<o:p></o:p></p>
<p class=MsoNormal> subvolumes gfs-ds-locks gfs-storage1-ds<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume gfs-storage1-ds<o:p></o:p></p>
<p class=MsoNormal> type protocol/client<o:p></o:p></p>
<p class=MsoNormal> option transport-timeout 10<o:p></o:p></p>
<p class=MsoNormal> option remote-subvolume gfs-ds-locks<o:p></o:p></p>
<p class=MsoNormal> option remote-host cyvmvqb3.world.net<o:p></o:p></p>
<p class=MsoNormal> option transport-type tcp/client<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume gfs-ds-locks<o:p></o:p></p>
<p class=MsoNormal> type features/posix-locks<o:p></o:p></p>
<p class=MsoNormal> subvolumes gfs-ds<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume gfs-ds<o:p></o:p></p>
<p class=MsoNormal> type storage/posix<o:p></o:p></p>
<p class=MsoNormal> option directory /media/gfs-ds<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>frame : type(0) op(0)<o:p></o:p></p>
<p class=MsoNormal>frame : type(0) op(0)<o:p></o:p></p>
<p class=MsoNormal>frame : type(0) op(0)<o:p></o:p></p>
<p class=MsoNormal>frame : type(0) op(0)<o:p></o:p></p>
<p class=MsoNormal>frame : type(0) op(0)<o:p></o:p></p>
<p class=MsoNormal>frame : type(0) op(0)<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>regards,<o:p></o:p></p>
<p class=MsoNormal>melvin<o:p></o:p></p>
</div>
</body>
</html>