<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style>
</head>
<body class='hmmessage'>
Hi,<br>Don't know if it's going to help you but I noticed something : <br>The issue is on the "chown" cmd, but strangely when I'm using Bash shell I don't have this error, when using KSH shell => the error come back<br>In the two case, cmd is executed as root.<br><br>Cmd that I'm running : tar cf - . | (cd /users99/ylalXXX/save ; tar xf -)<br><br><br>> Date: Fri, 10 Jun 2011 12:53:02 +0530<br>> From: shehjart@gluster.com<br>> To: christopher.anderlik@xidras.com<br>> CC: gluster-users@gluster.org; sokar6012@hotmail.com<br>> Subject: Re: [Gluster-users] NFS problem<br>> <br>> We'll need the crash stack trace also.<br>> <br>> <br>> Christopher Anderlik wrote:<br>> > here are our logs when nfs is crashing....<br>> > <br>> > <br>> > <br>> > <br>> > <br>> > [2011-06-10 08:54:14.900049] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: f8851fc2, ACCESS: NFS: 0(Call completed <br>> > successfully.), POSIX: 0(Success)<br>> > [2011-06-10 08:54:14.902002] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: f9851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:14.902037] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:14.902062] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: f9851fc2, <br>> > GETATTR: args: FH: hashcount 3, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 74c48fb3-d065-462b-83a9-e4558b042465<br>> > [2011-06-10 08:54:14.920579] D <br>> > [afr-transaction.c:976:afr_post_nonblocking_inodelk_cbk] <br>> > 0-ksc-replicate-0: Non blocking inodelks done. Proceeding to FOP<br>> > [2011-06-10 08:54:14.921099] D [client-lk.c:442:delete_granted_locks_fd] <br>> > 0-ksc-client-0: Number of locks cleared=0<br>> > [2011-06-10 08:54:14.921155] D [client-lk.c:442:delete_granted_locks_fd] <br>> > 0-ksc-client-1: Number of locks cleared=0<br>> > [2011-06-10 08:54:14.932629] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: f9851fc2, GETATTR: NFS: 0(Call completed <br>> > successfully.), POSIX: 0(Success)<br>> > [2011-06-10 08:54:14.932863] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: fa851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:14.932890] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:14.932907] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: fa851fc2, <br>> > ACCESS: args: FH: hashcount 3, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 74c48fb3-d065-462b-83a9-e4558b042465<br>> > [2011-06-10 08:54:14.961700] D [socket.c:193:__socket_rwv] <br>> > 0-ksc-client-0: EOF from peer 10.0.1.198:24031<br>> > [2011-06-10 08:54:14.961741] W <br>> > [socket.c:1494:__socket_proto_state_machine] 0-ksc-client-0: reading <br>> > from socket failed. Error (Transport endpoint is not connected), peer <br>> > (10.0.1.198:24031)<br>> > [2011-06-10 08:54:14.961757] D [socket.c:1768:socket_event_handler] <br>> > 0-transport: disconnecting now<br>> > [2011-06-10 08:54:14.961858] E [rpc-clnt.c:338:saved_frames_unwind] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) <br>> > [0x7f140e8c0acc] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) <br>> > [0x7f140e8c006a] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) <br>> > [0x7f140e8bfb78]))) 0-ksc-client-0: forced unwinding frame <br>> > type(GlusterFS 3.1) op(SETATTR(38)) called at 2011-06-10 08:54:14.920644<br>> > [2011-06-10 08:54:14.961880] I <br>> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-0: remote <br>> > operation failed: Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.961880] I <br>> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-0: remote <br>> > operation failed: Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.961940] I [client.c:1883:client_rpc_notify] <br>> > 0-ksc-client-0: disconnected<br>> > [2011-06-10 08:54:14.988468] W [socket.c:204:__socket_rwv] <br>> > 0-ksc-client-1: readv failed (Connection reset by peer)<br>> > [2011-06-10 08:54:14.988490] W <br>> > [socket.c:1494:__socket_proto_state_machine] 0-ksc-client-1: reading <br>> > from socket failed. Error (Connection reset by peer), peer <br>> > (10.0.1.199:24027)<br>> > [2011-06-10 08:54:14.988501] D [socket.c:1768:socket_event_handler] <br>> > 0-transport: disconnecting now<br>> > [2011-06-10 08:54:14.988501] D [socket.c:1768:socket_event_handler] <br>> > 0-transport: disconnecting now<br>> > [2011-06-10 08:54:14.988551] E [rpc-clnt.c:338:saved_frames_unwind] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) <br>> > [0x7f140e8c0acc] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) <br>> > [0x7f140e8c006a] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) <br>> > [0x7f140e8bfb78]))) 0-ksc-client-1: forced unwinding frame <br>> > type(GlusterFS 3.1) op(SETATTR(38)) called at 2011-06-10 08:54:14.920655<br>> > [2011-06-10 08:54:14.988568] I <br>> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-1: remote <br>> > operation failed: Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.988568] I <br>> > [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-ksc-client-1: remote <br>> > operation failed: Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.988599] D [client.c:77:client_submit_request] <br>> > 0-ksc-client-0: connection in disconnected state<br>> > [2011-06-10 08:54:14.988630] W [client3_1-fops.c:4379:client3_1_xattrop] <br>> > 0-ksc-client-0: failed to send the fop: Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.988657] D [name.c:157:client_fill_address_family] <br>> > 0-ksc-client-1: address-family not specified, guessing it to be inet/inet6<br>> > [2011-06-10 08:54:14.991719] D [common-utils.c:151:gf_resolve_ip6] <br>> > 0-resolver: returning ip-10.0.1.199 (port-24007) for hostname: <br>> > 10.0.1.199 and port: 24007<br>> > [2011-06-10 08:54:14.991786] I [socket.c:2272:socket_submit_request] <br>> > 0-ksc-client-1: not connected (priv->connected = 0)<br>> > [2011-06-10 08:54:14.991803] W [rpc-clnt.c:1411:rpc_clnt_submit] <br>> > 0-ksc-client-1: failed to submit rpc-request (XID: 0x70357x Program: <br>> > GlusterFS 3.1, ProgVers: 310, Proc: 33) to rpc-transport (ksc-client-1)<br>> > [2011-06-10 08:54:14.991819] D [afr-lk-common.c:409:transaction_lk_op] <br>> > 0-ksc-replicate-0: lk op is for a transaction<br>> > [2011-06-10 08:54:14.991833] D [client.c:77:client_submit_request] <br>> > 0-ksc-client-0: connection in disconnected state<br>> > [2011-06-10 08:54:14.991845] W [client3_1-fops.c:4735:client3_1_inodelk] <br>> > 0-ksc-client-0: failed to send the fop: Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.991862] W [rpc-clnt.c:1411:rpc_clnt_submit] <br>> > 0-ksc-client-1: failed to submit rpc-request (XID: 0x70358x Program: <br>> > GlusterFS 3.1, ProgVers: 310, Proc: 29) to rpc-transport (ksc-client-1)<br>> > [2011-06-10 08:54:14.991876] I <br>> > [client3_1-fops.c:1226:client3_1_inodelk_cbk] 0-ksc-client-1: remote <br>> > operation failed: Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.991904] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] <br>> > 0-nfs-nfsv3: XID: c71a69ff, CREATE: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected), FH: hashcount 4, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > f3e21f99-0aed-4042-8b81-16a804497b5b<br>> > [2011-06-10 08:54:14.991945] D [client.c:129:client_submit_request] <br>> > 0-ksc-client-1: rpc_clnt_submit failed<br>> > [2011-06-10 08:54:14.991967] D [client.c:129:client_submit_request] <br>> > 0-ksc-client-1: rpc_clnt_submit failed<br>> > [2011-06-10 08:54:14.992014] E [rpc-clnt.c:338:saved_frames_unwind] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) <br>> > [0x7f140e8c0acc] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) <br>> > [0x7f140e8c006a] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) <br>> > [0x7f140e8bfb78]))) 0-ksc-client-1: forced unwinding frame <br>> > type(GlusterFS 3.1) op(RELEASE(41)) called at 2011-06-10 08:54:14.921180<br>> > [2011-06-10 08:54:14.992046] E [rpc-clnt.c:338:saved_frames_unwind] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_notify+0x158) <br>> > [0x7f140e8c0acc] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x101) <br>> > [0x7f140e8c006a] <br>> > (-->/opt/glusterfs/3.2.0/lib64/libgfrpc.so.0(saved_frames_destroy+0x1c) <br>> > [0x7f140e8bfb78]))) 0-ksc-client-1: forced unwinding frame <br>> > type(GlusterFS 3.1) op(STAT(1)) called at 2011-06-10 08:54:14.932952<br>> > [2011-06-10 08:54:14.992060] I [client3_1-fops.c:411:client3_1_stat_cbk] <br>> > 0-ksc-client-1: remote operation failed: Transport endpoint is not <br>> > connected<br>> > [2011-06-10 08:54:14.992073] D [client.c:77:client_submit_request] <br>> > 0-ksc-client-0: connection in disconnected state<br>> > [2011-06-10 08:54:14.992091] W [client3_1-fops.c:2658:client3_1_stat] <br>> > 0-ksc-client-0: failed to send the fop Transport endpoint is not connected<br>> > [2011-06-10 08:54:14.992101] D [afr-inode-read.c:204:afr_stat_cbk] <br>> > 0-ksc-replicate-0: /pimp/htdocs/mountpoints.sh: all subvolumes tried, <br>> > going out<br>> > [2011-06-10 08:54:14.992115] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: fa851fc2, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:14.992181] I [client.c:1883:client_rpc_notify] <br>> > 0-ksc-client-1: disconnected<br>> > [2011-06-10 08:54:14.992197] E [afr-common.c:2546:afr_notify] <br>> > 0-ksc-replicate-0: All subvolumes are down. Going offline until atleast <br>> > one of them comes back up.<br>> > [2011-06-10 08:54:15.5285] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: fb851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.5312] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.5312] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.5344] D [nfs3-helpers.c:2292:nfs3_log_common_call] <br>> > 0-nfs-nfsv3: XID: fb851fc2, GETATTR: args: FH: hashcount 3, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 74c48fb3-d065-462b-83a9-e4558b042465<br>> > [2011-06-10 08:54:15.5381] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /pimp/htdocs/mountpoints.sh: no child is up<br>> > [2011-06-10 08:54:15.5399] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: fb851fc2, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.5783] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: fc851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3<br>> > [2011-06-10 08:54:15.5810] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - LOOKUP<br>> > [2011-06-10 08:54:15.5833] D <br>> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: fc851fc2, <br>> > LOOKUP: args: FH: hashcount 2, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > ff18fc56-c352-4caf-b98d-4eff71494acc, name: mountpoints.sh<br>> > [2011-06-10 08:54:15.5897] D [nfs3.c:1080:nfs3_fresh_lookup] <br>> > 0-nfs-nfsv3: inode needs fresh lookup<br>> > [2011-06-10 08:54:15.5954] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] <br>> > 0-nfs-nfsv3: XID: fc851fc2, LOOKUP: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid <br>> > 00000000-0000-0000-0000-000000000000, gfid <br>> > 00000000-0000-0000-0000-000000000000<br>> > [2011-06-10 08:54:15.11858] E [socket.c:1685:socket_connect_finish] <br>> > 0-ksc-client-1: connection to 10.0.1.199:24027 failed (Connection refused)<br>> > [2011-06-10 08:54:15.11886] D [socket.c:289:__socket_disconnect] <br>> > 0-ksc-client-1: shutdown() returned -1. Transport endpoint is not connected<br>> > [2011-06-10 08:54:15.11921] D [socket.c:193:__socket_rwv] <br>> > 0-ksc-client-1: EOF from peer 10.0.1.199:24027<br>> > [2011-06-10 08:54:15.11938] D <br>> > [socket.c:1494:__socket_proto_state_machine] 0-ksc-client-1: reading <br>> > from socket failed. Error (Transport endpoint is not connected), peer <br>> > (10.0.1.199:24027)<br>> > [2011-06-10 08:54:15.11954] D [socket.c:1768:socket_event_handler] <br>> > 0-transport: disconnecting now<br>> > [2011-06-10 08:54:15.158995] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: e1f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.159038] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.159038] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.159065] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: e1f47a24, <br>> > GETATTR: args: FH: hashcount 6, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 9b62d8e0-cf19-487e-b56a-9ac980294801<br>> > [2011-06-10 08:54:15.159105] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: <br>> > /vertrieb/htdocs/vertrieb_imrich/fetischtopliste.de/www/button.php: no <br>> > child is up<br>> > [2011-06-10 08:54:15.159126] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: e1f47a24, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.159501] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: e2f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3<br>> > [2011-06-10 08:54:15.159528] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - LOOKUP<br>> > [2011-06-10 08:54:15.159550] D <br>> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: e2f47a24, <br>> > LOOKUP: args: FH: hashcount 5, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > dd60aa24-78fa-4509-abee-acf32f12542f, name: button.php<br>> > [2011-06-10 08:54:15.159604] D [nfs3.c:1080:nfs3_fresh_lookup] <br>> > 0-nfs-nfsv3: inode needs fresh lookup<br>> > [2011-06-10 08:54:15.159654] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] <br>> > 0-nfs-nfsv3: XID: e2f47a24, LOOKUP: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid <br>> > 00000000-0000-0000-0000-000000000000, gfid <br>> > 00000000-0000-0000-0000-000000000000<br>> > [2011-06-10 08:54:15.160014] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: e3f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.160041] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.160080] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: e3f47a24, <br>> > GETATTR: args: FH: hashcount 5, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > dd60aa24-78fa-4509-abee-acf32f12542f<br>> > [2011-06-10 08:54:15.160113] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: <br>> > /vertrieb/htdocs/vertrieb_imrich/fetischtopliste.de/www: no child is up<br>> > [2011-06-10 08:54:15.160130] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: e3f47a24, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.160532] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: e4f47a24, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3<br>> > [2011-06-10 08:54:15.160554] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - LOOKUP<br>> > [2011-06-10 08:54:15.160554] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - LOOKUP<br>> > [2011-06-10 08:54:15.160569] D <br>> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: e4f47a24, <br>> > LOOKUP: args: FH: hashcount 4, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 434fc86c-5bed-4a38-b919-8822d38bd1a3, name: www<br>> > [2011-06-10 08:54:15.160611] D [nfs3.c:1080:nfs3_fresh_lookup] <br>> > 0-nfs-nfsv3: inode needs fresh lookup<br>> > [2011-06-10 08:54:15.160657] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] <br>> > 0-nfs-nfsv3: XID: e4f47a24, LOOKUP: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid <br>> > 00000000-0000-0000-0000-000000000000, gfid <br>> > 00000000-0000-0000-0000-000000000000<br>> > [2011-06-10 08:54:15.192962] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: c81a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.193000] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.193019] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: c81a69ff, <br>> > GETATTR: args: FH: hashcount 3, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 61273392-07cb-4427-bb48-2cf869031802<br>> > [2011-06-10 08:54:15.193050] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /tomorrowwinners.com/typo3temp/locks: no child is up<br>> > [2011-06-10 08:54:15.193069] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: c81a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.193526] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: c91a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3<br>> > [2011-06-10 08:54:15.193547] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - LOOKUP<br>> > [2011-06-10 08:54:15.193564] D <br>> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: c91a69ff, <br>> > LOOKUP: args: FH: hashcount 2, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 96012827-9101-40a1-b540-b6c47fe49e59, name: locks<br>> > [2011-06-10 08:54:15.193607] D [nfs3.c:1080:nfs3_fresh_lookup] <br>> > 0-nfs-nfsv3: inode needs fresh lookup<br>> > [2011-06-10 08:54:15.193654] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] <br>> > 0-nfs-nfsv3: XID: c91a69ff, LOOKUP: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid <br>> > 00000000-0000-0000-0000-000000000000, gfid <br>> > 00000000-0000-0000-0000-000000000000<br>> > [2011-06-10 08:54:15.193987] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: ca1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.194015] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.194037] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: ca1a69ff, <br>> > GETATTR: args: FH: hashcount 2, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 96012827-9101-40a1-b540-b6c47fe49e59<br>> > [2011-06-10 08:54:15.194070] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /tomorrowwinners.com/typo3temp: no child is up<br>> > [2011-06-10 08:54:15.194086] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: ca1a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.194566] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: cb1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3<br>> > [2011-06-10 08:54:15.194587] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - LOOKUP<br>> > [2011-06-10 08:54:15.194614] D <br>> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: cb1a69ff, <br>> > LOOKUP: args: FH: hashcount 1, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 7fecc27e-a8de-4fe5-adce-c5e00d35c7ac, name: typo3temp<br>> > [2011-06-10 08:54:15.194659] D [nfs3.c:1080:nfs3_fresh_lookup] <br>> > 0-nfs-nfsv3: inode needs fresh lookup<br>> > [2011-06-10 08:54:15.194703] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] <br>> > 0-nfs-nfsv3: XID: cb1a69ff, LOOKUP: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid <br>> > 00000000-0000-0000-0000-000000000000, gfid <br>> > 00000000-0000-0000-0000-000000000000<br>> > [2011-06-10 08:54:15.396934] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: cc1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.396989] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.396989] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.397017] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: cc1a69ff, <br>> > GETATTR: args: FH: hashcount 1, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 7fecc27e-a8de-4fe5-adce-c5e00d35c7ac<br>> > [2011-06-10 08:54:15.397064] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /tomorrowwinners.com: no child is up<br>> > [2011-06-10 08:54:15.397087] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: cc1a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.397441] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: cd1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 3<br>> > [2011-06-10 08:54:15.397468] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - LOOKUP<br>> > [2011-06-10 08:54:15.397492] D <br>> > [nfs3-helpers.c:2304:nfs3_log_fh_entry_call] 0-nfs-nfsv3: XID: cd1a69ff, <br>> > LOOKUP: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001, name: tomorrowwinners.com<br>> > [2011-06-10 08:54:15.397551] D [nfs3.c:1080:nfs3_fresh_lookup] <br>> > 0-nfs-nfsv3: inode needs fresh lookup<br>> > [2011-06-10 08:54:15.397600] D [nfs3-helpers.c:2477:nfs3_log_newfh_res] <br>> > 0-nfs-nfsv3: XID: cd1a69ff, LOOKUP: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected), FH: hashcount 0, exportid <br>> > 00000000-0000-0000-0000-000000000000, gfid <br>> > 00000000-0000-0000-0000-000000000000<br>> > [2011-06-10 08:54:15.397953] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: ce1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:15.397981] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:15.398004] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: ce1a69ff, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:15.398038] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:15.398055] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: ce1a69ff, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.600726] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: cf1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.600769] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.600769] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.600789] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: cf1a69ff, <br>> > GETATTR: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:15.600857] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: cf1a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.601525] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d01a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:15.601553] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:15.601576] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d01a69ff, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:15.601627] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:15.601646] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d01a69ff, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.725647] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: fd851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:15.725689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:15.725689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:15.725719] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: fd851fc2, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:15.725749] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:15.725769] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: fd851fc2, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.726209] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: fe851fc2, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:15.726236] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:15.726252] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: fe851fc2, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:15.726276] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:15.726292] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: fe851fc2, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.804546] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d11a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:15.804580] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.804580] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:15.804606] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d11a69ff, <br>> > GETATTR: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:15.804658] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d11a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:15.805060] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d21a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:15.805088] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:15.805111] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d21a69ff, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:15.805143] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:15.805159] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d21a69ff, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.8639] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d31a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:16.8698] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:16.20711] D [nfs3-helpers.c:2292:nfs3_log_common_call] <br>> > 0-nfs-nfsv3: XID: d31a69ff, GETATTR: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:16.20821] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d31a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.21517] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d41a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:16.21556] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:16.21575] D [nfs3-helpers.c:2292:nfs3_log_common_call] <br>> > 0-nfs-nfsv3: XID: d41a69ff, ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:16.21606] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:16.21625] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d41a69ff, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.225038] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d51a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:16.225096] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:16.225096] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:16.225118] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d51a69ff, <br>> > GETATTR: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:16.225193] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d51a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.225595] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d61a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:16.225622] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:16.225639] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d61a69ff, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:16.225669] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:16.225688] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d61a69ff, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.243210] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d71a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:16.243232] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:16.243232] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:16.243249] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d71a69ff, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:16.243274] I [afr-inode-read.c:270:afr_stat] <br>> > 0-ksc-replicate-0: /: no child is up<br>> > [2011-06-10 08:54:16.243291] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d71a69ff, ACCESS: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.243670] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d81a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:16.243697] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:16.243717] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d81a69ff, <br>> > GETATTR: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:16.243756] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d81a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.428628] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: d91a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 1<br>> > [2011-06-10 08:54:16.428689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:16.428689] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - GETATTR<br>> > [2011-06-10 08:54:16.428711] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: d91a69ff, <br>> > GETATTR: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > [2011-06-10 08:54:16.428787] D [nfs3-helpers.c:2424:nfs3_log_common_res] <br>> > 0-nfs-nfsv3: XID: d91a69ff, GETATTR: NFS: 5(I/O error), POSIX: <br>> > 107(Transport endpoint is not connected)<br>> > [2011-06-10 08:54:16.429129] D [rpcsvc.c:1940:nfs_rpcsvc_request_create] <br>> > 0-nfsrpc: RPC XID: da1a69ff, Ver: 2, Program: 100003, ProgVers: 3, Proc: 4<br>> > [2011-06-10 08:54:16.429157] D [rpcsvc.c:1357:nfs_rpcsvc_program_actor] <br>> > 0-nfsrpc: Actor found: NFS3 - ACCESS<br>> > [2011-06-10 08:54:16.429182] D <br>> > [nfs3-helpers.c:2292:nfs3_log_common_call] 0-nfs-nfsv3: XID: da1a69ff, <br>> > ACCESS: args: FH: hashcount 0, exportid <br>> > ea50df7c-ff08-4416-8fb3-59d09667cc51, gfid <br>> > 00000000-0000-0000-0000-000000000001<br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > Am 09.06.2011 17:09, schrieb anthony garnier:<br>> >> Hi,<br>> >><br>> >> I got the same problem as Juergen,<br>> >> My volume is a simple replicated volume with 2 host and GlusterFS 3.2.0<br>> >><br>> >> Volume Name: poolsave<br>> >> Type: Replicate<br>> >> Status: Started<br>> >> Number of Bricks: 2<br>> >> Transport-type: tcp<br>> >> Bricks:<br>> >> Brick1: ylal2950:/soft/gluster-data<br>> >> Brick2: ylal2960:/soft/gluster-data<br>> >> Options Reconfigured:<br>> >> diagnostics.brick-log-level: DEBUG<br>> >> network.ping-timeout: 20<br>> >> performance.cache-size: 512MB<br>> >> nfs.port: 2049<br>> >><br>> >> I'm running this command :<br>> >><br>> >> I get those error :<br>> >> tar: ./uvs00: owner not changed<br>> >> tar: could not stat ./uvs00/log/0906uvsGESEC.log<br>> >> tar: ./uvs00: group not changed<br>> >> tar: could not stat ./uvs00/log/0306uvsGESEC.log<br>> >> tar: ./uvs00/log: Input/output error<br>> >> cannot change back?: Unknown error 526<br>> >> tar: ./uvs00/log: owner not changed<br>> >> tar: ./uvs00/log: group not changed<br>> >> tar: tape blocksize error<br>> >><br>> >> And then I tried to "ls" in gluster mount :<br>> >> /bin/ls: .: Input/output error<br>> >><br>> >> only way is to restart the volume<br>> >><br>> >><br>> >> Here is the logfile in Debug mod :<br>> >><br>> >><br>> >> Given volfile:<br>> >> +------------------------------------------------------------------------------+ <br>> >><br>> >> 1: volume poolsave-client-0<br>> >> 2: type protocol/client<br>> >> 3: option remote-host ylal2950<br>> >> 4: option remote-subvolume /soft/gluster-data<br>> >> 5: option transport-type tcp<br>> >> 6: option ping-timeout 20<br>> >> 7: end-volume<br>> >> 8:<br>> >> 9: volume poolsave-client-1<br>> >> 10: type protocol/client<br>> >> 11: option remote-host ylal2960<br>> >> 12: option remote-subvolume /soft/gluster-data<br>> >> 13: option transport-type tcp<br>> >> 14: option ping-timeout 20<br>> >> 15: end-volume<br>> >> 16:<br>> >> 17: volume poolsave-replicate-0<br>> >> 18: type cluster/replicate<br>> >> 19: subvolumes poolsave-client-0 poolsave-client-1<br>> >> 20: end-volume<br>> >> 21:<br>> >> 22: volume poolsave-write-behind<br>> >> 23: type performance/write-behind<br>> >> 24: subvolumes poolsave-replicate-0<br>> >> 25: end-volume<br>> >> 26:<br>> >> 27: volume poolsave-read-ahead<br>> >> 28: type performance/read-ahead<br>> >> 29: subvolumes poolsave-write-behind<br>> >> 30: end-volume<br>> >> 31:<br>> >> 32: volume poolsave-io-cache<br>> >> 33: type performance/io-cache<br>> >> 34: option cache-size 512MB<br>> >> 35: subvolumes poolsave-read-ahead<br>> >> 36: end-volume<br>> >> 37:<br>> >> 38: volume poolsave-quick-read<br>> >> 39: type performance/quick-read<br>> >> 40: option cache-size 512MB<br>> >> 41: subvolumes poolsave-io-cache<br>> >> 42: end-volume<br>> >> 43:<br>> >> 44: volume poolsave-stat-prefetch<br>> >> 45: type performance/stat-prefetch<br>> >> 46: subvolumes poolsave-quick-read<br>> >> 47: end-volume<br>> >> 48:<br>> >> 49: volume poolsave<br>> >> 50: type debug/io-stats<br>> >> 51: option latency-measurement off<br>> >> 52: option count-fop-hits off<br>> >> 53: subvolumes poolsave-stat-prefetch<br>> >> 54: end-volume<br>> >> 55:<br>> >> 56: volume nfs-server<br>> >> 57: type nfs/server<br>> >> 58: option nfs.dynamic-volumes on<br>> >> 59: option rpc-auth.addr.poolsave.allow *<br>> >> 60: option nfs3.poolsave.volume-id 71e0dabf-4620-4b6d-b138-3266096b93b6<br>> >> 61: option nfs.port 2049<br>> >> 62: subvolumes poolsave<br>> >> 63: end-volume<br>> >><br>> >> +------------------------------------------------------------------------------+ <br>> >><br>> >> [2011-06-09 16:52:23.709018] I [rpc-clnt.c:1531:rpc_clnt_reconfig] <br>> >> 0-poolsave-client-0: changing<br>> >> port to 24014 (from 0)<br>> >> [2011-06-09 16:52:23.709211] I [rpc-clnt.c:1531:rpc_clnt_reconfig] <br>> >> 0-poolsave-client-1: changing<br>> >> port to 24011 (from 0)<br>> >> [2011-06-09 16:52:27.716417] I <br>> >> [client-handshake.c:1080:select_server_supported_programs]<br>> >> 0-poolsave-client-0: Using Program GlusterFS-3.1.0, Num (1298437), <br>> >> Version (310)<br>> >> [2011-06-09 16:52:27.716650] I <br>> >> [client-handshake.c:913:client_setvolume_cbk] 0-poolsave-client-0:<br>> >> Connected to 10.68.217.85:24014, attached to remote volume <br>> >> '/soft/gluster-data'.<br>> >> [2011-06-09 16:52:27.716679] I [afr-common.c:2514:afr_notify] <br>> >> 0-poolsave-replicate-0: Subvolume<br>> >> 'poolsave-client-0' came back up; going online.<br>> >> [2011-06-09 16:52:27.717020] I [afr-common.c:836:afr_fresh_lookup_cbk] <br>> >> 0-poolsave-replicate-0: added<br>> >> root inode<br>> >> [2011-06-09 16:52:27.729719] I <br>> >> [client-handshake.c:1080:select_server_supported_programs]<br>> >> 0-poolsave-client-1: Using Program GlusterFS-3.1.0, Num (1298437), <br>> >> Version (310)<br>> >> [2011-06-09 16:52:27.730014] I <br>> >> [client-handshake.c:913:client_setvolume_cbk] 0-poolsave-client-1:<br>> >> Connected to 10.68.217.86:24011, attached to remote volume <br>> >> '/soft/gluster-data'.<br>> >> [2011-06-09 17:01:35.537084] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_mkdir+0x1cc) <br>> >> [0x2aaaab3b88fc]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_mkdir+0x151) <br>> >> [0x2aaaab2948e1]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_mkdir+0xd2) <br>> >><br>> >> [0x2aaaab1856c2]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.546601] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) <br>> >> [0x2aaaab3b95bb]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) <br>> >> [0x2aaaab294ad5]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) <br>> >><br>> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.569755] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.569881] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.579674] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_mkdir+0x1cc) <br>> >> [0x2aaaab3b88fc]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_mkdir+0x151) <br>> >> [0x2aaaab2948e1]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_mkdir+0xd2) <br>> >><br>> >> [0x2aaaab1856c2]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.587907] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) <br>> >> [0x2aaaab3b95bb]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) <br>> >> [0x2aaaab294ad5]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) <br>> >><br>> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.612918] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) <br>> >> [0x2aaaab3b95bb]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) <br>> >> [0x2aaaab294ad5]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) <br>> >><br>> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.645357] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) <br>> >> [0x2aaaab3b95bb]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) <br>> >> [0x2aaaab294ad5]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) <br>> >><br>> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.660873] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.660955] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.665933] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.666057] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.671199] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.671241] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:<br>> >> remote operation failed: Directory not empty<br>> >> [2011-06-09 17:01:35.680959] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) <br>> >> [0x2aaaab3b95bb]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) <br>> >> [0x2aaaab294ad5]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) <br>> >><br>> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.715633] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) <br>> >> [0x2aaaab3b95bb]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) <br>> >> [0x2aaaab294ad5]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) <br>> >><br>> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.732798] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-0:<br>> >> remote operation failed: Permission denied<br>> >> [2011-06-09 17:01:35.733044] I <br>> >> [client3_1-fops.c:547:client3_1_rmdir_cbk] 0-poolsave-client-1:<br>> >> remote operation failed: Permission denied<br>> >> [2011-06-09 17:01:35.750009] W <br>> >> [stat-prefetch.c:178:sp_check_and_create_inode_ctx]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/nfs/server.so(nfs_fop_create+0x1db) <br>> >> [0x2aaaab3b95bb]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/debug/io-stats.so(io_stats_create+0x165) <br>> >> [0x2aaaab294ad5]<br>> >> (-->/usr/local/lib/glusterfs/3.2.0/xlator/performance/stat-prefetch.so(sp_create+0xbc) <br>> >><br>> >> [0x2aaaab185c9c]))) 0-poolsave-stat-prefetch: stat-prefetch context is <br>> >> present in inode (ino:0<br>> >> gfid:00000000-0000-0000-0000-000000000000) when it is supposed to be <br>> >> not present<br>> >> [2011-06-09 17:01:35.784610] W <br>> >> [socket.c:1494:__socket_proto_state_machine] 0-poolsave-client-0:<br>> >> reading from socket failed. Error (Transport endpoint is not <br>> >> connected), peer (10.68.217.85:24014)<br>> >> [2011-06-09 17:01:35.784745] E [rpc-clnt.c:338:saved_frames_unwind]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) <br>> >> [0x2ab58145ef8e]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) <br>> >> [0x2ab58145eefe]))) 0-poolsave-client-0:<br>> >> forced unwinding frame type(GlusterFS 3.1) op(SETATTR(38)) called at <br>> >> 2011-06-09 17:01:35.752080<br>> >> [2011-06-09 17:01:35.784770] I <br>> >> [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-poolsave-client-0:<br>> >> remote operation failed: Transport endpoint is not connected<br>> >> [2011-06-09 17:01:35.784811] E [rpc-clnt.c:338:saved_frames_unwind]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) <br>> >> [0x2ab58145ef8e]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) <br>> >> [0x2ab58145eefe]))) 0-poolsave-client-0:<br>> >> forced unwinding frame type(GlusterFS 3.1) op(STAT(1)) called at <br>> >> 2011-06-09 17:01:35.752414<br>> >> [2011-06-09 17:01:35.784828] I <br>> >> [client3_1-fops.c:411:client3_1_stat_cbk] 0-poolsave-client-0: remote<br>> >> operation failed: Transport endpoint is not connected<br>> >> [2011-06-09 17:01:35.784875] I [client.c:1883:client_rpc_notify] <br>> >> 0-poolsave-client-0: disconnected<br>> >> [2011-06-09 17:01:35.785400] W [socket.c:204:__socket_rwv] <br>> >> 0-poolsave-client-1: readv failed<br>> >> (Connection reset by peer)<br>> >> [2011-06-09 17:01:35.785435] W <br>> >> [socket.c:1494:__socket_proto_state_machine] 0-poolsave-client-1:<br>> >> reading from socket failed. Error (Connection reset by peer), peer <br>> >> (10.68.217.86:24011)<br>> >> [2011-06-09 17:01:35.785496] E [rpc-clnt.c:338:saved_frames_unwind]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) <br>> >> [0x2ab58145ef8e]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) <br>> >> [0x2ab58145eefe]))) 0-poolsave-client-1:<br>> >> forced unwinding frame type(GlusterFS 3.1) op(SETATTR(38)) called at <br>> >> 2011-06-09 17:01:35.752089<br>> >> [2011-06-09 17:01:35.785516] I <br>> >> [client3_1-fops.c:1640:client3_1_setattr_cbk] 0-poolsave-client-1:<br>> >> remote operation failed: Transport endpoint is not connected<br>> >> [2011-06-09 17:01:35.785542] W <br>> >> [client3_1-fops.c:4379:client3_1_xattrop] 0-poolsave-client-0: failed<br>> >> to send the fop: Transport endpoint is not connected<br>> >> [2011-06-09 17:01:35.817662] I [socket.c:2272:socket_submit_request] <br>> >> 0-poolsave-client-1: not<br>> >> connected (priv->connected = 0)<br>> >> [2011-06-09 17:01:35.817698] W [rpc-clnt.c:1411:rpc_clnt_submit] <br>> >> 0-poolsave-client-1: failed to<br>> >> submit rpc-request (XID: 0x576x Program: GlusterFS 3.1, ProgVers: 310, <br>> >> Proc: 33) to rpc-transport<br>> >> (poolsave-client-1)<br>> >> [2011-06-09 17:01:35.817721] W <br>> >> [client3_1-fops.c:4735:client3_1_inodelk] 0-poolsave-client-0: failed<br>> >> to send the fop: Transport endpoint is not connected<br>> >> [2011-06-09 17:01:35.817744] W [rpc-clnt.c:1411:rpc_clnt_submit] <br>> >> 0-poolsave-client-1: failed to<br>> >> submit rpc-request (XID: 0x577x Program: GlusterFS 3.1, ProgVers: 310, <br>> >> Proc: 29) to rpc-transport<br>> >> (poolsave-client-1)<br>> >> [2011-06-09 17:01:35.817780] I <br>> >> [client3_1-fops.c:1226:client3_1_inodelk_cbk] 0-poolsave-client-1:<br>> >> remote operation failed: Transport endpoint is not connected<br>> >> [2011-06-09 17:01:35.817897] E [rpc-clnt.c:338:saved_frames_unwind]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2ab58145f7f9]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) <br>> >> [0x2ab58145ef8e]<br>> >> (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0xe) <br>> >> [0x2ab58145eefe]))) 0-poolsave-client-1:<br>> >> forced unwinding frame type(GlusterFS 3.1) op(STAT(1)) called at <br>> >> 2011-06-09 17:01:35.784870<br>> >> [2011-06-09 17:01:35.817918] I <br>> >> [client3_1-fops.c:411:client3_1_stat_cbk] 0-poolsave-client-1: remote<br>> >> operation failed: Transport endpoint is not connected<br>> >> [2011-06-09 17:01:35.817969] I [client.c:1883:client_rpc_notify] <br>> >> 0-poolsave-client-1: disconnected<br>> >> [2011-06-09 17:01:35.817988] E [afr-common.c:2546:afr_notify] <br>> >> 0-poolsave-replicate-0: All subvolumes<br>> >> are down. Going offline until atleast one of them comes back up.<br>> >> [2011-06-09 17:01:35.818007] E [socket.c:1685:socket_connect_finish] <br>> >> 0-poolsave-client-1: connection<br>> >> to 10.68.217.86:24011 failed (Connection refused)<br>> >> [2011-06-09 17:01:35.818606] I [afr.h:838:AFR_LOCAL_INIT] <br>> >> 0-poolsave-replicate-0: no subvolumes up<br>> >> [2011-06-09 17:01:35.819129] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /uvs00/log:<br>> >> no child is up<br>> >> [2011-06-09 17:01:35.819354] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /uvs00/log:<br>> >> no child is up<br>> >> [2011-06-09 17:01:35.820090] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /uvs00: no<br>> >> child is up<br>> >> [2011-06-09 17:01:35.820760] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.821212] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.821600] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.822123] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.822511] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.822975] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.823286] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.823583] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:35.823857] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:47.518006] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >> [2011-06-09 17:01:49.39204] E [socket.c:1685:socket_connect_finish] <br>> >> 0-poolsave-client-0: connection<br>> >> to 10.68.217.85:24014 failed (Connection refused)<br>> >> [2011-06-09 17:01:49.136932] I [afr-inode-read.c:270:afr_stat] <br>> >> 0-poolsave-replicate-0: /: no child is up<br>> >><br>> >><br>> >><br>> >> > Message: 7<br>> >> > Date: Thu, 9 Jun 2011 12:56:39 +0530<br>> >> > From: Shehjar Tikoo <shehjart@gluster.com><br>> >> > Subject: Re: [Gluster-users] Glusterfs 3.2.0 NFS Problem<br>> >> > To: J?rgen Winkler <juergen.winkler@xidras.com><br>> >> > Cc: gluster-users@gluster.org<br>> >> > Message-ID: <4DF075AF.3040509@gluster.com><br>> >> > Content-Type: text/plain; charset="us-ascii"; format=flowed<br>> >> ><br>> >> > This can happen if all your servers were unreachable for a few <br>> >> seconds. The<br>> >> > situation must have rectified during the restart. We could confirm <br>> >> if you<br>> >> > change the log level on nfs to DEBUG and send us the log.<br>> >> ><br>> >> > Thanks<br>> >> > -Shehjar<br>> >> ><br>> >> > Ju"rgen Winkler wrote:<br>> >> > > Hi,<br>> >> > ><br>> >> > > i noticed a strange behavior with NFS and Glusterfs 3.2.0 , 3 of our<br>> >> > > Servers are loosing the Mount but when you restart the Volume on the<br>> >> > > Server it works again without a remount.<br>> >> > ><br>> >> > > On the server i noticed this entries in the Glusterfs/Nfs <br>> >> log-file when<br>> >> > > the mount on the Client becomes unavailable :<br>> >> > ><br>> >> > > [2011-06-08 14:37:02.568693] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:02.569212] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:02.611910] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:02.624477] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.288272] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.296150] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.309247] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.320939] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.321786] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.333609] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.334089] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.344662] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.352666] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.354195] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.360446] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.369331] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.471556] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:04.480013] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:05.639700] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:05.652535] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.578469] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.588949] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.590395] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.591414] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.591932] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.592596] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.639317] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:07.652919] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.332435] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.340622] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.349360] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.349550] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.360445] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.369497] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.369752] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.382097] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > > [2011-06-08 14:37:09.382387] I [afr-inode-read.c:270:afr_stat]<br>> >> > > 0-ksc-replicate-0: /: no child is up<br>> >> > ><br>> >> > ><br>> >> > > Thx for the help<br>> >> > ><br>> >> > > _______________________________________________<br>> >> > > Gluster-users mailing list<br>> >> > > Gluster-users@gluster.org<br>> >> > > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users<br>> >> ><br>> >> ><br>> >> ><br>> >> > ------------------------------<br>> >> ><br>> >> > _______________________________________________<br>> >> > Gluster-users mailing list<br>> >> > Gluster-users@gluster.org<br>> >> > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users<br>> >> ><br>> >> ><br>> >> > End of Gluster-users Digest, Vol 38, Issue 14<br>> >> > *********************************************<br>> >><br>> >><br>> >><br>> >> _______________________________________________<br>> >> Gluster-users mailing list<br>> >> Gluster-users@gluster.org<br>> >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users<br>> > <br>> <br> </body>
</html>