[2012-03-08 23:31:29.878233] I [glusterfsd.c:1493:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.2.5 [2012-03-08 23:31:29.880715] I [glusterd.c:550:init] 0-management: Using /etc/glusterd as working directory [2012-03-08 23:31:29.881808] E [rpc-transport.c:677:rpc_transport_load] 0-rpc-transport: /usr/lib/glusterfs/3.2.5/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2012-03-08 23:31:29.881826] E [rpc-transport.c:681:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2012-03-08 23:31:29.881841] W [rpcsvc.c:1288:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2012-03-08 23:31:29.881906] I [glusterd.c:96:glusterd_uuid_init] 0-glusterd: generated UUID: e8714f53-448c-4189-adb6-d4c9e3b34183 Given volfile: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option working-directory /etc/glusterd 4: option transport-type socket,rdma 5: option transport.socket.keepalive-time 10 6: option transport.socket.keepalive-interval 2 7: end-volume 8: +------------------------------------------------------------------------------+ [2012-03-08 23:31:31.872867] I [glusterd-handler.c:796:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-03-08 23:31:31.875445] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1023) [2012-03-08 23:33:25.480973] W [glusterfsd.c:727:cleanup_and_exit] (-->/lib/libc.so.6(clone+0x6d) [0x7ff917a4170d] (-->/lib/libpthread.so.0(+0x69ca) [0x7ff917ce49ca] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x406335]))) 0-: received signum (15), shutting down [2012-03-09 00:07:27.676562] I [glusterfsd.c:1493:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.2.5 [2012-03-09 00:07:27.691472] I [glusterd.c:550:init] 0-management: Using /etc/glusterd as working directory [2012-03-09 00:07:27.693383] E [rpc-transport.c:677:rpc_transport_load] 0-rpc-transport: /usr/lib/glusterfs/3.2.5/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2012-03-09 00:07:27.693402] E [rpc-transport.c:681:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2012-03-09 00:07:27.693417] W [rpcsvc.c:1288:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2012-03-09 00:07:27.695500] I [glusterd.c:88:glusterd_uuid_init] 0-glusterd: retrieved UUID: e8714f53-448c-4189-adb6-d4c9e3b34183 Given volfile: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option working-directory /etc/glusterd 4: option transport-type socket,rdma 5: option transport.socket.keepalive-time 10 6: option transport.socket.keepalive-interval 2 7: end-volume 8: +------------------------------------------------------------------------------+ [2012-03-09 00:07:30.952678] I [glusterd-handler.c:900:glusterd_handle_create_volume] 0-glusterd: Received create volume req [2012-03-09 00:07:31.17822] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by e8714f53-448c-4189-adb6-d4c9e3b34183 [2012-03-09 00:07:31.17882] I [glusterd-handler.c:420:glusterd_op_txn_begin] 0-glusterd: Acquired local lock [2012-03-09 00:07:31.21133] I [glusterd-op-sm.c:6737:glusterd_op_ac_send_stage_op] 0-glusterd: Sent op req to 0 peers [2012-03-09 00:07:31.39033] I [glusterd-op-sm.c:6854:glusterd_op_ac_send_commit_op] 0-glusterd: Sent op req to 0 peers [2012-03-09 00:07:31.39091] I [glusterd-op-sm.c:7250:glusterd_op_txn_complete] 0-glusterd: Cleared local lock [2012-03-09 00:07:31.41629] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1023) [2012-03-09 00:07:47.123979] I [glusterd-handler.c:1079:glusterd_handle_cli_start_volume] 0-glusterd: Received start vol reqfor volume west [2012-03-09 00:07:47.124043] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by e8714f53-448c-4189-adb6-d4c9e3b34183 [2012-03-09 00:07:47.124057] I [glusterd-handler.c:420:glusterd_op_txn_begin] 0-glusterd: Acquired local lock [2012-03-09 00:07:47.127594] I [glusterd-op-sm.c:6737:glusterd_op_ac_send_stage_op] 0-glusterd: Sent op req to 0 peers [2012-03-09 00:07:47.628870] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0001 [2012-03-09 00:07:47.677762] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0002 [2012-03-09 00:07:47.725290] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0003 [2012-03-09 00:07:47.823985] I [glusterd-op-sm.c:6854:glusterd_op_ac_send_commit_op] 0-glusterd: Sent op req to 0 peers [2012-03-09 00:07:47.824061] I [glusterd-op-sm.c:7250:glusterd_op_txn_complete] 0-glusterd: Cleared local lock [2012-03-09 00:07:47.827256] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1022) [2012-03-09 00:07:47.905874] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0001 on port 24009 [2012-03-09 00:07:47.907210] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0002 on port 24010 [2012-03-09 00:07:47.908276] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0003 on port 24011 [2012-03-09 00:07:47.941663] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1018) [2012-03-09 00:07:47.941758] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1017) [2012-03-09 00:07:47.941832] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1016) [2012-03-09 00:07:52.492884] I [glusterd-handler.c:796:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-03-09 00:07:52.493270] I [glusterd-handler.c:796:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-03-09 00:07:52.494616] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1015) [2012-03-09 00:15:47.964431] W [glusterfsd.c:727:cleanup_and_exit] (-->/lib/libc.so.6(clone+0x6d) [0x7f3b9306570d] (-->/lib/libpthread.so.0(+0x69ca) [0x7f3b933089ca] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x406335]))) 0-: received signum (15), shutting down [2012-03-09 00:15:49.990062] I [glusterfsd.c:1493:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.2.5 [2012-03-09 00:15:49.992503] I [glusterd.c:550:init] 0-management: Using /etc/glusterd as working directory [2012-03-09 00:15:49.993589] E [rpc-transport.c:677:rpc_transport_load] 0-rpc-transport: /usr/lib/glusterfs/3.2.5/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2012-03-09 00:15:49.993607] E [rpc-transport.c:681:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2012-03-09 00:15:49.993622] W [rpcsvc.c:1288:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2012-03-09 00:15:49.993733] I [glusterd.c:88:glusterd_uuid_init] 0-glusterd: retrieved UUID: e8714f53-448c-4189-adb6-d4c9e3b34183 [2012-03-09 00:15:51.2985] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-0 [2012-03-09 00:15:51.3062] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-1 [2012-03-09 00:15:51.3078] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-2 [2012-03-09 00:15:51.521007] I [glusterd-utils.c:1067:glusterd_volume_start_glusterfs] 0-: brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0001 already started [2012-03-09 00:15:51.525144] I [glusterd-utils.c:1067:glusterd_volume_start_glusterfs] 0-: brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0002 already started [2012-03-09 00:15:51.527791] I [glusterd-utils.c:1067:glusterd_volume_start_glusterfs] 0-: brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0003 already started [2012-03-09 00:15:51.531914] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster nfsd running in pid: 1126 [2012-03-09 00:15:52.540387] I [glusterd-utils.c:2297:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully [2012-03-09 00:15:52.540563] I [glusterd-utils.c:2302:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully [2012-03-09 00:15:52.540714] I [glusterd-utils.c:2307:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully Given volfile: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option working-directory /etc/glusterd 4: option transport-type socket,rdma 5: option transport.socket.keepalive-time 10 6: option transport.socket.keepalive-interval 2 7: end-volume 8: +------------------------------------------------------------------------------+ [2012-03-09 00:15:52.586770] I [glusterd-handler.c:796:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-03-09 00:15:52.587236] I [glusterd-handler.c:796:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-03-09 00:15:52.589535] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1020) [2012-03-09 00:15:52.624954] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1019) [2012-03-09 00:15:52.625057] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1018) [2012-03-09 00:15:52.625131] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1017) [2012-03-09 00:15:55.651244] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1020) [2012-03-09 00:15:55.651819] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1016) [2012-03-09 00:15:55.652309] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1015) [2012-03-09 00:15:58.270573] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0003 on port 24011 [2012-03-09 00:15:58.280852] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0001 on port 24009 [2012-03-09 00:15:58.281273] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0002 on port 24010 [2012-03-09 00:15:58.680697] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1014) [2012-03-09 00:15:58.681227] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1013) [2012-03-09 00:15:58.681681] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1012) [2012-03-09 00:24:39.74807] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1022) [2012-03-09 00:24:39.75301] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1021) [2012-03-09 00:24:39.76215] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1020) [2012-03-09 00:28:16.571113] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1023) [2012-03-09 00:28:24.426274] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1021) [2012-03-09 00:28:24.427012] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1020) [2012-03-09 00:28:24.428113] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1016) [2012-03-09 15:19:09.715196] I [glusterd-handler.c:1306:glusterd_handle_add_brick] 0-glusterd: Received add brick req [2012-03-09 15:19:09.718196] I [glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by e8714f53-448c-4189-adb6-d4c9e3b34183 [2012-03-09 15:19:09.718248] I [glusterd-handler.c:420:glusterd_op_txn_begin] 0-glusterd: Acquired local lock [2012-03-09 15:19:09.718309] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0004 [2012-03-09 15:19:09.720163] I [glusterd-op-sm.c:6737:glusterd_op_ac_send_stage_op] 0-glusterd: Sent op req to 0 peers [2012-03-09 15:19:09.722542] I [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0004 [2012-03-09 15:19:09.723521] I [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick [2012-03-09 15:19:09.723598] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0004 [2012-03-09 15:19:09.825503] I [glusterd-utils.c:909:glusterd_service_stop] 0-: Stopping gluster nfsd running in pid: 1287 [2012-03-09 15:19:10.837002] I [glusterd-utils.c:2297:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully [2012-03-09 15:19:10.837156] I [glusterd-utils.c:2302:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully [2012-03-09 15:19:10.837279] I [glusterd-utils.c:2307:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully [2012-03-09 15:19:10.884939] I [glusterd-op-sm.c:6854:glusterd_op_ac_send_commit_op] 0-glusterd: Sent op req to 0 peers [2012-03-09 15:19:10.885007] I [glusterd-op-sm.c:7250:glusterd_op_txn_complete] 0-glusterd: Cleared local lock [2012-03-09 15:19:10.888448] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1020) [2012-03-09 15:19:10.913455] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1023) [2012-03-09 15:19:10.913899] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1021) [2012-03-09 15:19:10.914054] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1020) [2012-03-09 15:19:10.914918] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1019) [2012-03-09 15:19:10.998323] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0004 on port 24012 [2012-03-09 15:19:11.64712] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1018) [2012-03-09 15:19:11.64808] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1017) [2012-03-09 15:19:11.64886] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1016) [2012-03-09 15:19:11.64966] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1015) [2012-03-09 15:19:13.974025] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1011) [2012-03-09 15:19:15.109117] I [glusterd-handler.c:796:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-03-09 15:19:15.109534] I [glusterd-handler.c:796:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-03-09 15:19:15.111936] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1014) [2012-03-09 15:19:22.706881] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 1 on west [2012-03-09 15:19:22.814582] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1013) [2012-03-09 15:19:26.727415] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:19:26.729886] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1012) [2012-03-09 15:19:41.318240] W [glusterfsd.c:727:cleanup_and_exit] (-->/lib/libc.so.6(clone+0x6d) [0x7f2d9b39370d] (-->/lib/libpthread.so.0(+0x69ca) [0x7f2d9b6369ca] (-->/usr/sbin/glusterd(glusterfs_sigwaiter+0xd5) [0x406335]))) 0-: received signum (15), shutting down [2012-03-09 15:20:10.305095] I [glusterfsd.c:1493:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.2.5 [2012-03-09 15:20:10.326616] I [glusterd.c:550:init] 0-management: Using /etc/glusterd as working directory [2012-03-09 15:20:10.327916] E [rpc-transport.c:677:rpc_transport_load] 0-rpc-transport: /usr/lib/glusterfs/3.2.5/rpc-transport/rdma.so: cannot open shared object file: No such file or directory [2012-03-09 15:20:10.327937] E [rpc-transport.c:681:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2012-03-09 15:20:10.327952] W [rpcsvc.c:1288:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed [2012-03-09 15:20:10.339647] I [glusterd.c:88:glusterd_uuid_init] 0-glusterd: retrieved UUID: e8714f53-448c-4189-adb6-d4c9e3b34183 [2012-03-09 15:20:11.513852] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-0 [2012-03-09 15:20:11.513907] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-1 [2012-03-09 15:20:11.513923] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-2 [2012-03-09 15:20:11.513938] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-3 [2012-03-09 15:20:12.51535] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0001 [2012-03-09 15:20:12.104954] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0002 [2012-03-09 15:20:12.159152] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0003 [2012-03-09 15:20:12.213728] I [glusterd-utils.c:1097:glusterd_volume_start_glusterfs] 0-: About to start glusterfs for brick ip-10-160-150-128.us-west-1.compute.internal:/gluster/brick0004 Given volfile: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option working-directory /etc/glusterd 4: option transport-type socket,rdma 5: option transport.socket.keepalive-time 10 6: option transport.socket.keepalive-interval 2 7: end-volume 8: +------------------------------------------------------------------------------+ [2012-03-09 15:20:12.344797] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0004 on port 24012 [2012-03-09 15:20:12.346542] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0003 on port 24011 [2012-03-09 15:20:12.353532] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0001 on port 24009 [2012-03-09 15:20:12.355212] I [glusterd-pmap.c:237:pmap_registry_bind] 0-pmap: adding brick /gluster/brick0002 on port 24010 [2012-03-09 15:20:12.378314] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1019) [2012-03-09 15:20:12.378408] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1018) [2012-03-09 15:20:12.378483] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1017) [2012-03-09 15:20:12.378556] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.160.150.128:1016) [2012-03-09 15:20:23.111162] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:20:23.113198] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1015) [2012-03-09 15:20:25.720671] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 1 on west [2012-03-09 15:20:25.783197] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1014) [2012-03-09 15:20:28.28231] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:20:28.30312] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1013) [2012-03-09 15:21:40.486673] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:21:40.489193] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1019) [2012-03-09 15:21:42.612064] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 1 on west [2012-03-09 15:21:42.614313] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1018) [2012-03-09 15:21:47.224203] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:21:47.226182] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1017) [2012-03-09 15:21:48.741984] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:21:48.744064] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1016) [2012-03-09 15:23:36.531703] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.170.83.13:1023) [2012-03-09 15:23:39.679954] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:23:39.682055] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1019) [2012-03-09 15:23:41.680289] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 1 on west [2012-03-09 15:23:41.682393] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1018) [2012-03-09 15:23:43.482375] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 3 on west [2012-03-09 15:23:43.484220] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1017) [2012-03-09 15:23:46.634563] I [glusterd-rebalance.c:616:glusterd_rebalance_cmd_attempted_log] 0-glusterd: Received rebalance volume 2 on west [2012-03-09 15:23:46.638583] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1016)