<html><head></head><body>To kill a zombie process, you have to kill the parent process.<br>
<br>
ps -p 23744 -o ppid=<br>
<br>
If the result is 1, then you are stuck rebooting. Otherwise, kill that process.<br>
<br>
Deleting a filename does not close the named pipe, so that caused the failure below.<br><br><div class="gmail_quote">Joel Young <jdy@cryregarder.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre class="k9mail">On Tue, Jul 30, 2013 at 10:49 PM, Kaushal M <kshlmster@gmail.com> wrote:<br /><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">I think I've found the problem. The problem is not with the brick port, but instead with<br />the unix domain socket used for communication between glusterd and glusterfsd.</blockquote><br />Makes sense.<br /><br /><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;">So this is most likely due the zombie process 23744 still listening on the unix<br />domain socket. Only one bind can be performed on a unix domain socket. If<br />another bind is tried we get an EADDRINUSE error.<br /><br />Can you kill 23744, remove /var/run/5a538b707ce5dbf525ba6d01835863bb.socket<br />and restart the brick using 'gluster volume start'. This should allow it to start.</blockquote><br />It isn't possible to kill 23744 as
it is zombie. fuser on the socket<br />doesn't report any<br />users. I did remove /var/run/5a53...<br /><br />"gluster volume start home" doesn't work as the volume is already<br />started (and mounted<br />and in use by users so I'd rather not shutdown the cluster). I tried a<br />"systemctl restart glusterd.service" which did not restart the brick<br />but did leave the following<br />in /var/log/bricks/lhome-gluster_home.log:<br /><br />[2013-07-31 16:04:59.716771] I [glusterfsd.c:1910:main]<br />0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version<br />3.4.0 (/usr/sbin/glusterfsd -s ir2 --volfile-id<br />home.ir2.lhome-gluster_home -p<br />/var/lib/glusterd/vols/home/run/ir2-lhome-gluster_home.pid -S<br />/var/run/5a538b707ce5dbf525ba6d01835863bb.socket --brick-name<br />/lhome/gluster_home -l<br />/var/log/glusterfs/bricks/lhome-gluster_home.log --xlator-option<br />*-posix.glusterd-uuid=9d2d74bf-9055-47a6-b3df-8c2057ea1dd9<br />--brick-port 49157
--xlator-option home-server.listen-port=49157)<br />[2013-07-31 16:04:59.719901] I [socket.c:3480:socket_init]<br />0-socket.glusterfsd: SSL support is NOT enabled<br />[2013-07-31 16:04:59.719936] I [socket.c:3495:socket_init]<br />0-socket.glusterfsd: using system polling thread<br />[2013-07-31 16:04:59.720242] I [socket.c:3480:socket_init]<br />0-glusterfs: SSL support is NOT enabled<br />[2013-07-31 16:04:59.720256] I [socket.c:3495:socket_init]<br />0-glusterfs: using system polling thread<br />[2013-07-31 16:04:59.752491] I [graph.c:239:gf_add_cmdline_options]<br />0-home-server: adding option 'listen-port' for volume 'home-server'<br />with value '49157'<br />[2013-07-31 16:04:59.752514] I [graph.c:239:gf_add_cmdline_options]<br />0-home-posix: adding option 'glusterd-uuid' for volume 'home-posix'<br />with value '9d2d74bf-9055-47a6-b3df-8c2057ea1dd9'<br />[2013-07-31 16:04:59.753960] W [options.c:848:xl_opt_validate]<br />0-home-server: option 'listen-port' is deprecated,
preferred is<br />'transport.socket.listen-port', continuing with correction<br />[2013-07-31 16:04:59.754000] I [socket.c:3480:socket_init]<br />0-tcp.home-server: SSL support is NOT enabled<br />[2013-07-31 16:04:59.754025] I [socket.c:3495:socket_init]<br />0-tcp.home-server: using system polling thread<br />[2013-07-31 16:04:59.754075] E [socket.c:695:__socket_server_bind]<br />0-tcp.home-server: binding to failed: Address already in use<br />[2013-07-31 16:04:59.754091] E [socket.c:698:__socket_server_bind]<br />0-tcp.home-server: Port is already in use<br />[2013-07-31 16:04:59.754108] W [rpcsvc.c:1394:rpcsvc_transport_create]<br />0-rpc-service: listening on transport failed<br />[2013-07-31 16:04:59.754128] W [server.c:1092:init] 0-home-server:<br />creation of listener failed<br />[2013-07-31 16:04:59.754140] E [xlator.c:390:xlator_init]<br />0-home-server: Initialization of volume 'home-server' failed, review<br />your volfile again<br />[2013-07-31 16:04:59.754151] E
[graph.c:292:glusterfs_graph_init]<br />0-home-server: initializing translator failed<br />[2013-07-31 16:04:59.754162] E [graph.c:479:glusterfs_graph_activate]<br />0-graph: init failed<br />[2013-07-31 16:04:59.754404] W [glusterfsd.c:1002:cleanup_and_exit]<br />(-->/lib64/<a href="http://libgfrpc.so">libgfrpc.so</a>.0(rpc_clnt_handle_reply+0x90) [0x7f5794b5db10]<br />(-->/usr/sbin/glusterfsd(mgmt_getspec_cbk+0x2fd) [0x7f5795216bcd]<br />(-->/usr/sbin/glusterfsd(glusterfs_process_volfp+0x103)<br />[0x7f5795212603]))) 0-: received signum (0), shutting down<br /><br /><br />Which seems like it worked and then tried again and failed?<br /><br />Thanks!<br /><br />Joel<br /><hr /><br />Gluster-users mailing list<br />Gluster-users@gluster.org<br /><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br /></pre></blockquote></div></body></html>