<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Courrier New";}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Arial","sans-serif";
color:black;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hi<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I have a problem with glusterd 3.2.7. I have two nodes each configured with one brick in a replicate configuration. Both have been running for 9 months. One
node recently started to crash with kernel core dump. The issue was a corrupt ext4 filesystem on the glusterfs brick. I unmounted the volume and fixed the filesystem. Now I have problems starting glusterd:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:33.526714] I [glusterfsd.c:1493:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">2.7<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:33.538144] I [glusterd.c:550:init] 0-management: Using /var/lib/glusterd as working directory<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:33.539148] E [rpc-transport.c:677:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.2.7/rpc-tra<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">nsport/rdma.so: cannot open shared object file: No such file or directory<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:33.539175] E [rpc-transport.c:681:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transpor<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">t-type 'rdma' is not valid or not found on this machine<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:33.539183] W [rpcsvc.c:1288:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the t<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">ransport failed<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:33.539263] I [glusterd.c:88:glusterd_uuid_init] 0-glusterd: retrieved UUID: 8b6e691c-e4c1-4107-944f-fee9c<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">fb55269<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:34.854729] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-0<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:34.854778] E [glusterd-store.c:1820:glusterd_store_retrieve_volume] 0-: Unknown key: brick-1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:35.104082] I [glusterd-handler.c:3409:glusterd_friend_add] 0-glusterd: connect returned 0<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:35.107779] E [common-utils.c:125:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Name or service not know<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">n)<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">[2014-06-19 10:20:35.107843] E [name.c:253:af_inet_client_get_remote_sockaddr] 0-management: DNS resolution failed on host<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">all<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">pending frames:<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New""><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">patchset: git://git.gluster.com/glusterfs.git<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">signal received: 11<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">time of crash: 2014-06-19 10:20:35<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">configuration details:<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">argp 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">backtrace 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">dlfcn 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">fdatasync 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">libpthread 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">llistxattr 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">setfsid 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">spinlock 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">epoll.h 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">xattr.h 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">st_atim.tv_nsec 1<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">package-string: glusterfs 3.2.7<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/lib64/libc.so.6(+0x32920)[0x7f10fe1ff920]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/glusterfs/3.2.7/xlator/mgmt/glusterd.so(glusterd_friend_sm+0x25)[0x7f10fc761055]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/glusterfs/3.2.7/xlator/mgmt/glusterd.so(glusterd_peer_rpc_notify+0x1d6)[0x7f10fc749026]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/libgfrpc.so.0(rpc_clnt_start+0x17)[0x7f10feba38b7]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/glusterfs/3.2.7/xlator/mgmt/glusterd.so(glusterd_rpc_create+0x108)[0x7f10fc749f18]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/glusterfs/3.2.7/xlator/mgmt/glusterd.so(glusterd_friend_add+0x33c)[0x7f10fc74a30c]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/glusterfs/3.2.7/xlator/mgmt/glusterd.so(glusterd_store_retrieve_peers+0x4d6)[0x7f10fc7932f6]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/glusterfs/3.2.7/xlator/mgmt/glusterd.so(glusterd_restore+0x78)[0x7f10fc794cb8]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/glusterfs/3.2.7/xlator/mgmt/glusterd.so(init+0xbec)[0x7f10fc74884c]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/libglusterfs.so.0(xlator_init+0x52)[0x7f10fedc4af2]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/libglusterfs.so.0(glusterfs_graph_init+0x31)[0x7f10fedf57a1]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/lib64/libglusterfs.so.0(glusterfs_graph_activate+0x88)[0x7f10fedf5938]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/sbin/glusterd(glusterfs_process_volfp+0xd5)[0x404b35]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/sbin/glusterd(glusterfs_volumes_init+0x145)[0x404d45]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/sbin/glusterd(main+0x295)[0x405e85]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f10fe1ebcdd]<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt;font-family:"Courrier New"">/usr/sbin/glusterd[0x4036c9]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Any pointers on solving this would be helpful.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<font face="arial" size="1" color="#686868">
<br />
<br />
<hr />
PLEASE CONSIDER THE ENVIRONMENT BEFORE PRINTING THIS E-MAIL.<br />
Close Brothers Limited (Company No. 195626) and Close Brothers Group plc (Company No. 520241) are registered in England and Wales, and their registered office is 10 Crown Place, London EC2A 4FT.<br />
Close Brothers Limited is authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and Prudential Regulation Authority. Firm reference number 124750.
<hr />
The information in this e-mail and any files transmitted with it is confidential and may be legally privileged. It is intended solely for the addressee and others authorised to receive it. If you are not the intended recipient, any disclosure, copying, distribution or action taken in reliance on its contents is prohibited and may be unlawful.
<hr />
</font>
</body>
</html>