<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div><font color="#11053b">I am seeing this crashing happening, I am working on the self healing errors as well, not sure if the two are related. I would appreciate any direction on trying to resolve the issue, I have clients dropping connection daily. </font></div><div><font color="#11053b"><br></font></div><div><font color="#11053b"><br></font></div><div><font color="#11053b"><br></font></div><div><font color="#11053b">[2013-10-26 15:35:46.935903] E [afr-self-heal-common.c:2160:afr_self_heal_completion_cbk] 0-ENTV04EP-replicate-9: background meta-data self-heal failed on /</font></div><div><font color="#11053b">[2013-10-26 15:35:47.944225] E [afr-self-heal-metadata.c:472:afr_sh_metadata_fix] 0-ENTV04EP-replicate-9: Unable to self-heal permissions/ownership of '/' (possible split-brain). Please fix the file on all backend volumes</font></div><div><font color="#11053b">[2013-10-26 15:35:47.946969] E [afr-self-heal-common.c:2160:afr_self_heal_completion_cbk] 0-ENTV04EP-replicate-9: background meta-data self-heal failed on /</font></div><div><font color="#11053b">[2013-10-26 15:35:48.952501] E [afr-self-heal-metadata.c:472:afr_sh_metadata_fix] 0-ENTV04EP-replicate-9: Unable to self-heal permissions/ownership of '/' (possible split-brain). Please fix the file on all backend volumes</font></div><div><font color="#11053b">[2013-10-26 15:35:48.952685] E [afr-self-heal-common.c:2160:afr_self_heal_completion_cbk] 0-ENTV04EP-replicate-9: background meta-data self-heal failed on /</font></div><div><font color="#11053b">[2013-10-26 15:35:51.503966] E [afr-self-heal-metadata.c:472:afr_sh_metadata_fix] 0-ENTV04EP-replicate-9: Unable to self-heal permissions/ownership of '/' (possible split-brain). Please fix the file on all backend volumes</font></div><div><font color="#11053b">[2013-10-26 15:35:51.504217] E [afr-self-heal-common.c:2160:afr_self_heal_completion_cbk] 0-ENTV04EP-replicate-9: background meta-data self-heal failed on /</font></div><div><font color="#11053b">[2013-10-26 15:35:52.077008] W [client3_1-fops.c:1183:client3_1_fgetxattr_cbk] 0-ENTV04EP-client-17: remote operation failed: No data available</font></div><div><font color="#11053b">pending frames:</font></div><div><font color="#11053b">frame : type(1) op(STAT)</font></div><div><font color="#11053b">frame : type(1) op(STAT)</font></div><div><font color="#11053b">frame : type(1) op(READDIR)</font></div><div><font color="#11053b">frame : type(1) op(LOOKUP)</font></div><div><font color="#11053b">frame : type(1) op(LOOKUP)</font></div><div><font color="#11053b">frame : type(1) op(LOOKUP)</font></div><div><font color="#11053b">frame : type(1) op(LOOKUP)</font></div><div><font color="#11053b">frame : type(1) op(LOOKUP)</font></div><div><font color="#11053b">frame : type(1) op(LOOKUP)</font></div><div><font color="#11053b"><br></font></div><div><font color="#11053b">patchset: <a href="git://git.gluster.com/glusterfs.git">git://git.gluster.com/glusterfs.git</a></font></div><div><font color="#11053b">signal received: 11</font></div><div><font color="#11053b">time of crash: 2013-10-26 15:35:52configuration details:</font></div><div><font color="#11053b">argp 1</font></div><div><font color="#11053b">backtrace 1</font></div><div><font color="#11053b">dlfcn 1</font></div><div><font color="#11053b">fdatasync 1</font></div><div><font color="#11053b">libpthread 1</font></div><div><font color="#11053b">llistxattr 1</font></div><div><font color="#11053b">setfsid 1</font></div><div><font color="#11053b">spinlock 1</font></div><div><font color="#11053b">epoll.h 1</font></div><div><font color="#11053b">xattr.h 1</font></div><div><font color="#11053b">st_atim.tv_nsec 1</font></div><div><font color="#11053b">package-string: glusterfs 3.3.2</font></div><div><font color="#11053b">/lib64/libc.so.6[0x3d86432960]</font></div><div><font color="#11053b">/usr/lib64/glusterfs/3.3.2/xlator/cluster/distribute.so(dht_migration_complete_check_task+0x495)[0x7f7d7a58d915]</font></div><div><font color="#11053b">/usr/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x3d86c4bb52]</font></div><div><font color="#11053b">/lib64/libc.so.6[0x3d86443bb0]</font></div><div><font color="#11053b">---------</font></div><div><font color="#11053b">[root@ENTSNV04010EP glusterfs]# tail -n 100 mnt-ENTV04EP.log | fpaste</font></div><div><font color="#11053b">Uploading (15.7KiB)...</font></div><div><font color="#11053b"><a href="http://ur1.ca/fxyly">http://ur1.ca/fxyly</a> -> <a href="http://paste.fedoraproject.org/49648/38280209">http://paste.fedoraproject.org/49648/38280209</a></font></div><div><font color="#11053b">[root@ENTSNV04010EP glusterfs]# </font></div><div><br></div></body></html>