<html><body><div style="color:#000; background-color:#fff; font-family:Courier New, courier, monaco, monospace, sans-serif;font-size:13px"><div class="" style="">Hi everyone,</div><div class="" style=""><br class="" style=""></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: 'Courier New', courier, monaco, monospace, sans-serif; font-style: normal; background-color: transparent;" class="">I have this really confusing split-brain here that's bothering me. I am running glusterfs 3.4.2 over linux 2.6.34. I have a replica 2 volume 'testvol' that is It seems I cannot read/stat/edit the file in question, and `gluster volume heal testvol info split-brain` shows nothing. Here are the logs from the fuse-mount for the volume:</div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: 'Courier New', courier, monaco, monospace, sans-serif; font-style: normal; background-color: transparent;" class=""><br class="" style=""></div><div
style="color: rgb(0, 0, 0); font-size: 13px; font-family: 'Courier New', courier, monaco, monospace, sans-serif; font-style: normal; background-color: transparent;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:53:02.867111] W [fuse-bridge.c:1172:fuse_err_cbk] 0-glusterfs-fuse: 4560969: FLUSH() ERR => -1 (Input/output error) </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.007799] W [page.c:991:__ioc_page_error] 0-testvol-io-cache: page error for page = 0x7fd5c8529d20 & waitq = 0x7fd5c8067d40 </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.007854] W [fuse-bridge.c:2089:fuse_readv_cbk] 0-glusterfs-fuse: 4561103: READ => -1
(Input/output error) </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.008018] W [page.c:991:__ioc_page_error] 0-testvol-io-cache: page error for page = 0x7fd5c8607ee0 & waitq = 0x7fd5c8067d40 </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.008056] W [fuse-bridge.c:2089:fuse_readv_cbk] 0-glusterfs-fuse: 4561104: READ => -1 (Input/output error) </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.008233] W [page.c:991:__ioc_page_error] 0-testvol-io-cache: page error for page = 0x7fd5c8066f30 & waitq = 0x7fd5c8067d40 </font><br style="font-family:
Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.008269] W [fuse-bridge.c:2089:fuse_readv_cbk] 0-glusterfs-fuse: 4561105: READ => -1 (Input/output error) </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.008800] W [page.c:991:__ioc_page_error] 0-testvol-io-cache: page error for page = 0x7fd5c860bcf0 & waitq = 0x7fd5c863b1f0 </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.008839] W [fuse-bridge.c:2089:fuse_readv_cbk] 0-glusterfs-fuse: 4561107: READ => -1 (Input/output error) </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family:
Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.009365] W [page.c:991:__ioc_page_error] 0-testvol-io-cache: page error for page = 0x7fd5c85fd120 & waitq = 0x7fd5c8067d40 </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.009413] W [fuse-bridge.c:2089:fuse_readv_cbk] 0-glusterfs-fuse: 4561109: READ => -1 (Input/output error) </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.040549] W [afr-open.c:213:afr_open] 0-testvol-replicate-0: failed to open as split brain seen, returning EIO </font><br style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><font style="font-family: Verdana, Arial, Helvetica, sans-serif;" class="">[2014-09-29 07:54:16.040594] W
[fuse-bridge.c:915:fuse_fd_cbk] 0-glusterfs-fuse: 4561142: OPEN() /SECLOG/20140908.d/SECLOG_00000000000000427425_00000000000000000000.log => -1 (Input/output error)</font><br class="" style=""></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: 'Courier New', courier, monaco, monospace, sans-serif; font-style: normal; background-color: transparent;" class=""><br class="" style=""></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-style: normal; background-color: transparent;" class=""><span style="font-family: verdana, helvetica, sans-serif;">Could somebody please give me some clue on where to begin? I checked the xattrs on <span class="">/SECLOG/20140908.d/SECLOG_00000000000000427425_00000000000000000000.log and it seems the changelogs are [0, 0] on both replicas, and the gfid's match.</span></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-style: normal; font-family: verdana, helvetica, sans-serif;
background-color: transparent;" class=""><span style="font-family: verdana, helvetica, sans-serif;"><span class=""><br></span></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-style: normal; font-family: verdana, helvetica, sans-serif; background-color: transparent;" class=""><span style="font-family: verdana, helvetica, sans-serif;"><span class="">Thank you very much for any help on this.</span></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-style: normal; font-family: verdana, helvetica, sans-serif; background-color: transparent;" class=""><span style="font-family: verdana, helvetica, sans-serif;"><span class="">Anirban</span></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: Verdana, Arial, Helvetica, sans-serif; font-style: normal; background-color: transparent;" class=""><span style="font-family: Verdana, Arial, Helvetica, sans-serif;" class=""><br></span></div><div style="color: rgb(0,
0, 0); font-size: 13px; font-family: 'Courier New', courier, monaco, monospace, sans-serif; font-style: normal; background-color: transparent;" class=""><br class="" style=""></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: 'Courier New', courier, monaco, monospace, sans-serif; font-style: normal; background-color: transparent;" class=""><br class="" style=""></div></div></body></html>