<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 01/09/2013 11:03 AM, Song wrote:<br>
</div>
<blockquote cite="mid:006601cdee2a$e7638ab0$b62aa010$@com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:宋体;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@宋体";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We have a glusterfs
clusters, version is 3.2.7. The volume info is as below:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Volume Name: gfs1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Type:
Distributed-Replicate<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Status: Started<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Number of Bricks: 94 x 3
= 282<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Transport-type: tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We native mount the
volume in all cluster servers. When we access the file
“/XMTEXT/gfs1_000/000/000/095” on one server, the error is
split brain.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">While we can access the
same file on another server.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">At the same time, after
re-mount the volume at error server, access the same file is
ok. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The glusterfs has cached
some information? This case has happened more than one.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The log is as following
when split brain.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:29.554505] W
[afr-common.c:931:afr_detect_self_heal_by_lookup_status]
0-gfs1-replicate-5: split brain detected during lookup of
/XMTEXT/gfs1_000/000/000/095.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:29.554566] I [afr-common.c:1039:afr_launch_self_heal]
0-gfs1-replicate-5: background data gfid self-heal
triggered. path: /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:29.555299] I
[afr-self-heal-common.c:1290:sh_missing_entries_create]
0-gfs1-replicate-5: no missing files -
/XMTEXT/gfs1_000/000/000/095. proceeding to metadata check<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:29.555507] I
[afr-self-heal-common.c:1050:afr_sh_missing_entries_done]
0-gfs1-replicate-5: split brain found, aborting selfheal of
/XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:29.555531] E
[afr-self-heal-common.c:2190:afr_self_heal_completion_cbk]
0-gfs1-replicate-5: background data gfid self-heal failed
on /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:35.598229] W
[afr-common.c:931:afr_detect_self_heal_by_lookup_status]
0-gfs1-replicate-5: split brain detected during lookup of
/XMTEXT/gfs1_000/000/000/095.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:35.598282] I [afr-common.c:1039:afr_launch_self_heal]
0-gfs1-replicate-5: background data gfid self-heal
triggered. path: /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:35.598939] I
[afr-self-heal-common.c:1290:sh_missing_entries_create]
0-gfs1-replicate-5: no missing files -
/XMTEXT/gfs1_000/000/000/095. proceeding to metadata check<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:35.599139] I
[afr-self-heal-common.c:1050:afr_sh_missing_entries_done]
0-gfs1-replicate-5: split brain found, aborting selfheal of
/XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:35.599176] E
[afr-self-heal-common.c:2190:afr_self_heal_completion_cbk]
0-gfs1-replicate-5: background data gfid self-heal failed
on /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:38.192819] W
[afr-common.c:931:afr_detect_self_heal_by_lookup_status]
0-gfs1-replicate-5: split brain detected during lookup of
/XMTEXT/gfs1_000/000/000/095.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:38.192875] I [afr-common.c:1039:afr_launch_self_heal]
0-gfs1-replicate-5: background data gfid self-heal
triggered. path: /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:38.193486] I
[afr-self-heal-common.c:1290:sh_missing_entries_create]
0-gfs1-replicate-5: no missing files -
/XMTEXT/gfs1_000/000/000/095. proceeding to metadata check<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:38.193708] I
[afr-self-heal-common.c:1050:afr_sh_missing_entries_done]
0-gfs1-replicate-5: split brain found, aborting selfheal of
/XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:38.193731] E
[afr-self-heal-common.c:2190:afr_self_heal_completion_cbk]
0-gfs1-replicate-5: background data gfid self-heal failed
on /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:38.193937] W [afr-open.c:168:afr_open]
0-gfs1-replicate-5: failed to open as split brain seen,
returning EIO<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
09:57:38.194033] W [fuse-bridge.c:693:fuse_fd_cbk]
0-glusterfs-fuse: 3162527: OPEN()
/XMTEXT/gfs1_000/000/000/095 => -1 (Input/output error)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
10:08:12.569821] W
[afr-common.c:931:afr_detect_self_heal_by_lookup_status]
0-gfs1-replicate-5: split brain detected during lookup of
/XMTEXT/gfs1_000/000/000/095.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
10:08:12.569891] I [afr-common.c:1039:afr_launch_self_heal]
0-gfs1-replicate-5: background data gfid self-heal
triggered. path: /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
10:08:12.571538] I
[afr-self-heal-common.c:1290:sh_missing_entries_create]
0-gfs1-replicate-5: no missing files -
/XMTEXT/gfs1_000/000/000/095. proceeding to metadata check<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
10:08:12.572684] I
[afr-self-heal-common.c:1050:afr_sh_missing_entries_done]
0-gfs1-replicate-5: split brain found, aborting selfheal of
/XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
10:08:12.572732] E
[afr-self-heal-common.c:2190:afr_self_heal_completion_cbk]
0-gfs1-replicate-5: background data gfid self-heal failed
on /XMTEXT/gfs1_000/000/000/095<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
10:08:12.580006] W [afr-open.c:168:afr_open]
0-gfs1-replicate-5: failed to open as split brain seen,
returning EIO<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-07
10:08:12.580103] W [fuse-bridge.c:693:fuse_fd_cbk]
0-glusterfs-fuse: 3164490: OPEN()
/XMTEXT/gfs1_000/000/000/095 => -1 (Input/output error)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Thanks!<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a>
<a class="moz-txt-link-freetext" href="https://lists.nongnu.org/mailman/listinfo/gluster-devel">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
Song,<br>
It seems like the file is in gfid-split-brain. To confirm,
could you provide the output of following command from backends.<br>
getfattr -d -m . -e hex <file-in-split-brain><br>
<br>
Pranith.<br>
</body>
</html>