<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<div class="moz-cite-prefix">On 09/11/2014 01:13 PM, Ilya Ivanov
wrote:<br>
</div>
<blockquote
cite="mid:CAA=KoeJo3FgzwKVi3C1d9W2HpFwfjZLd-5o_VSjOm-cc2v37UQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>Makes some sense. Yes, I meant "make a backup and
delete", rather than just delete.<br>
<br>
</div>
If I may suggest, putting that debug link somewhere more
visible would be be good, too. I wouldn't find without your
help.<br>
</div>
</div>
</blockquote>
Justin, where shall we put the doc?<br>
<br>
Pranith<br>
<blockquote
cite="mid:CAA=KoeJo3FgzwKVi3C1d9W2HpFwfjZLd-5o_VSjOm-cc2v37UQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div><br>
</div>
Thank you for the assistance.<br>
<br>
<div>
<div><br>
<br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep 11, 2014 at 9:14 AM,
Pranith Kumar Karampuri <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:pkarampu@redhat.com"
target="_blank">pkarampu@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span class=""> <br>
<div>On 09/11/2014 11:37 AM, Ilya Ivanov wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">I don't understand why there's such a
complicated process to recover when I can just look
at both files, decide which one I need and delete
another one.<br>
</div>
</blockquote>
</span> If the file needs to be deleted the whole file
needs to be copied which is fine for small files but for
big files like VM images it takes less time if the file
already exists and it syncs only the parts of files that
are different from the good copy.<br>
One more reason is if the parent directory from which the
file is deleted from is the source then self-heal will
delete the file from other directory rather than creating
it. SO instead of deleting the file may be it is a better
practise to make a copy of the file somewhere and delete
it. We shall update the document as well with this new
information. Thanks for the feedback. In 3.7 it is going
to be simplified. We are giving a command to fix the
split-brains where the user gets to choose the file and it
will do the rest.
<div>
<div class="h5"><br>
<br>
Pranith<br>
<blockquote type="cite">
<div dir="ltr"><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep 11, 2014 at
7:56 AM, Pranith Kumar Karampuri <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:pkarampu@redhat.com"
target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span> <br>
<div>On 09/11/2014 09:29 AM, Ilya Ivanov
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Right... I deleted it and now all
appears to be fine.<br>
<br>
</div>
Still, could you please elaborate on
"gfid split-brain"?<br>
</div>
</blockquote>
</span> Could you go through <a
moz-do-not-send="true"
href="https://github.com/gluster/glusterfs/blob/master/doc/debugging/split-brain.md"
target="_blank">https://github.com/gluster/glusterfs/blob/master/doc/debugging/split-brain.md</a><br>
Let us know if you would like something to
be more clearer and we can add that and
improve the document.<span><font
color="#888888"><br>
<br>
Pranith</font></span>
<div>
<div><br>
<blockquote type="cite">
<div dir="ltr"><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep
11, 2014 at 5:32 AM, Pranith Kumar
Karampuri <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:pkarampu@redhat.com"
target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF"> <br>
<div>On 09/11/2014 12:16 AM,
Ilya Ivanov wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Any insight?<br>
</div>
</blockquote>
Was the other file's gfid <span>d3def9e1-c6d0-4b7d-a322-b5019305182e?<br>
Could you check if this file
exists in
<brick>/.glusterfs/d3/de/<br>
When a file is deleted this
file also needs to be
deleted if there are no more
hardlinks to the file<span><font
color="#888888"><br>
<br>
Pranith<br>
</font></span></span>
<div>
<div>
<blockquote type="cite">
<div class="gmail_extra"><br>
<div
class="gmail_quote">On
Tue, Sep 9, 2014 at
8:35 AM, Ilya Ivanov
<span dir="ltr"><<a
moz-do-not-send="true" href="mailto:bearwere@gmail.com" target="_blank">bearwere@gmail.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0
0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">What's
a gfid
split-brain and
how is it
different from
"normal"
split-brain?<br>
<div><br>
</div>
<div>I accessed
the file with
stat, but
"heal info"
still shows
"Number of
entries: 1"<br>
</div>
<div><br>
[root@gluster1
gluster]#
getfattr -d
-m. -e hex
gv01/123<br>
# getfattr -d
-m. -e hex
gv01/123<br>
# file:
gv01/123<br>
trusted.afr.gv01-client-0=0x000000000000000000000000<br>
trusted.afr.gv01-client-1=0x000000000000000000000000<br>
trusted.gfid=0x35f86f4561134ba0bd1b94ef70179d4d<br>
<br>
[root@gluster1
gluster]#
getfattr -d
-m. -e hex
gv01<br>
# file: gv01<br>
trusted.afr.gv01-client-0=0x000000000000000000000000<br>
trusted.afr.gv01-client-1=0x000000000000000000000000<br>
trusted.gfid=0x00000000000000000000000000000001<br>
trusted.glusterfs.dht=0x000000010000000000000000ffffffff<br>
trusted.glusterfs.volume-id=0x31a2c4c486ca4344b838d2c2e6c716c1<br>
<br>
<br>
</div>
</div>
<div
class="gmail_extra">
<div>
<div><br>
<div
class="gmail_quote">On
Tue, Sep 9,
2014 at 8:19
AM, Pranith
Kumar
Karampuri <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
text="#000000"
bgcolor="#FFFFFF"><span> <br>
<div>On
09/09/2014
11:35 AM, Ilya
Ivanov wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">
<div>Ahh,
thank you, now
I get it. I
deleted it on
one node and
it replicated
to another
one. Now I get
the following
output:<br>
<br>
[root@gluster1
var]# gluster
volume heal
gv01 info<br>
Brick
gluster1:/home/gluster/gv01/<br>
<gfid:d3def9e1-c6d0-4b7d-a322-b5019305182e><br>
Number of
entries: 1<br>
<br>
Brick
gluster2:/home/gluster/gv01/<br>
Number of
entries: 0<br>
<br>
</div>
<div>Is it
normal? Why
the number of
entries isn't
reset to 0?<br>
</div>
</div>
</blockquote>
</span> If you
access the
file using
ls/stat etc,
it will be
fixed. But
before that
could you
please post
the output of
'getfattr -d
-m. -e hex
file/path/in/backend/brick'
and 'getfattr
-d -m. -e hex
parent/dir/to/file/path/in/backend/brick'<br>
<br>
Pranith<span><br>
<blockquote
type="cite">
<div dir="ltr"><br>
<br>
And why
wouldn't the
file show up
in split-brain
before,
anyway?<br>
</div>
</blockquote>
</span> Gfid
split-brains
are not shown
in
heal-info-split-brain
yet.<span><font
color="#888888"><br>
<br>
Pranith</font></span><span><br>
<blockquote
type="cite">
<div dir="ltr"><br>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Tue, Sep 9,
2014 at 7:46
AM, Pranith
Kumar
Karampuri <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
text="#000000"
bgcolor="#FFFFFF"><span> <br>
<div>On
09/09/2014
01:54 AM, Ilya
Ivanov wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">
<div>
<div>Hello.<br>
<br>
</div>
I've Gluster
3.5.2 on
Centos 6. A
primitive
replicated
volume, as
describe <a
moz-do-not-send="true"
href="https://www.digitalocean.com/community/tutorials/how-to-create-a-redundant-storage-pool-using-glusterfs-on-ubuntu-servers"
target="_blank">here</a>. I tried to simulate split-brain by temporarily
disconnecting
the nodes and
creating a
file with the
same name and
different
contents. That
worked.<br>
<br>
</div>
<div>The
question is,
how do I fix
it now? All
the tutorials
suggest
deleting the
file from one
of the nodes.
I can't do
that, it
reports
"Input/output
error". The
file won't
even show up
in "gluster
volume heal
gv00 info
split-brain".
That shows 0
entries.<br>
</div>
</div>
</blockquote>
</span> The
deletion needs
to happen on
one of the
bricks, not
from the mount
point.<br>
<br>
Pranith<br>
<blockquote
type="cite"><span>
<div dir="ltr">
<div>I can see
the file in
"gluster
volume heal
gv00 info
heal-failed",
though.<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</span>
<pre>_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
<br
clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</span></div>
</blockquote>
</div>
<br>
<br
clear="all">
<br>
</div>
</div>
<span><font
color="#888888">--
<br>
<div dir="ltr">Ilya.<br>
</div>
</font></span></div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>