<div dir="ltr"><div><div>Makes some sense. Yes, I meant "make a backup and delete", rather than just delete.<br><br></div>If I may suggest, putting that debug link somewhere more visible would be be good, too. I wouldn't find without your help.<br><br></div>Thank you for the assistance.<br><br><div><div><br><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 11, 2014 at 9:14 AM, Pranith Kumar Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span class="">
<br>
<div>On 09/11/2014 11:37 AM, Ilya Ivanov
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">I don't understand why there's such a complicated
process to recover when I can just look at both files, decide
which one I need and delete another one.<br>
</div>
</blockquote></span>
If the file needs to be deleted the whole file needs to be copied
which is fine for small files but for big files like VM images it
takes less time if the file already exists and it syncs only the
parts of files that are different from the good copy.<br>
One more reason is if the parent directory from which the file is
deleted from is the source then self-heal will delete the file from
other directory rather than creating it. SO instead of deleting the
file may be it is a better practise to make a copy of the file
somewhere and delete it. We shall update the document as well with
this new information. Thanks for the feedback. In 3.7 it is going to
be simplified. We are giving a command to fix the split-brains where
the user gets to choose the file and it will do the rest.<div><div class="h5"><br>
<br>
Pranith<br>
<blockquote type="cite">
<div dir="ltr"><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep 11, 2014 at 7:56 AM,
Pranith Kumar Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span> <br>
<div>On 09/11/2014 09:29 AM, Ilya Ivanov wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Right... I deleted it and now all appears to be
fine.<br>
<br>
</div>
Still, could you please elaborate on "gfid
split-brain"?<br>
</div>
</blockquote>
</span> Could you go through
<a href="https://github.com/gluster/glusterfs/blob/master/doc/debugging/split-brain.md" target="_blank">https://github.com/gluster/glusterfs/blob/master/doc/debugging/split-brain.md</a><br>
Let us know if you would like something to be more clearer
and we can add that and improve the document.<span><font color="#888888"><br>
<br>
Pranith</font></span>
<div>
<div><br>
<blockquote type="cite">
<div dir="ltr"><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep 11, 2014 at
5:32 AM, Pranith Kumar Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<div>On 09/11/2014 12:16 AM, Ilya Ivanov
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Any insight?<br>
</div>
</blockquote>
Was the other file's gfid <span>d3def9e1-c6d0-4b7d-a322-b5019305182e?<br>
Could you check if this file exists in
<brick>/.glusterfs/d3/de/<br>
When a file is deleted this file also
needs to be deleted if there are no more
hardlinks to the file<span><font color="#888888"><br>
<br>
Pranith<br>
</font></span></span>
<div>
<div>
<blockquote type="cite">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Sep
9, 2014 at 8:35 AM, Ilya Ivanov <span dir="ltr"><<a href="mailto:bearwere@gmail.com" target="_blank">bearwere@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">What's a gfid
split-brain and how is it
different from "normal"
split-brain?<br>
<div><br>
</div>
<div>I accessed the file with
stat, but "heal info" still
shows "Number of entries: 1"<br>
</div>
<div><br>
[root@gluster1 gluster]#
getfattr -d -m. -e hex
gv01/123<br>
# getfattr -d -m. -e hex
gv01/123<br>
# file: gv01/123<br>
trusted.afr.gv01-client-0=0x000000000000000000000000<br>
trusted.afr.gv01-client-1=0x000000000000000000000000<br>
trusted.gfid=0x35f86f4561134ba0bd1b94ef70179d4d<br>
<br>
[root@gluster1 gluster]#
getfattr -d -m. -e hex gv01<br>
# file: gv01<br>
trusted.afr.gv01-client-0=0x000000000000000000000000<br>
trusted.afr.gv01-client-1=0x000000000000000000000000<br>
trusted.gfid=0x00000000000000000000000000000001<br>
trusted.glusterfs.dht=0x000000010000000000000000ffffffff<br>
trusted.glusterfs.volume-id=0x31a2c4c486ca4344b838d2c2e6c716c1<br>
<br>
<br>
</div>
</div>
<div class="gmail_extra">
<div>
<div><br>
<div class="gmail_quote">On
Tue, Sep 9, 2014 at 8:19
AM, Pranith Kumar
Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span>
<br>
<div>On 09/09/2014
11:35 AM, Ilya
Ivanov wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Ahh,
thank you, now
I get it. I
deleted it on
one node and
it replicated
to another
one. Now I get
the following
output:<br>
<br>
[root@gluster1
var]# gluster
volume heal
gv01 info<br>
Brick
gluster1:/home/gluster/gv01/<br>
<gfid:d3def9e1-c6d0-4b7d-a322-b5019305182e><br>
Number of
entries: 1<br>
<br>
Brick
gluster2:/home/gluster/gv01/<br>
Number of
entries: 0<br>
<br>
</div>
<div>Is it
normal? Why
the number of
entries isn't
reset to 0?<br>
</div>
</div>
</blockquote>
</span> If you
access the file
using ls/stat etc,
it will be fixed.
But before that
could you please
post the output of
'getfattr -d -m. -e
hex
file/path/in/backend/brick'
and 'getfattr -d -m.
-e hex
parent/dir/to/file/path/in/backend/brick'<br>
<br>
Pranith<span><br>
<blockquote type="cite">
<div dir="ltr"><br>
<br>
And why
wouldn't the
file show up
in split-brain
before,
anyway?<br>
</div>
</blockquote>
</span> Gfid
split-brains are not
shown in
heal-info-split-brain
yet.<span><font color="#888888"><br>
<br>
Pranith</font></span><span><br>
<blockquote type="cite">
<div dir="ltr"><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Tue, Sep 9,
2014 at 7:46
AM, Pranith
Kumar
Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span> <br>
<div>On
09/09/2014
01:54 AM, Ilya
Ivanov wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Hello.<br>
<br>
</div>
I've Gluster
3.5.2 on
Centos 6. A
primitive
replicated
volume, as
describe <a href="https://www.digitalocean.com/community/tutorials/how-to-create-a-redundant-storage-pool-using-glusterfs-on-ubuntu-servers" target="_blank">here</a>. I tried to simulate split-brain by temporarily
disconnecting
the nodes and
creating a
file with the
same name and
different
contents. That
worked.<br>
<br>
</div>
<div>The
question is,
how do I fix
it now? All
the tutorials
suggest
deleting the
file from one
of the nodes.
I can't do
that, it
reports
"Input/output
error". The
file won't
even show up
in "gluster
volume heal
gv00 info
split-brain".
That shows 0
entries.<br>
</div>
</div>
</blockquote>
</span> The
deletion needs
to happen on
one of the
bricks, not
from the mount
point.<br>
<br>
Pranith<br>
<blockquote type="cite"><span>
<div dir="ltr">
<div>I can see
the file in
"gluster
volume heal
gv00 info
heal-failed",
though.<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</span>
<pre>_______________________________________________
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</span></div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
</div>
</div>
<span><font color="#888888">--
<br>
<div dir="ltr">Ilya.<br>
</div>
</font></span></div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Ilya.<br>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br><br clear="all"><br>-- <br><div dir="ltr">Ilya.<br><div></div><div></div><div></div><div></div><div></div><div></div></div>
</div>