<div dir="ltr">I don&#39;t understand why there&#39;s such a complicated process to recover when I can just look at both files, decide which one I need and delete another one.<br><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 11, 2014 at 7:56 AM, Pranith Kumar Karampuri <span dir="ltr">&lt;<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF"><span class="">
    <br>
    <div>On 09/11/2014 09:29 AM, Ilya Ivanov
      wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">
        <div>Right... I deleted it and now all appears to be fine.<br>
          <br>
        </div>
        Still, could you please elaborate on &quot;gfid split-brain&quot;?<br>
      </div>
    </blockquote></span>
    Could you go through
<a href="https://github.com/gluster/glusterfs/blob/master/doc/debugging/split-brain.md" target="_blank">https://github.com/gluster/glusterfs/blob/master/doc/debugging/split-brain.md</a><br>
    Let us know if you would like something to be more clearer and we
    can add that and improve the document.<span class="HOEnZb"><font color="#888888"><br>
    <br>
    Pranith</font></span><div><div class="h5"><br>
    <blockquote type="cite">
      <div dir="ltr"><br>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Thu, Sep 11, 2014 at 5:32 AM,
          Pranith Kumar Karampuri <span dir="ltr">&lt;<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>&gt;</span> wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF"> <br>
              <div>On 09/11/2014 12:16 AM, Ilya Ivanov wrote:<br>
              </div>
              <blockquote type="cite">
                <div dir="ltr">Any insight?<br>
                </div>
              </blockquote>
              Was the other file&#39;s gfid <span>d3def9e1-c6d0-4b7d-a322-b5019305182e?<br>
                Could you check if this file exists in
                &lt;brick&gt;/.glusterfs/d3/de/<br>
                When a file is deleted this file also needs to be
                deleted if there are no more hardlinks to the file<span><font color="#888888"><br>
                    <br>
                    Pranith<br>
                  </font></span></span>
              <div>
                <div>
                  <blockquote type="cite">
                    <div class="gmail_extra"><br>
                      <div class="gmail_quote">On Tue, Sep 9, 2014 at
                        8:35 AM, Ilya Ivanov <span dir="ltr">&lt;<a href="mailto:bearwere@gmail.com" target="_blank">bearwere@gmail.com</a>&gt;</span>
                        wrote:<br>
                        <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                          <div dir="ltr">What&#39;s a gfid split-brain and
                            how is it different from &quot;normal&quot;
                            split-brain?<br>
                            <div><br>
                            </div>
                            <div>I accessed the file with stat, but
                              &quot;heal info&quot; still shows &quot;Number of
                              entries: 1&quot;<br>
                            </div>
                            <div><br>
                              [root@gluster1 gluster]# getfattr -d -m.
                              -e hex gv01/123<br>
                              # getfattr -d -m. -e hex gv01/123<br>
                              # file: gv01/123<br>
trusted.afr.gv01-client-0=0x000000000000000000000000<br>
trusted.afr.gv01-client-1=0x000000000000000000000000<br>
trusted.gfid=0x35f86f4561134ba0bd1b94ef70179d4d<br>
                              <br>
                              [root@gluster1 gluster]# getfattr -d -m.
                              -e hex gv01<br>
                              # file: gv01<br>
trusted.afr.gv01-client-0=0x000000000000000000000000<br>
trusted.afr.gv01-client-1=0x000000000000000000000000<br>
trusted.gfid=0x00000000000000000000000000000001<br>
trusted.glusterfs.dht=0x000000010000000000000000ffffffff<br>
trusted.glusterfs.volume-id=0x31a2c4c486ca4344b838d2c2e6c716c1<br>
                              <br>
                              <br>
                            </div>
                          </div>
                          <div class="gmail_extra">
                            <div>
                              <div><br>
                                <div class="gmail_quote">On Tue, Sep 9,
                                  2014 at 8:19 AM, Pranith Kumar
                                  Karampuri <span dir="ltr">&lt;<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>&gt;</span>
                                  wrote:<br>
                                  <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                    <div text="#000000" bgcolor="#FFFFFF"><span> <br>
                                        <div>On 09/09/2014 11:35 AM,
                                          Ilya Ivanov wrote:<br>
                                        </div>
                                        <blockquote type="cite">
                                          <div dir="ltr">
                                            <div>Ahh, thank you, now I
                                              get it. I deleted it on
                                              one node and it replicated
                                              to another one. Now I get
                                              the following output:<br>
                                              <br>
                                              [root@gluster1 var]#
                                              gluster volume heal gv01
                                              info<br>
                                              Brick
                                              gluster1:/home/gluster/gv01/<br>
&lt;gfid:d3def9e1-c6d0-4b7d-a322-b5019305182e&gt;<br>
                                              Number of entries: 1<br>
                                              <br>
                                              Brick
                                              gluster2:/home/gluster/gv01/<br>
                                              Number of entries: 0<br>
                                              <br>
                                            </div>
                                            <div>Is it normal? Why the
                                              number of entries isn&#39;t
                                              reset to 0?<br>
                                            </div>
                                          </div>
                                        </blockquote>
                                      </span> If you access the file
                                      using ls/stat etc, it will be
                                      fixed. But before that could you
                                      please post the output of
                                      &#39;getfattr -d -m. -e hex
                                      file/path/in/backend/brick&#39; and
                                      &#39;getfattr -d -m. -e hex
                                      parent/dir/to/file/path/in/backend/brick&#39;<br>
                                      <br>
                                      Pranith<span><br>
                                        <blockquote type="cite">
                                          <div dir="ltr"><br>
                                            <br>
                                            And why wouldn&#39;t the file
                                            show up in split-brain
                                            before, anyway?<br>
                                          </div>
                                        </blockquote>
                                      </span> Gfid split-brains are not
                                      shown in heal-info-split-brain
                                      yet.<span><font color="#888888"><br>
                                          <br>
                                          Pranith</font></span><span><br>
                                        <blockquote type="cite">
                                          <div dir="ltr"><br>
                                          </div>
                                          <div class="gmail_extra"><br>
                                            <div class="gmail_quote">On
                                              Tue, Sep 9, 2014 at 7:46
                                              AM, Pranith Kumar
                                              Karampuri <span dir="ltr">&lt;<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>&gt;</span>
                                              wrote:<br>
                                              <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                <div text="#000000" bgcolor="#FFFFFF"><span>
                                                    <br>
                                                    <div>On 09/09/2014
                                                      01:54 AM, Ilya
                                                      Ivanov wrote:<br>
                                                    </div>
                                                    <blockquote type="cite">
                                                      <div dir="ltr">
                                                        <div>
                                                          <div>Hello.<br>
                                                          <br>
                                                          </div>
                                                          I&#39;ve Gluster
                                                          3.5.2 on
                                                          Centos 6. A
                                                          primitive
                                                          replicated
                                                          volume, as
                                                          describe <a href="https://www.digitalocean.com/community/tutorials/how-to-create-a-redundant-storage-pool-using-glusterfs-on-ubuntu-servers" target="_blank">here</a>. I tried to simulate split-brain by temporarily
                                                          disconnecting
                                                          the nodes and
                                                          creating a
                                                          file with the
                                                          same name and
                                                          different
                                                          contents. That
                                                          worked.<br>
                                                          <br>
                                                        </div>
                                                        <div>The
                                                          question is,
                                                          how do I fix
                                                          it now? All
                                                          the tutorials
                                                          suggest
                                                          deleting the
                                                          file from one
                                                          of the nodes.
                                                          I can&#39;t do
                                                          that, it
                                                          reports
                                                          &quot;Input/output
                                                          error&quot;. The
                                                          file won&#39;t
                                                          even show up
                                                          in &quot;gluster
                                                          volume heal
                                                          gv00 info
                                                          split-brain&quot;.
                                                          That shows 0
                                                          entries.<br>
                                                        </div>
                                                      </div>
                                                    </blockquote>
                                                  </span> The deletion
                                                  needs to happen on one
                                                  of the bricks, not
                                                  from the mount point.<br>
                                                  <br>
                                                  Pranith<br>
                                                  <blockquote type="cite"><span>
                                                      <div dir="ltr">
                                                        <div>I can see
                                                          the file in
                                                          &quot;gluster
                                                          volume heal
                                                          gv00 info
                                                          heal-failed&quot;,
                                                          though.<br>
                                                        </div>
                                                        <div><br>
                                                        </div>
                                                        <div><br>
                                                        </div>
                                                        <div>
                                                          <div>-- <br>
                                                          <div dir="ltr">Ilya.<br>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </div>
                                                      <br>
                                                      <fieldset></fieldset>
                                                      <br>
                                                    </span>
                                                    <pre>_______________________________________________
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
                                                  </blockquote>
                                                  <br>
                                                </div>
                                              </blockquote>
                                            </div>
                                            <br>
                                            <br clear="all">
                                            <br>
                                            -- <br>
                                            <div dir="ltr">Ilya.<br>
                                            </div>
                                          </div>
                                        </blockquote>
                                        <br>
                                      </span></div>
                                  </blockquote>
                                </div>
                                <br>
                                <br clear="all">
                                <br>
                              </div>
                            </div>
                            <span><font color="#888888">-- <br>
                                <div dir="ltr">Ilya.<br>
                                </div>
                              </font></span></div>
                        </blockquote>
                      </div>
                      <br>
                      <br clear="all">
                      <br>
                      -- <br>
                      <div dir="ltr">Ilya.<br>
                      </div>
                    </div>
                  </blockquote>
                  <br>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
        <br clear="all">
        <br>
        -- <br>
        <div dir="ltr">Ilya.<br>
        </div>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br><br clear="all"><br>-- <br><div dir="ltr">Ilya.<br><div></div><div></div><div></div><div></div><div></div><div></div></div>
</div>