<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 11/16/2013 01:42 AM, Anand Avati
wrote:<br>
</div>
<blockquote
cite="mid:CAFboF2wZneo=nov=aBsP3257s-+43PfFDGS1rm1-_vkaaLHEPg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>Ravi,</div>
<div>We should not mix up data and entry operation domains, if a
file is in data split brain that should not stop a user from
rename/link/unlink operations on the file.</div>
<div><br>
</div>
<div>
Regarding your concern about complications while healing - we
should change our "manual fixing" instructions to:</div>
<div><br>
</div>
<div>- go to backend, access through gfid path or normal path</div>
<div>- rmxattr the afr changelogs</div>
<div>- truncate the file to 0 bytes (like "> filename")</div>
<div><br>
</div>
<div>Accessing the path through gfid and truncating to 0 bytes
addresses your concerns about hardlinks/renames.</div>
<div><br>
</div>
<div>Avati</div>
</div>
<div class="gmail_extra"><br>
</div>
</blockquote>
<meta http-equiv="CONTENT-TYPE" content="text/html;
charset=ISO-8859-1">
<p class="western" style="margin-bottom: 0in">All,<br>
<br>
I
have tabulated what operations must/ musn't be permitted in case
of
different split brains. Some of the columns are '?' as I am not
sure
what the expected behaviour should be. Could we have this
validated? <br>
</p>
<p class="western" style="margin-bottom: 0in"><br>
</p>
<meta http-equiv="CONTENT-TYPE" content="text/html;
charset=ISO-8859-1">
<table cols="5" cellspacing="0" border="0">
<colgroup width="139"></colgroup> <colgroup width="119"></colgroup>
<colgroup width="180"></colgroup> <colgroup width="224"></colgroup>
<colgroup width="192"></colgroup> <tbody>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" rowspan="3" align="CENTER"
bgcolor="#CFE7F5" height="53" valign="MIDDLE"><big><b>File
Operation permitted</b></big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" colspan="4" align="CENTER"
bgcolor="#00CCCC" valign="MIDDLE"><big><b>Type of Split
Brain</b></big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big><b>Data SB</b></big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big><b>Metadata SB</b></big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" colspan="2" align="CENTER"
valign="MIDDLE"><big><b>Entry SB</b></big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big><b><br>
</b></big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big><b><br>
</b></big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big><b>Same entry gfid
mismatch SB</b></big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big><b>Different entries</b></big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>write</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes (currently no)</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>read</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes (currently no)</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="18"><big>getfattr</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>lookup</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>?</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>?</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="18"><big>stat/fstat</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>?</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>?</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="18"><big>setfattr</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>touch</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>hard link
creation</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>soft link
creation</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>rename</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>no</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="18"><big>chown</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Currently No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="18"><big>chmod</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Currently No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="17"><big>unlink</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Currently No</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>Yes</big></td>
</tr>
<tr>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="LEFT" height="18"><big>readdir</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>N/A</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>N/A</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>?</big></td>
<td style="border-top: 1px solid #000000; border-bottom: 1px
solid #000000; border-left: 1px solid #000000; border-right:
1px solid #000000" align="CENTER"><big>?</big></td>
</tr>
</tbody>
</table>
<title></title>
<meta name="GENERATOR" content="LibreOffice 3.6 (Linux)">
<style>
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Arial"; font-size:x-small }
-->
</style>
<title></title>
<meta name="GENERATOR" content="LibreOffice 3.6 (Linux)">
<style type="text/css">
<!--
@page { margin: 0.79in }
P { margin-bottom: 0.08in }
P.western { font-size: 12pt }
-->
</style><big><big><br>
- stat() also reports the file size. If a data split-brained
file has different sizes, should stat succeed?<br>
- Likewise if metadata split brain is due to different access
permissions, say one brick has file chmod'ed with 777 and the
other brick has it with 744, should we allow read/write if the
corresponding permission bits are *not* conflciting ? ( as of
today they aren't allowed)<br>
</big><br>
<big>Also,In the table above, Entry Split brain has 2 cases- <br>
i) where same entry has different gfids<br>
ii) each brick has different entries for the same directory
(which can cause deleted files to appear in case of conservative
merge). <br>
Should we allow readdir in either case?<br>
<br>
Thanks,<br>
Ravi<br>
</big>
</big><br>
<blockquote
cite="mid:CAFboF2wZneo=nov=aBsP3257s-+43PfFDGS1rm1-_vkaaLHEPg@mail.gmail.com"
type="cite">
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Nov 13, 2013 at 3:01 AM,
Ravishankar N <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Currenly in glusterfs, when there is a data splt-brain
(only) on a file, we disallow the following operations from
the mount-point by returning EIO to the application:<br>
- Writes to the file (truncate, dd, echo, cp etc)<br>
- Reads to the file (cat)<br>
- Reading extended attributes (getfattr) [1]<br>
<br>
However we do permit the following operations:<br>
-creating hardlinks<br>
-creating symlinks<br>
-mv<br>
-setattr<br>
-chmod<br>
-chown<br>
--touch<br>
-ls<br>
-stat<br>
<br>
While it makes sense to allow `ls` and `stat`, is it okay to
add checks in the FOPS to disallow the other operations?
Allowing creation of links and changing file attributes only
seems to complicate things before the admin can go to the
backend bricks and resolve the splitbrain (by deleteing all
but the healthy copy of the file including hardlinks). More
so if the file is renamed before addressing the split-brain.<br>
Please share your thoughs.<br>
<br>
Thanks,<br>
Ravi<br>
<br>
[1] <a moz-do-not-send="true"
href="http://review.gluster.org/#/c/5988/" target="_blank">http://review.gluster.org/#/c/5988/</a><br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a moz-do-not-send="true"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>