<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: times new roman,new york,times,serif; font-size: 12pt; color: #000000'>Waiting for further comments.<div><br></div><div>Will resend the design after review is done.</div><div><br></div><div>With regards,</div><div>Shishir<br><br><hr id="zwchr"><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Anand Avati" <anand.avati@gmail.com><br><b>To: </b>"Shishir Gowda" <sgowda@redhat.com><br><b>Cc: </b>gluster-devel@nongnu.org<br><b>Sent: </b>Wednesday, January 23, 2013 1:51:23 PM<br><b>Subject: </b>Re: [Gluster-devel] DHT: Making Dir rename op's crash consistent<br><br><br><br><div class="gmail_quote">On Sun, Oct 28, 2012 at 11:30 PM, Shishir Gowda <span dir="ltr"><<a href="mailto:sgowda@redhat.com" target="_blank">sgowda@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi All,<br>
<br>
This is a proposed enhancement to DHT directory rename operations to make it recoverable in-case of crashes.<br>
<br>
Please feel free to review/comment on the design. There also 2 open issues which need to tackled (see below in recovery logic)<br>
<br>
We propose to add 2 new on-disk xattrs SRC(<key to be decided>:destination path) and DST(<key to be decided>:src path/gfid).<br>
<br>
Consider these scenarios<br>
<br>
case1. Only source directory exists<br>
case2. Both source, and destination directories exist.<br>
<br>
The tasks for rename would be as follows:<br>
<br>
1. Set SRC key on all source directories<br>
2. If step 1 fails, remove xattrs, and fail rename<br>
3. If case2, set xattrs on destination directories<br>
4. If failure in case2, ignore<br>
5. Rename directories (opendir on dst, readdir(ENOEMPTY error), rename dst_hashed subvol first, and then rest)<br></blockquote><div><br></div><div>We probably need to guarantee empty dst dirs on _all_ dst servers before progressing to step 6?(else, fail ENOTEMPTY)?</div><div>I agree, can add this additional stage</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
6. If step 5 fails with any error other than ENOTCONN, fail rename, and remove xattrs<br>
7. If failure is because of ENOTCONN, proceed with rename and return a success.<br>
<br>
Recovery steps (once the brick comes up):<br>
<br>
1. On lookup/readdir (NFS requirement?) query for these SRC and DST key.<br>
2. If SRC key is found , validate:<br>
a. If mtime is less than 5 seconds of the lookup request, then do not heal, as rename might be in progress (Can we make this more fool proof?)<br>
b. If dst does, not exist, proceed<br></blockquote><div><br></div><div>This is a bit confusing. Wasn't DST xattr set only on the destination directory in step 3?</div><div> dst == directory DST == xattr-key</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
c. If dst exists, check its key and see if they match. If mismatch, do not rename, as it might lead to gfid mis-match.</blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
3. Proceed with checks rename of directories (similar to step 5 of above (rename).<br>
4. If successful, remove xattrs, return success.<br>
5. If failure what needs to be done? (other rename's might have succeeded, this might fail due to ENOTEMPTY(even due to race)<br>
<br>
<br>
As for subvol down, we can't guarantee in the scenarios of brick going down after stage 1(setxattr).<br>
<br>
Brick going down before start of subvolume: We do not allow rename to progress anywhere.<br>
<br>
If a brick goes down after setxattr, if it has files, or files are created after its up (possible race), then we cant recover.<br>
<br>
<br>
With regards,<br>
Shishir<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br>
</div><br></div></div></body></html>