<html><body><div style="font-family: trebuchet ms,sans-serif; font-size: 8pt; color: #000000"><div><br></div><div><br></div><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;" data-mce-style="border-left: 2px solid #1010FF; margin-left: 5px; padding-left: 5px; color: #000; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Anand Avati" <avati@gluster.org><br><b>To: </b>"Vijay Bellur" <vbellur@redhat.com><br><b>Cc: </b>"Krishnan Parthasarathi" <kparthas@redhat.com>, "Anand Avati" <aavati@redhat.com>, "Raghavendra Gowdappa" <rgowdapp@redhat.com>, "Varun Shastry" <vshastry@redhat.com>, "Pranith Kumar Karampuri" <pkarampu@redhat.com>, "Venky Shankar" <vshankar@redhat.com>, "Kaushal M" <kaushal@redhat.com>, "Rajesh Joseph" <rjoseph@redhat.com>, "Kotresh Hiremath Ravishankar" <khiremat@redhat.com>, gluster-devel@nongnu.org<br><b>Sent: </b>Friday, March 7, 2014 12:21:54 AM<br><b>Subject: </b>Re: [Gluster-devel] Barrier design issues wrt volume snapshot<br><div><br></div><div dir="ltr"><br><div class="gmail_extra"><br><div><br></div><div class="gmail_quote">On Thu, Mar 6, 2014 at 12:21 AM, Vijay Bellur <span dir="ltr"><<a href="mailto:vbellur@redhat.com" target="_blank" data-mce-href="mailto:vbellur@redhat.com">vbellur@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style="margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;">Adding gluster-devel.<br><div><br></div><br> On 03/06/2014 01:15 PM, Krishnan Parthasarathi wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style="margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;">All,<br><div><br></div> In recent discussions around design (and implementation) of the barrier<br> feature, couple of things came to light.<br><div><br></div> 1) changelog xlator needs barrier xlator to block unlink and rename FOPs<br> in the call path. This is apart from the current list of FOPs that are blocked<br> in their call back path.<br> This is to make sure that the changelog has a bounded queue of unlink and rename FOPs,<br> from the time barriering is enabled, to be drained, committed to changelog file and published.<br></blockquote></blockquote><div><br></div><div>Why is this necessary?</div></div></div></div></blockquote><div><br></div><div><span style="font-size: small;">FOPs that are still coming through after enabling barrier (assuming that barrier is done in the call path) would end up in a non-consumable changelog. For these operations, geo-rep would resort to FS crawl based on xtime which does not handle unlinks and renames.</span><br></div><div><br></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;" data-mce-style="border-left: 2px solid #1010FF; margin-left: 5px; padding-left: 5px; color: #000; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style="margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style="margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;">2) It is possible in a pure distribute volume that the following sequence of FOPs could result<br> in snapshots of bricks disagreeing on inode type for a file or directory.<br><div><br></div> t1: snap b1<br> t2: unlink /a<br> t3: mkdir /a<br> t4: snap b2<br><div><br></div> where, b1 and b2 are bricks of a pure distribute volume V.<br><div><br></div> The above sequence can happen with the current barrier xlator design, since we allow unlink FOPs<br> to go through to the disk and only block their acknowledgement to the application. This implies<br> a concurrent mkdir on the same name could succeed, since DHT doesn't serialize unlink and mkdir FOPs,<br> unlike AFR.<br><div><br></div> Avati,<br><div><br></div> I hear that you have a solution for problem 2). Could you please start the discussion on this thread?<br> It would help us to decide how to go about with the barrier xlator implementation.<br></blockquote></blockquote><div><br></div><div><br></div><div>The solution is really a long pending implementation of dentry serialization in the resolver of protocol server. Today we allow multiple FOPs to happen in parallel which modify the same dentry. This results in hairy races (including non atomicity of rename) and has been kept open for a while now. Implementing the dentry serialization in the resolver will "solve" 2 as a side effect. Hence that is a better approach than making changes in the barrier translator.</div><div><br></div><div>Avati </div></div></div></div></blockquote><div><br></div></div></body></html>