<br><br><div class="gmail_quote">On Thu, Sep 8, 2011 at 6:34 PM, Emmanuel Dreyfus <span dir="ltr"><<a href="mailto:manu@netbsd.org">manu@netbsd.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi<br>
<br>
I experience a nasty data corruption bug that I can track down to a bad<br>
interraction between how the NetBSD kernel resize files and an operation<br>
reordering ability glusterfs seems to have.<br>
<br>
Consider the exchanges below that I logged at the FUSE level:<br>
<br>
>> SETATTR size = 12178 -> 32394<br>
>> WRITE size = 12178<br>
<< WRITE size = 12178 -> 32394<br>
>> GETATTR size = 32394<br>
<< GETATTR size = 32394 -> 12178<br>
<< SETATTR size = 32394 -> 32394<br>
<br>
SETATTR si sent before WRITE, but WRITE completes first. Then GETATTR<br>
kicks in and it seems SETATTR is not yet completed, since the old size<br>
is obtained.<br>
<br>
This is a real problem, because the NetBSD kernel will detect a file<br>
shrink in the GETATTR vnode method, and it will truncate the file to<br>
match the new size. The result is that the WRITE that was just done is<br>
discarded, and data is replaced by a chunk of zeroes.<br>
<br>
In what conditions glusterfs performs operation reordering? Is it a bug<br>
or a feature? Linux does not truncate in the same situation?</blockquote><div><br></div><div><br></div><div>GlusterFS does guarantee order of completing operation between two system calls if the second system call starts _after_ the return of the first system call. When two system calls are concurrently in progress there is no expected order of completion. There is no such order of completion in NFS either. Linux's expectation of ordering of system calls is this as well. From what you describe, it looks like NetBSD would have to implement synchronization in its VFS just the way Linux does (and to live within the expectations set by NFS and FUSE like filesystems).</div>
<div><br></div><div>Avati</div><div><br></div></div>