<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">I don't think I have these enabled. How can I confirm that?<div><br><div><div>On Sep 7, 2014, at 12:57 AM, Anand Avati <<a href="mailto:avati@gluster.org">avati@gluster.org</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">The only reason O_APPEND gets stripped on the server side, is because of one of the following xlators:<div><br></div><div>- stripe</div><div>- quiesce</div><div>- crypt</div><div><br></div><div>If you have any of these, please try unloading/reconfiguring without these features and try again.</div><div><br></div><div>Thanks</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Sep 6, 2014 at 3:31 PM, mike <span dir="ltr"><<a href="mailto:mike@luminatewireless.com" target="_blank">mike@luminatewireless.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I was able to narrow it down to smallish python script.<br>
<br>
I've attached that to the bug.<br>
<br>
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1138970" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1138970</a><br>
<div class="HOEnZb"><div class="h5"><br>
<br>
On Sep 6, 2014, at 1:05 PM, Justin Clift <<a href="mailto:justin@gluster.org">justin@gluster.org</a>> wrote:<br>
<br>
> Thanks Mike, this is good stuff. :)<br>
><br>
> + Justin<br>
><br>
><br>
> On 06/09/2014, at 8:19 PM, mike wrote:<br>
>> I upgraded the client to Gluster 3.5.2, but there is no difference.<br>
>><br>
>> The bug is almost certainly in the Fuse client. If I remount the filesystem with NFS, the problem is no longer observable.<br>
>><br>
>> I spent a little time looking through the xlator/fuse-bridge to see where the offsets are coming from, but I'm really not familiar enough with the code, so it is slow going.<br>
>><br>
>> Unfortunately, I'm still having trouble reproducing this in a python script that could be readily attached to a bug report.<br>
>><br>
>> I'll take a crack at that again, but I will a file a bug anyway for completeness.<br>
>><br>
>> On Sep 5, 2014, at 7:10 PM, mike <<a href="mailto:mike@luminatewireless.com">mike@luminatewireless.com</a>> wrote:<br>
>><br>
>>> I have narrowed down the source of the bug.<br>
>>><br>
>>> Here is an strace of glusterfsd <a href="http://fpaste.org/131455/40996378/" target="_blank">http://fpaste.org/131455/40996378/</a><br>
>>><br>
>>> The first line represents a write that does *not* make it into the underlying file.<br>
>>><br>
>>> The last line is the write that stomps the earlier write.<br>
>>><br>
>>> As I said, the client file is opened in O_APPEND mode, but on the glusterfsd side, the file is just O_CREAT|O_WRONLY. The means the offsets to pwrite() need to be valid.<br>
>>><br>
>>> I correlated this to a tcpdump I took and I can see that in fact, the RPCs being sent have the wrong offset. Interestingly, glusterfs.write-is-append = 0, which I wouldn't have expected.<br>
>>><br>
>>> I think the bug lies in the glusterfs fuse client.<br>
>>><br>
>>> As to your question about Gluster 3.5.2, I may be able to do that if I am unable to find the bug in the source.<br>
>>><br>
>>> -Mike<br>
>>><br>
>>> On Sep 5, 2014, at 6:16 PM, Justin Clift <<a href="mailto:justin@gluster.org">justin@gluster.org</a>> wrote:<br>
>>><br>
>>>> On 06/09/2014, at 12:10 AM, mike wrote:<br>
>>>>> I have found that the O_APPEND flag is key to this failure - I had overlooked that flag when reading the strace and trying to cobble up a minimal reproduction.<br>
>>>>><br>
>>>>> I now have a small pair of python scripts that can reliably reproduce this failure.<br>
>>>><br>
>>>><br>
>>>> As a thought, is there a reasonable way you can test this on GlusterFS 3.5.2?<br>
>>>><br>
>>>> There were some important bug fixes in 3.5.2 (from 3.5.1).<br>
>>>><br>
>>>> Note I'm not saying yours is one of them, I'm just asking if it's<br>
>>>> easy to test and find out. :)<br>
>>>><br>
>>>> Regards and best wishes,<br>
>>>><br>
>>>> Justin Clift<br>
>>>><br>
>>>> --<br>
>>>> GlusterFS - <a href="http://www.gluster.org/" target="_blank">http://www.gluster.org</a><br>
>>>><br>
>>>> An open source, distributed file system scaling to several<br>
>>>> petabytes, and handling thousands of clients.<br>
>>>><br>
>>>> My personal twitter: <a href="http://twitter.com/realjustinclift" target="_blank">twitter.com/realjustinclift</a><br>
>>>><br>
>>><br>
>><br>
><br>
> --<br>
> GlusterFS - <a href="http://www.gluster.org/" target="_blank">http://www.gluster.org</a><br>
><br>
> An open source, distributed file system scaling to several<br>
> petabytes, and handling thousands of clients.<br>
><br>
> My personal twitter: <a href="http://twitter.com/realjustinclift" target="_blank">twitter.com/realjustinclift</a><br>
><br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</div></div></blockquote></div><br></div>
</blockquote></div><br></div></body></html>