<div dir="ltr"><div>Thanks Pranith - has a bug been filed so we can track this?<br><br></div>-JM<br><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 4, 2014 at 12:47 PM, Pranith Kumar Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
hi Peter,<br>
Sorry for the delay in replying to this mail. I am able to
reproduce the bug consistently. Disabling stat-prefetch reduced the
number of times the errors come but it hasn't eliminated the issue.<br>
<br>
Following the strace output was interesting. The problem always
seems to be because the uid is not matching:<br>
stat("/mnt/fuse1/test-target/test1409848960.3", {st_dev=makedev(0,
41), st_ino=12165775161408537538, st_mode=S_IFDIR|0550, st_nlink=2,
*st_uid=0*, st_gid=9999, st_blksize=131072, st_blocks=1, st_size=6,
st_atime=2014/09/04-22:12:40, st_mtime=2014/09/04-22:12:40,
st_ctime=2014/09/04-22:12:40}) = 0<br>
<br>
uid is coming as 0 and gid is 9999. If we do a stat after the run is
over it is showing things correctly. I am yet to isolate the
problem. I will keep you updated once I find something.<br>
<br>
Pranith<div><div class="h5"><br>
<br>
<div>On 08/22/2014 11:15 PM, Peter Drake
wrote:<br>
</div>
</div></div><blockquote type="cite"><div><div class="h5">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">
<div dir="ltr">I have a replicated Gluster setup, 2 servers
(fs-1 and fs-2) x 1 brick. I have two clients (also on
fs-1 and fs-2) which mount the Gluster volume at /mnt/gfs
(/mnt/gfs type fuse.glusterfs
(rw,default_permissions,allow_other,max_read=131072)).
These clients have scripts which perform various file
operations. One operation they perform looks like this
(note this is pseudocode, the actual script is PHP):</div>
<div dir="ltr">
<div><br>
</div>
<div>1. @mkdir(/mnt/gfs/<span style="white-space:pre-wrap">somedir,
0550</span>);<br>
</div>
<div>2. chown(1234, /mnt/gfs/somedir);</div>
<div>3. chgrp(1234, /mnt/gfs/somedir);</div>
<div><br>
</div>
<div>Note that line 1 may fail on either client because
the directory may have been created on the other client.
These errors are suppressed/ignored. When this
operation is performed simultaneously on both clients,
it usually succeeds in creating a directory with the
expected permissions and ownership. Intermittently
however, we see that these directories are not owned by
the expected user and group.</div>
<div><br>
</div>
<div>I've created a PHP script which can be run on two
clients simultaneously to reproduce the error: <a href="https://gist.github.com/pdrakeweb/ae046b4c70a42309be43" target="_blank">https://gist.github.com/pdrakeweb/ae046b4c70a42309be43</a></div>
<div><br>
</div>
<div>
<div><font face="arial, helvetica, sans-serif">The only
log entry I can find that appears to be related is
from fs-1's mnt-gfs.log file:</font></div>
<div>
<p><font face="arial, helvetica, sans-serif">[2014-08-22
12:27:57.661778] I
[dht-layout.c:640:dht_layout_normalize]
0-test-fs-cluster-1-dht: found anomalies in
/test-target/test1408710477.7. holes=1 overlaps=0</font></p>
</div>
</div>
<div>This occurs in both Gluster 3.4.1 and 3.5.2 (the only
two versions I have tested for this). I am unable to
reproduce the problem on a local (non-gluster)
filesystem. I'd appreciate any insight people might
have into what is going on here and whether this is a
bug in Gluster.<br>
</div>
<div><br>
</div>
<div><span style="font-family:arial,helvetica,sans-serif;white-space:pre-wrap">--
</span><br>
</div>
<div>
<div dir="ltr"><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"><b>Peter
Drake </b>|<b> Cloud Software Engineer</b> | </span><a href="http://acquia.com/" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span></a><br>
<div><span style="font-family:Arial;color:rgb(6,124,187);font-weight:bold;vertical-align:baseline;background-color:transparent"></span><br>
<font face="arial, helvetica, sans-serif"><span style="vertical-align:baseline;font-weight:bold;background-color:transparent">O: </span><a href="tel:781.238.4236" value="+17812384236" target="_blank">781.238.4236</a></font></div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"><b>E:</b><font color="#1155cc"> </font><font color="#000000"><a href="mailto:peter.drake@acquia.com" target="_blank">peter.drake@acquia.com</a></font></span><span style="font-family:Arial;color:rgb(6,124,187);vertical-align:baseline;background-color:transparent"> </span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent">|
Skype: pdrakeweb<br>
</span></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">W:</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> <a href="http://www.acquia.com/" target="_blank">http://www.acquia.com</a></span><a href="http://www.acquia.com/" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);vertical-align:baseline;background-color:transparent"></span></a></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Address:</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> 25
Corporate Drive 4th Floor, Burlington, MA 01803</span></p>
<div>
<div><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span><br>
</div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" target="_blank"><span style="font-family:Arial;font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span><span style="font-family:Arial;vertical-align:baseline;background-color:transparent"> ranked
#1 Software Vendor on the 2012 Inc 500</span></a><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span></p>
</div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/acquia-named-forbes-100-list-america-s-most-promising-companies-nov" target="_blank"><span style="font-family:Arial;font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span><span style="font-family:Arial;vertical-align:baseline;background-color:transparent"> named
One of America’s Most Promising Companies by
Forbes</span></a><span style="font-family:Arial;color:rgb(6,124,187);text-decoration:underline;vertical-align:baseline;background-color:transparent"></span></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Drupal
Sites:</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> </span><span style="font-family:Arial;vertical-align:baseline;background-color:transparent"><a href="http://drupalshowcase.com/" target="_blank">http://drupalshowcase.com</a></span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span></p>
<div><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Twitter</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> <a href="http://www.twitter.com/Acquia" target="_blank">http://www.twitter.com/Acquia</a></span></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"><b>Peter
Drake </b>|<b> Cloud Software Engineer</b> | </span><a href="http://acquia.com/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;font-weight:bold;text-decoration:initial;vertical-align:baseline">Acquia</span></a><br>
<div><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;font-weight:bold;vertical-align:baseline"></span><br>
<font face="arial, helvetica, sans-serif"><span style="vertical-align:baseline;background-color:transparent;font-weight:bold">O: </span><a href="tel:781.238.4236" value="+17812384236" target="_blank">781.238.4236</a></font></div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"><b>E:</b><font color="#1155cc"> </font><font color="#000000"><a href="mailto:peter.drake@acquia.com" target="_blank">peter.drake@acquia.com</a></font></span><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;vertical-align:baseline"> </span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial">|
Skype: pdrakeweb<br>
</span></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">W:</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> <a href="http://www.acquia.com" target="_blank">http://www.acquia.com</a></span><a href="http://www.acquia.com/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;vertical-align:baseline"></span></a></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Address:</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> 25
Corporate Drive 4th Floor, Burlington, MA 01803</span></p>
<div>
<div><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span><br>
</div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;background-color:transparent;font-weight:bold;vertical-align:baseline">Acquia</span><span style="font-family:Arial;background-color:transparent;vertical-align:baseline"> ranked
#1 Software Vendor on the 2012 Inc 500</span></a><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span></p>
</div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/acquia-named-forbes-100-list-america-s-most-promising-companies-nov" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;background-color:transparent;font-weight:bold;vertical-align:baseline">Acquia</span><span style="font-family:Arial;background-color:transparent;vertical-align:baseline"> named
One of America’s Most Promising Companies by Forbes</span></a><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;text-decoration:underline;vertical-align:baseline"></span></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Drupal
Sites:</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> </span><span style="font-family:Arial;background-color:transparent;vertical-align:baseline"><a href="http://drupalshowcase.com" target="_blank">http://drupalshowcase.com</a></span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span></p>
<div><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Twitter</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> <a href="http://www.twitter.com/Acquia" target="_blank">http://www.twitter.com/Acquia</a></span></div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><pre>_______________________________________________
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
<br>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>