<div dir="ltr"><div class="gmail_quote"><div dir="ltr"><div dir="ltr">I have a replicated Gluster setup, 2 servers (fs-1 and fs-2) x 1 brick. I have two clients (also on fs-1 and fs-2) which mount the Gluster volume at /mnt/gfs (/mnt/gfs type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)). These clients have scripts which perform various file operations. One operation they perform looks like this (note this is pseudocode, the actual script is PHP):</div>
<div dir="ltr"><div><br></div><div>1. @mkdir(/mnt/gfs/<span style="white-space:pre-wrap">somedir, 0550</span>);<br></div><div>2. chown(1234, /mnt/gfs/somedir);</div><div>3. chgrp(1234, /mnt/gfs/somedir);</div><div><br></div>
<div>Note that line 1 may fail on either client because the directory may have been created on the other client. These errors are suppressed/ignored. When this operation is performed simultaneously on both clients, it usually succeeds in creating a directory with the expected permissions and ownership. Intermittently however, we see that these directories are not owned by the expected user and group.</div>
<div><br></div><div>I've created a PHP script which can be run on two clients simultaneously to reproduce the error: <a href="https://gist.github.com/pdrakeweb/ae046b4c70a42309be43" target="_blank">https://gist.github.com/pdrakeweb/ae046b4c70a42309be43</a></div>
<div><br></div><div><div><font face="arial, helvetica, sans-serif">The only log entry I can find that appears to be related is from fs-1's mnt-gfs.log file:</font></div><div><p><font face="arial, helvetica, sans-serif">[2014-08-22 12:27:57.661778] I [dht-layout.c:640:dht_layout_normalize] 0-test-fs-cluster-1-dht: found anomalies in /test-target/test1408710477.7. holes=1 overlaps=0</font></p>
</div><div><pre style="white-space:pre-wrap;word-wrap:break-word"><pre style="word-wrap:break-word;white-space:pre-wrap"></pre></pre></div></div><div>This occurs in both Gluster 3.4.1 and 3.5.2 (the only two versions I have tested for this). I am unable to reproduce the problem on a local (non-gluster) filesystem. I'd appreciate any insight people might have into what is going on here and whether this is a bug in Gluster.<br>
</div><div><br></div><div><span style="font-family:arial,helvetica,sans-serif;white-space:pre-wrap">-- </span><br></div><div><div dir="ltr"><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"><b>Peter Drake </b>|<b> Cloud Software Engineer</b> | </span><a href="http://acquia.com/" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span></a><br>
<div><span style="font-family:Arial;color:rgb(6,124,187);font-weight:bold;vertical-align:baseline;background-color:transparent"></span><br><font face="arial, helvetica, sans-serif"><span style="vertical-align:baseline;font-weight:bold;background-color:transparent">O: </span><a href="tel:781.238.4236" value="+17812384236" target="_blank">781.238.4236</a></font></div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"><b>E:</b><font color="#1155cc"> </font><font color="#000000"><a href="mailto:peter.drake@acquia.com" target="_blank">peter.drake@acquia.com</a></font></span><span style="font-family:Arial;color:rgb(6,124,187);vertical-align:baseline;background-color:transparent"> </span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent">| Skype: pdrakeweb<br>
</span></p><p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">W:</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> <a href="http://www.acquia.com/" target="_blank">http://www.acquia.com</a></span><a href="http://www.acquia.com/" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);vertical-align:baseline;background-color:transparent"></span></a></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Address:</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> 25 Corporate Drive 4th Floor, Burlington, MA 01803</span></p>
<div><div><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span><br></div><p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" target="_blank"><span style="font-family:Arial;font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span><span style="font-family:Arial;vertical-align:baseline;background-color:transparent"> ranked #1 Software Vendor on the 2012 Inc 500</span></a><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span></p>
</div><p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/acquia-named-forbes-100-list-america-s-most-promising-companies-nov" target="_blank"><span style="font-family:Arial;font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span><span style="font-family:Arial;vertical-align:baseline;background-color:transparent"> named One of America’s Most Promising Companies by Forbes</span></a><span style="font-family:Arial;color:rgb(6,124,187);text-decoration:underline;vertical-align:baseline;background-color:transparent"></span></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Drupal Sites:</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> </span><span style="font-family:Arial;vertical-align:baseline;background-color:transparent"><a href="http://drupalshowcase.com/" target="_blank">http://drupalshowcase.com</a></span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span></p>
<div><span style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Twitter</span><span style="vertical-align:baseline;font-family:Arial;background-color:transparent"> <a href="http://www.twitter.com/Acquia" target="_blank">http://www.twitter.com/Acquia</a></span></div>
<div><br></div></div></div></div><div dir="ltr"></div>
</div>
</div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"><b>Peter Drake </b>|<b> Cloud Software Engineer</b> | </span><a href="http://acquia.com/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;font-weight:bold;text-decoration:initial;vertical-align:baseline">Acquia</span></a><br>
<div><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;font-weight:bold;vertical-align:baseline"></span><br><font face="arial, helvetica, sans-serif"><span style="vertical-align:baseline;background-color:transparent;font-weight:bold">O: </span>781.238.4236</font></div>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"><b>E:</b><font color="#1155cc"> </font><font color="#000000"><a href="mailto:peter.drake@acquia.com" target="_blank">peter.drake@acquia.com</a></font></span><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;vertical-align:baseline"> </span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial">| Skype: pdrakeweb<br>
</span></p><p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">W:</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> <a href="http://www.acquia.com" target="_blank">http://www.acquia.com</a></span><a href="http://www.acquia.com/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;vertical-align:baseline"></span></a></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Address:</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> 25 Corporate Drive 4th Floor, Burlington, MA 01803</span></p>
<div><div><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span><br></div><p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;background-color:transparent;font-weight:bold;vertical-align:baseline">Acquia</span><span style="font-family:Arial;background-color:transparent;vertical-align:baseline"> ranked #1 Software Vendor on the 2012 Inc 500</span></a><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span></p>
</div><p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><a href="http://www.acquia.com/about-us/newsroom/press-releases/acquia-named-forbes-100-list-america-s-most-promising-companies-nov" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial;background-color:transparent;font-weight:bold;vertical-align:baseline">Acquia</span><span style="font-family:Arial;background-color:transparent;vertical-align:baseline"> named One of America’s Most Promising Companies by Forbes</span></a><span style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;text-decoration:underline;vertical-align:baseline"></span></p>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Drupal Sites:</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> </span><span style="font-family:Arial;background-color:transparent;vertical-align:baseline"><a href="http://drupalshowcase.com" target="_blank">http://drupalshowcase.com</a></span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span></p>
<div><span style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Twitter</span><span style="vertical-align:baseline;background-color:transparent;font-family:Arial"> <a href="http://www.twitter.com/Acquia" target="_blank">http://www.twitter.com/Acquia</a></span></div>
</div>
</div>