<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<div class="moz-cite-prefix">On 09/04/2014 10:45 PM, Pranith Kumar
Karampuri wrote:<br>
</div>
<blockquote cite="mid:54089E4D.10009@redhat.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<br>
<div class="moz-cite-prefix">On 09/04/2014 10:34 PM, John Mark
Walker wrote:<br>
</div>
<blockquote
cite="mid:CAMUFnfbZYfnqbG-w+N6BmQL2WA09Q3Fi5m_vMgUwxZuEZH8R3g@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>Thanks Pranith - has a bug been filed so we can track
this?<br>
</div>
</div>
</blockquote>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://bugzilla.redhat.com/show_bug.cgi?id=1138386">https://bugzilla.redhat.com/show_bug.cgi?id=1138386</a><br>
</blockquote>
Bug is happening because of synchronization problem between
'distribute' xlator on the mount which created the
directory(directory self-heal code path) and the other mount while
setting the new uid:gid<br>
For detailed RCA check:
<a class="moz-txt-link-freetext" href="https://bugzilla.redhat.com/show_bug.cgi?id=1138386#c4">https://bugzilla.redhat.com/show_bug.cgi?id=1138386#c4</a><br>
<br>
CCdevs who work on dht<br>
<br>
Pranith<br>
<blockquote cite="mid:54089E4D.10009@redhat.com" type="cite"> <br>
Pranith<br>
<blockquote
cite="mid:CAMUFnfbZYfnqbG-w+N6BmQL2WA09Q3Fi5m_vMgUwxZuEZH8R3g@mail.gmail.com"
type="cite">
<div dir="ltr">
<div><br>
</div>
-JM<br>
<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu, Sep 4, 2014 at 12:47 PM,
Pranith Kumar Karampuri <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:pkarampu@redhat.com"
target="_blank">pkarampu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> hi Peter,<br>
Sorry for the delay in replying to this mail. I
am able to reproduce the bug consistently. Disabling
stat-prefetch reduced the number of times the errors
come but it hasn't eliminated the issue.<br>
<br>
Following the strace output was interesting. The problem
always seems to be because the uid is not matching:<br>
stat("/mnt/fuse1/test-target/test1409848960.3",
{st_dev=makedev(0, 41), st_ino=12165775161408537538,
st_mode=S_IFDIR|0550, st_nlink=2, *st_uid=0*,
st_gid=9999, st_blksize=131072, st_blocks=1, st_size=6,
st_atime=2014/09/04-22:12:40,
st_mtime=2014/09/04-22:12:40,
st_ctime=2014/09/04-22:12:40}) = 0<br>
<br>
uid is coming as 0 and gid is 9999. If we do a stat
after the run is over it is showing things correctly. I
am yet to isolate the problem. I will keep you updated
once I find something.<br>
<br>
Pranith
<div>
<div class="h5"><br>
<br>
<div>On 08/22/2014 11:15 PM, Peter Drake wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="h5">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">
<div dir="ltr">I have a replicated Gluster
setup, 2 servers (fs-1 and fs-2) x 1
brick. I have two clients (also on fs-1
and fs-2) which mount the Gluster volume
at /mnt/gfs (/mnt/gfs type fuse.glusterfs
(rw,default_permissions,allow_other,max_read=131072)).
These clients have scripts which perform
various file operations. One operation
they perform looks like this (note this is
pseudocode, the actual script is PHP):</div>
<div dir="ltr">
<div><br>
</div>
<div>1. @mkdir(/mnt/gfs/<span
style="white-space:pre-wrap">somedir,
0550</span>);<br>
</div>
<div>2. chown(1234, /mnt/gfs/somedir);</div>
<div>3. chgrp(1234, /mnt/gfs/somedir);</div>
<div><br>
</div>
<div>Note that line 1 may fail on either
client because the directory may have
been created on the other client. These
errors are suppressed/ignored. When
this operation is performed
simultaneously on both clients, it
usually succeeds in creating a directory
with the expected permissions and
ownership. Intermittently however, we
see that these directories are not owned
by the expected user and group.</div>
<div><br>
</div>
<div>I've created a PHP script which can
be run on two clients simultaneously to
reproduce the error: <a
moz-do-not-send="true"
href="https://gist.github.com/pdrakeweb/ae046b4c70a42309be43"
target="_blank">https://gist.github.com/pdrakeweb/ae046b4c70a42309be43</a></div>
<div><br>
</div>
<div>
<div><font face="arial, helvetica,
sans-serif">The only log entry I can
find that appears to be related is
from fs-1's mnt-gfs.log file:</font></div>
<div>
<p><font face="arial, helvetica,
sans-serif">[2014-08-22
12:27:57.661778] I
[dht-layout.c:640:dht_layout_normalize]
0-test-fs-cluster-1-dht: found
anomalies in
/test-target/test1408710477.7.
holes=1 overlaps=0</font></p>
</div>
</div>
<div>This occurs in both Gluster 3.4.1 and
3.5.2 (the only two versions I have
tested for this). I am unable to
reproduce the problem on a local
(non-gluster) filesystem. I'd
appreciate any insight people might have
into what is going on here and whether
this is a bug in Gluster.<br>
</div>
<div><br>
</div>
<div><span
style="font-family:arial,helvetica,sans-serif;white-space:pre-wrap">--
</span><br>
</div>
<div>
<div dir="ltr"><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"><b>Peter
Drake </b>|<b> Cloud Software
Engineer</b> | </span><a
moz-do-not-send="true"
href="http://acquia.com/"
target="_blank"><span
style="font-family:Arial;color:rgb(6,124,187);font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span></a><br>
<div><span
style="font-family:Arial;color:rgb(6,124,187);font-weight:bold;vertical-align:baseline;background-color:transparent"></span><br>
<font face="arial, helvetica,
sans-serif"><span
style="vertical-align:baseline;font-weight:bold;background-color:transparent">O: </span><a
moz-do-not-send="true"
href="tel:781.238.4236"
value="+17812384236"
target="_blank">781.238.4236</a></font></div>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"><b>E:</b><font
color="#1155cc"> </font><font
color="#000000"><a
moz-do-not-send="true"
href="mailto:peter.drake@acquia.com"
target="_blank">peter.drake@acquia.com</a></font></span><span
style="font-family:Arial;color:rgb(6,124,187);vertical-align:baseline;background-color:transparent"> </span><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent">|
Skype: pdrakeweb<br>
</span></p>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">W:</span><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"> <a
moz-do-not-send="true"
href="http://www.acquia.com/"
target="_blank">http://www.acquia.com</a></span><a
moz-do-not-send="true"
href="http://www.acquia.com/"
target="_blank"><span
style="font-family:Arial;color:rgb(6,124,187);vertical-align:baseline;background-color:transparent"></span></a></p>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Address:</span><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"> 25
Corporate Drive 4th Floor,
Burlington, MA 01803</span></p>
<div>
<div><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span><br>
</div>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><a
moz-do-not-send="true"
href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing"
target="_blank"><span
style="font-family:Arial;font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span><span
style="font-family:Arial;vertical-align:baseline;background-color:transparent"> ranked
#1 Software Vendor on the 2012
Inc 500</span></a><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span></p>
</div>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><a
moz-do-not-send="true"
href="http://www.acquia.com/about-us/newsroom/press-releases/acquia-named-forbes-100-list-america-s-most-promising-companies-nov"
target="_blank"><span
style="font-family:Arial;font-weight:bold;vertical-align:baseline;background-color:transparent">Acquia</span><span
style="font-family:Arial;vertical-align:baseline;background-color:transparent"> named
One of America’s Most Promising
Companies by Forbes</span></a><span
style="font-family:Arial;color:rgb(6,124,187);text-decoration:underline;vertical-align:baseline;background-color:transparent"></span></p>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Drupal
Sites:</span><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"> </span><span
style="font-family:Arial;vertical-align:baseline;background-color:transparent"><a
moz-do-not-send="true"
href="http://drupalshowcase.com/"
target="_blank">http://drupalshowcase.com</a></span><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"></span></p>
<div><span
style="vertical-align:baseline;font-family:Arial;font-weight:bold;background-color:transparent">Twitter</span><span
style="vertical-align:baseline;font-family:Arial;background-color:transparent"> <a
moz-do-not-send="true"
href="http://www.twitter.com/Acquia"
target="_blank">http://www.twitter.com/Acquia</a></span></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr"><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"><b>Peter
Drake </b>|<b> Cloud Software Engineer</b> |
</span><a moz-do-not-send="true"
href="http://acquia.com/"
style="color:rgb(17,85,204)" target="_blank"><span
style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;font-weight:bold;text-decoration:initial;vertical-align:baseline">Acquia</span></a><br>
<div><span
style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;font-weight:bold;vertical-align:baseline"></span><br>
<font face="arial, helvetica, sans-serif"><span
style="vertical-align:baseline;background-color:transparent;font-weight:bold">O: </span><a
moz-do-not-send="true"
href="tel:781.238.4236"
value="+17812384236" target="_blank">781.238.4236</a></font></div>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"><b>E:</b><font
color="#1155cc"> </font><font
color="#000000"><a
moz-do-not-send="true"
href="mailto:peter.drake@acquia.com"
target="_blank">peter.drake@acquia.com</a></font></span><span
style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;vertical-align:baseline"> </span><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial">|
Skype: pdrakeweb<br>
</span></p>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">W:</span><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"> <a
moz-do-not-send="true"
href="http://www.acquia.com"
target="_blank">http://www.acquia.com</a></span><a
moz-do-not-send="true"
href="http://www.acquia.com/"
style="color:rgb(17,85,204)"
target="_blank"><span
style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;vertical-align:baseline"></span></a></p>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Address:</span><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"> 25
Corporate Drive 4th Floor, Burlington, MA
01803</span></p>
<div>
<div><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span><br>
</div>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><a
moz-do-not-send="true"
href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing"
style="color:rgb(17,85,204)"
target="_blank"><span
style="font-family:Arial;background-color:transparent;font-weight:bold;vertical-align:baseline">Acquia</span><span
style="font-family:Arial;background-color:transparent;vertical-align:baseline"> ranked
#1 Software Vendor on the 2012 Inc 500</span></a><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span></p>
</div>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><a
moz-do-not-send="true"
href="http://www.acquia.com/about-us/newsroom/press-releases/acquia-named-forbes-100-list-america-s-most-promising-companies-nov"
style="color:rgb(17,85,204)"
target="_blank"><span
style="font-family:Arial;background-color:transparent;font-weight:bold;vertical-align:baseline">Acquia</span><span
style="font-family:Arial;background-color:transparent;vertical-align:baseline"> named
One of America’s Most Promising
Companies by Forbes</span></a><span
style="font-family:Arial;color:rgb(6,124,187);background-color:transparent;text-decoration:underline;vertical-align:baseline"></span></p>
<p dir="ltr"
style="margin-top:0pt;margin-bottom:0pt"><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Drupal
Sites:</span><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"> </span><span
style="font-family:Arial;background-color:transparent;vertical-align:baseline"><a
moz-do-not-send="true"
href="http://drupalshowcase.com"
target="_blank">http://drupalshowcase.com</a></span><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"></span></p>
<div><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial;font-weight:bold">Twitter</span><span
style="vertical-align:baseline;background-color:transparent;font-family:Arial"> <a
moz-do-not-send="true"
href="http://www.twitter.com/Acquia"
target="_blank">http://www.twitter.com/Acquia</a></span></div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a moz-do-not-send="true"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>