<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Ravishankar,<br>
<br>
thank you for the explanation.<br>
I expected a performance hit after such a long shutdown, the only
problem is I couldn't understand<br>
if the healing was going or not.<br>
After launching the gluster volume heal vol1 full I can see files
in the .glusterfs/indices/xattrop/ directory<br>
to decrease, but to this rate it would take two weeks to finish,
maybe I would rather delete and recreate the volume<br>
from scratch and with 3.5.<br>
<br>
Thanks<br>
Ivano<br>
<br>
On 5/27/14 7:35 PM, Ravishankar N wrote:<br>
</div>
<blockquote cite="mid:5384CCEE.6010703@redhat.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div class="moz-cite-prefix">On 05/27/2014 08:47 PM, Ivano Talamo
wrote:<br>
</div>
<blockquote cite="mid:5384AC90.1000801@roma1.infn.it" type="cite">Dear
all, <br>
<br>
we have a replicated volume (2 servers with 1 brick each) on
Scientific Linux 6.2 with gluster 3.4. <br>
Everything was running fine until we shutdown of of the two and
kept it down for 2 months. <br>
When it came up again the volume could be healed and we have the
following symptoms <br>
(call #1 the always-up server, #2 the server that was kept
down): <br>
<br>
-doing I/O on the volume has very bad performances (impossible
to keep VM images on it) <br>
<br>
</blockquote>
A replica's bricks are not supposed to be intentionally kept down
even for hours, leave alone months <span class="moz-smiley-s2"><span>
:-( </span></span>; If you do; then when it does come
backup, there would be tons of stuff to heal, so a performance hit
is expected.<br>
<blockquote cite="mid:5384AC90.1000801@roma1.infn.it" type="cite">-on
#1 there's 3997354 files on .glusterfs/indices/xattrop/ and the
number doesn't go down <br>
<br>
</blockquote>
When #2 was down, did the I/O involve directory renames? (see if
there are entries on .glusterfs/landfill on #2). If yes then this
is a known issue and a fix is in progress : <a
moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://review.gluster.org/#/c/7879/">http://review.gluster.org/#/c/7879/</a><br>
<br>
<blockquote cite="mid:5384AC90.1000801@roma1.infn.it" type="cite">-on
#1 gluster volume heal vol1 info the first time takes a lot to
end and doesn't show nothing. <br>
</blockquote>
This is fixed in glusterfs 3.5 where heal info is much more
responsive.<br>
<blockquote cite="mid:5384AC90.1000801@roma1.infn.it" type="cite">after
that it prints "Another transaction is in progress. Please try
again after sometime." <br>
<br>
Furthermore on #1 glusterhd.log is full of messages like this: <br>
[2014-05-27 15:07:44.145326] W
[client-rpc-fops.c:1538:client3_3_inodelk_cbk] 0-vol1-client-0:
remote operation failed: No such file or directory <br>
[2014-05-27 15:07:44.145880] W
[client-rpc-fops.c:1640:client3_3_entrylk_cbk] 0-vol1-client-0:
remote operation failed: No such file or directory <br>
[2014-05-27 15:07:44.146070] E
[afr-self-heal-entry.c:2296:afr_sh_post_nonblocking_entry_cbk]
0-vol1-replicate-0: Non Blocking entrylks failed for
<gfid:bfbe65db-7426-4ca0-bf0b-7d1a28de2052>. <br>
[2014-05-27 15:13:34.772856] E
[afr-self-heal-data.c:1270:afr_sh_data_open_cbk]
0-vol1-replicate-0: open of
<gfid:18a358e0-23d3-4f56-8d74-f5cc38a0d0ea> failed on
child vol1-client-0 (No such file or directory) <br>
<br>
On #2 bricks I see some updates, ie. new filenames appearing and
.glusterfs/indices/xattrop/ is usually empy. <br>
<br>
Do you know what's happening? How can we fix this? <br>
</blockquote>
You could try a `gluster volume heal vol1 full` to see if the
bricks get synced.<br>
<br>
Regards,<br>
Ravi<br>
<blockquote cite="mid:5384AC90.1000801@roma1.infn.it" type="cite">
<br>
thank you, <br>
Ivano <br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>