<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<div class="moz-cite-prefix">On 1/21/2014 11:05 PM, Mingfan Lu
wrote:<br>
</div>
<blockquote
cite="mid:CAL2gTc1DsM_F1rKP3Pmrx1pO3_2B==ypHPvD2U8ifUA3Ux7b4A@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<div dir="ltr">
<div>
<div>
<div>I have a volume (distribute-replica (*3)), today i
found an interesting problem<br>
<br>
</div>
<div>node22 node23 and node24 are the replica-7 from client
A<br>
</div>
<div>but the annoying thing is when I create dir or write
file from client to replica-7,<br>
</div>
<div><br>
date;dd if=/dev/zero of=49 bs=1MB count=120<br>
Wed Jan 22 11:51:41 CST 2014<br>
120+0 records in<br>
120+0 records out<br>
120000000 bytes (120 MB) copied, 1.96257 s, 61.1 MB/s<br>
</div>
<div><br>
</div>
<div>but I could only find node23 & node24 have the find<br>
</div>
<div>---------------<br>
node23,node24<br>
---------------<br>
/mnt/xfsd/test-volume/test/49<br>
<br>
</div>
<div>in clientA, I use find command<br>
</div>
<div><br>
</div>
<div>I use another machine as client B, and mount the test
volume (newly mounted)<br>
</div>
<div>to run<b> find /mnt/xfsd/test-volume/test/49</b><br>
<br>
</div>
<div>from Client A, the three nodes have the file now.<br>
<br>
---------------<br>
node22,node23.node24<br>
---------------<br>
/mnt/xfsd/test-volume/test/49<br>
<br>
</div>
<div>but in Client A, I delete the file
/mnt/xfsd/test-volume/test/49, node22 still have the file
in brick.<br>
<br>
---------------<br>
node22<br>
---------------<br>
/mnt/xfsd/test-volume/test/49<br>
</div>
<div><br>
</div>
<div>but if i delete the new created files from Client B )<br>
</div>
<div>my question is why node22 have no newly created/write
dirs/files? I have to use find to trigger the self-heal to
fix that?<br>
</div>
<br>
</div>
from ClientA's log, I find something like:<br>
<br>
I [afr-self-heal-data.c:712:afr_sh_data_fix]
0-test-volume-replicate-7: no active sinks for performing
self-heal on file /test/49<br>
<br>
</div>
It is harmless for it is information level?<br>
<div><br>
<div>
<div>I also see something like:<br>
[2014-01-19 10:23:48.422757] E
[afr-self-heal-entry.c:2376:afr_sh_post_nonblocking_entry_cbk]
0-test-volume-replicate-7: Non Blocking entrylks failed
for /test/video/2014/01.<br>
[2014-01-19 10:23:48.423042] E
[afr-self-heal-common.c:2160:afr_self_heal_completion_cbk]
0-test-volume-replicate-7: background entry self-heal
failed on /test/video/2014/01</div>
</div>
</div>
</div>
</blockquote>
From the paths you are listing, it looks like you may be mounting
the bricks, not the gluster volume.<br>
<br>
You MUST mount the gluster volume, not the bricks that make up the
volume. In your example, the mount looks like it is mounting the
xfs volume. Your mount command should be something like:<br>
<blockquote>mount <host name>:test volume
/mount/gluster/test-volume<br>
</blockquote>
If a brick is part of a gluster volume, the brick must NEVER be
written to directly. Yes, what you write MAY eventually be
duplicated over to the other nodes, but if and when that happens is
unpredictable. It will give the unpredictable replication results
that you are seeing.<br>
<br>
The best way to test is to run "mount". If the line where you are
mounting the gluster volume doesn't say "glusterfs" on it, you have
it wrong. Also, the line you use in /etc/fstab must say
"glusterfs", not "xfs" or "ext4".<br>
<br>
If you are in doubt, include the output of "mount" in your next
email to the list.<br>
<br>
Ted Miller<br>
Elkhart, IN, USA<br>
<br>
</body>
</html>