You should try upgrading to the 2.0.0 release and try again. They fixed all sorts of bugs.<div><br></div><div>liam<br><div><br><div class="gmail_quote">On Thu, May 7, 2009 at 8:21 AM, Sacerdoti, Federico <span dir="ltr"><<a href="mailto:Federico.Sacerdoti@deshawresearch.com">Federico.Sacerdoti@deshawresearch.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hello,<br>
<br>
I am evaluating glusterfs and have seen some strange behavior with<br>
remove. I have gluster/2.0.0rc4 setup on 10 linux nodes connected with<br>
GigE. The config is Nufa/fuse with one storage brick per server, as seen<br>
in the attached nufa.vol config file, which I use for both clients and<br>
servers.<br>
<br>
My experiment is to launch 10 parallel writers, each of whom writes<br>
32GiB worth of data in small files (2MB) to a shared gluster-fuse<br>
mounted filesystem. The files are named uniquely per client, so each<br>
file is only written once. This worked well, and I am seeing performance<br>
close to that of native disk, even with 8-writers per node.<br>
<br>
However when I do a parallel "rm -rf writedir/" on the 10 nodes, where<br>
writedir is the directory written in by the parallel writers described<br>
above, I see strange effects. There are 69,000 UNLINK errors in the<br>
glusterfsd.log of one server, in the form shown below. This alone is not<br>
surprising as the operation is ocurring in parallel. However the remove<br>
took much longer than expected, 92min, and more surprisingly the rm<br>
command exited 0 but files remained in the writedir!<br>
<br>
I ran rm -rf writedir from a single client, and it too exited 0 but left<br>
the writedir non-empty. Is this expected?<br>
<br>
Thanks,<br>
Federico<br>
<br>
--From glusterfsd.log--<br>
2009-05-04 11:35:15 E [fuse-bridge.c:964:fuse_unlink_cbk]<br>
glusterfs-fuse: 5764889: UNLINK() /write.2MB.runid1.p1/5 => -1 (No such<br>
file or directory)<br>
2009-05-04 11:35:15 E [dht-common.c:1294:dht_err_cbk] nufa: subvolume<br>
drdan0192 returned -1 (No such file or directory)<br>
2009-05-04 11:35:15 E [fuse-bridge.c:964:fuse_unlink_cbk]<br>
glusterfs-fuse: 5764894: UNLINK() /write.2MB.runid1.p1/51 => -1 (No such<br>
file or directory)<br>
--end--<br>
<<nufa.vol>><br>
<br>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br>
<br></blockquote></div><br></div></div>