<font><font face="verdana,sans-serif">Hi Shishir,</font></font><div><font face="verdana, sans-serif"><br></font></div><div><font face="verdana, sans-serif">Thanks for your attention.<br></font><div><font><font face="verdana,sans-serif"><br>
</font></font></div><div><font><font face="verdana,sans-serif">Hmm - your explanation makes some sense, but those 'T' files don't show up in the client view of the dir - only in the brick view. Is that valid?</font></font></div>
<div><font><font face="verdana,sans-serif"><br></font></font></div><div><font><font face="verdana,sans-serif">I'm using 3.3 on 4 ubuntu 12.04 servers over DDR IPoIB, and the command to initiate the remove brick was:</font></font></div>
<div><font><font face="verdana,sans-serif"><br></font></font></div><div><font><font face="verdana,sans-serif">$ gluster volume remove-brick gli pbs3ib:/bducgl start</font></font></div><div><font><font face="verdana,sans-serif"><br>
</font></font></div><div><font><font face="verdana,sans-serif">and the current status is:</font></font></div><div><font><font face="verdana,sans-serif"><br></font></font></div><div><font><font face="verdana,sans-serif"><div>
$ gluster volume remove-brick gli pbs3ib:/bducgl status</div><div> Node Rebalanced-files size scanned failures status</div><div> --------- ----------- ----------- ----------- ----------- ------------</div>
<div> localhost 0 0 137702 21406 stopped</div><div> pbs2ib 0 0 168991 6921 stopped</div><div> pbs3ib 724683 594890945282 4402804 0 in progress</div>
<div> pbs4ib 0 0 169081 7923 stopped</div><div><br></div></font></font></div><div><font><font face="verdana,sans-serif">(the failures were the same as were seen as when I tried the rebalance command previously).</font></font></div>
<div><font><font face="verdana,sans-serif"><br></font></font></div><div><font><font face="verdana,sans-serif">Best</font></font></div><div><font><font face="verdana,sans-serif">harry<br></font></font><br><div class="gmail_quote">
On Mon, Aug 20, 2012 at 7:09 PM, Shishir Gowda <span dir="ltr"><<a href="mailto:sgowda@redhat.com" target="_blank">sgowda@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Harry,<br>
<br>
These are valid files in glusterfs-dht xlator configured volumes. These are known as link files, which dht uses to maintain files on the hashed subvol, when the actual data resides in non hashed subvolumes(rename can lead to these). The cleanup of these files will be taken care of by running rebalance.<br>
Can you please provide the gluster version you are using, and the remove brick command you used?<br>
<br>
With regards,<br>
Shishir<br>
<br>
----- Original Message -----<br>
From: "Harry Mangalam" <<a href="mailto:hjmangalam@gmail.com">hjmangalam@gmail.com</a>><br>
To: "gluster-users" <<a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
Sent: Tuesday, August 21, 2012 5:01:05 AM<br>
Subject: [Gluster-users] files on gluster brick that have '---------T' designation.<br>
<br>
<br>
I have a working but unbalanced gluster config where one brick has about 2X the usage of the 3 others. I started a remove-brick to force a resolution of this problem (Thanks to JD for the help!), but it's going very slowly, about 2.2MB/s over DDR IPoIB or ~2.3 files/s. In investigating the problem, I may have found a partial explanation - I have found 100s of thousands (maybe millions) of zero-length files existing on the problem brick that do not exist on the client view that have the designation ' ---------T' via 'ls -l'<br>
<br>
<br>
ie:<br>
<br>
<br>
<br>
/bducgl/alamng/Research/Yuki/newF20/runF20_2513/data:<br>
total 0<br>
---------T 2 root root 0 2012-08-04 11:23 backward_sm1003<br>
---------T 2 root root 0 2012-08-04 11:23 backward_sm1007<br>
---------T 2 root root 0 2012-08-04 11:23 backward_sm1029<br>
<br>
<br>
I suspect that these are the ones that are responsible for the enormous expansion of the storage space on this brick and the very slow speed of the 'remove-brick' operation.<br>
<br>
<br>
Does this sound possible? Can I delete these files on the brick to resolve the imbalance? If not, is there a way to process them in some better way to rationalize the imbalance?<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine<br>
[m/c 2225] / 92697 Google Voice Multiplexer: <a href="tel:%28949%29%20478-4487" value="+19494784487">(949) 478-4487</a><br>
415 South Circle View Dr, Irvine, CA, 92697 [shipping]<br>
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)<br>
<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br>Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine<br>[m/c 2225] / 92697 Google Voice Multiplexer: (949) 478-4487<br>415 South Circle View Dr, Irvine, CA, 92697 [shipping]<br>
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)<br><br>
</div></div>