<div dir="ltr">Same here, we try to keep them under 80% too. <br></div><div class="gmail_extra"><br><div class="gmail_quote">2014-10-06 19:40 GMT-03:00 Dan Mons <span dir="ltr"><<a href="mailto:dmons@cuttingedge.com.au" target="_blank">dmons@cuttingedge.com.au</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Yup, pretty common for us. Once we hit ~90% on either of our two<br>
production clusters (107 TB usable each), performance takes a beating.<br>
<br>
I don't consider this a problem, per se. Most file systems (clustered<br>
or otherwise) are the same. I consider a high water mark for any<br>
production file system to be 80% (and I consider that vendor<br>
agnostic), at which time action should be taken to begin clean up.<br>
That's good sysadminning 101.<br>
<br>
-Dan<br>
<br>
----------------<br>
Dan Mons<br>
Unbreaker of broken things<br>
Cutting Edge<br>
<a href="http://cuttingedge.com.au" target="_blank">http://cuttingedge.com.au</a><br>
<div><div class="h5"><br>
<br>
On 7 October 2014 08:36, Ellison, Bob <<a href="mailto:bob.ellison@ccur.com">bob.ellison@ccur.com</a>> wrote:<br>
> Hello,<br>
><br>
> My glusterfs-3.4.2-1.el6 is having a performance issue. It was working fine<br>
> until the 100TB file system hit ~90% full. I was seeing around 90Mb/s for<br>
> the last 10 months. This then dropped to 40Mb/s. Since nothing changed on<br>
> the system, I focused on the transition to the 90% full file system. I also<br>
> found that the 6 undelaying XFS files systems were pretty fragmented (~56%).<br>
><br>
><br>
><br>
> We are using gluster to achieve a large flat file system. This is a single<br>
> server/node configuration, so no network issues are involved.<br>
><br>
><br>
><br>
> As the problem is on a production system, I setup a smaller test system. I<br>
> monitored performance and was able to duplicate the problem (90MB/s up to<br>
> 90% full, then a drop off in performance thereafter). The closer to 100%<br>
> full, the lower the throughput.<br>
><br>
><br>
><br>
> I then started deleting content from the test server. I was surprised to<br>
> find that the performance did NOT increase – it stayed the same. I took the<br>
> test system to file system 50% full but still saw 40Mb/s!<br>
><br>
><br>
><br>
> The test I set up was designed to fragment the XFS partitions (to mimic the<br>
> production system state). I defragmented online successfully, however<br>
> performance did not increase.<br>
><br>
><br>
><br>
> I am currently trying a rebalance across the 6 XFS partitions to see if that<br>
> helps.<br>
><br>
><br>
><br>
> I was wondering if anyone remembers a problem like this? Is there a chance<br>
> that the rebalance will get me back to the normally seen performance? Would<br>
> upgrading gluster fix this?<br>
><br>
><br>
><br>
> Thanks,<br>
><br>
> Bob<br>
><br>
><br>
</div></div>> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></blockquote></div><br><br clear="all"><br>-- <br><div dir="ltr">Pavlik Salles Juan José<div>Blog - <a href="http://viviendolared.blogspot.com" target="_blank">http://viviendolared.blogspot.com</a></div></div>
</div>