<html><head></head><body>If you experience pain using any filesystem, you should see your doctor. <br>
<br>
If you're not actually experiencing pain, perhaps you should avoid hyperbole and instead talk about what version you tried, what your tests were, how you tried to fix it, and what the results were. <br>
<br>
If you're using a current version with a kernel that has readdirplus support for fuse it shouldn't be that bad. If it is, file a bug report - especially if you have the skills to help diagnose the problem. <br><br><div class="gmail_quote">On April 14, 2014 11:30:26 PM PDT, Liam Slusser <lslusser@gmail.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div dir="ltr"><br /><div>I had about 100 million files in Gluster and it was unbelievably painfully slow. We had to ditch it for other technology.</div></div><div class="gmail_extra"><br /><br /><div class="gmail_quote">On Mon, Apr 14, 2014 at 11:24 PM, Franco Broi <span dir="ltr"><<a href="mailto:franco.broi@iongeo.com" target="_blank">franco.broi@iongeo.com</a>></span> wrote:<br />
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br />
I seriously doubt this is the right filesystem for you, we have problems<br />
listing directories with a few hundred files, never mind millions.<br />
<div><div class="h5"><br />
On Tue, 2014-04-15 at 10:45 +0900, Terada Michitaka wrote:<br />
> Dear All,<br />
><br />
><br />
><br />
> I have a problem with slow writing when there are 10 million files.<br />
> (Top level directories are 2,500.)<br />
><br />
><br />
> I configured GlusterFS distributed cluster(3 nodes).<br />
> Each node's spec is below.<br />
><br />
><br />
> CPU: Xeon E5-2620 (2.00GHz 6 Core)<br />
> HDD: SATA 7200rpm 4TB*12 (RAID 6)<br />
> NW: 10GBEth<br />
> GlusterFS : glusterfs 3.4.2 built on Jan 3 2014 12:38:06<br />
><br />
> This cluster(volume) is mounted on CentOS via FUSE client.<br />
> This volume is storage of our application and I want to store 3<br />
> hundred million to 5 billion files.<br />
><br />
><br />
> I performed a writing test, writing 32KByte file × 10 million to this<br />
> volume, and encountered a problem.<br />
><br />
><br />
> (1) Writing is so slow and slow down as number of files increases.<br />
> In non clustering situation(one node), this node's writing speed is<br />
> 40 MByte/sec at random,<br />
> But writing speed is 3.6MByte/sec on that cluster.<br />
> (2) ls command is very slow.<br />
> About 20 second. Directory creation takes about 10 seconds at<br />
> lowest.<br />
><br />
><br />
> Question:<br />
><br />
> 1)5 Billion files are possible to store in GlusterFS?<br />
> Has someone succeeded to store billion files to GlusterFS?<br />
><br />
> 2) Could you give me a link for a tuning guide or some information of<br />
> tuning?<br />
><br />
> Thanks.<br />
><br />
><br />
> -- Michitaka Terada<br />
</div></div>> _______________________________________________<br />
> Gluster-users mailing list<br />
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br />
> <a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br />
<br />
<br />
_______________________________________________<br />
Gluster-users mailing list<br />
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br />
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></blockquote></div><br /></div>
<p style="margin-top: 2.5em; margin-bottom: 1em; border-bottom: 1px solid #000"></p><pre class="k9mail"><hr /><br />Gluster-users mailing list<br />Gluster-users@gluster.org<br /><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre></blockquote></div><br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my brevity.</body></html>