<div dir="ltr"><div>Dear All, <br></div><div><br></div><div>I have a problem with slow writing when there are 10 million files.</div><div>(Top level directories are 2,500.)</div><div><br></div><div>I configured GlusterFS distributed cluster(3 nodes).</div>
<div>Each node's spec is below.</div><div><br></div><div> CPU: Xeon E5-2620 (2.00GHz 6 Core)</div><div> HDD: SATA 7200rpm 4TB*12 (RAID 6)</div><div> NW: 10GBEth</div><div> GlusterFS : glusterfs 3.4.2 built on Jan 3 2014 12:38:06</div>
<div> </div><div>This cluster(volume) is mounted on CentOS via FUSE client. </div><div><div>This volume is storage of our application and I want to store 3 hundred million to 5 billion files.</div></div><div><br></div><div>
I performed a writing test, writing 32KByte file × 10 million to this volume, and encountered a problem.</div><div><br></div><div>(1) Writing is so slow and slow down as number of files increases.</div><div> In non clustering situation(one node), this node's writing speed is 40 MByte/sec at random, </div>
<div> But writing speed is 3.6MByte/sec on that cluster.</div><div>(2) ls command is very slow.</div><div> About 20 second. Directory creation takes about 10 seconds at lowest.</div><div><br></div><div>Question:</div><div>
</div><div> 1)5 Billion files are possible to store in GlusterFS?</div><div> Has someone succeeded to store billion files to GlusterFS?</div><div> </div><div> 2) Could you give me a link for a tuning guide or some information of tuning?</div>
<div> </div><div>Thanks.</div><div><br></div><div>-- Michitaka Terada</div></div>