<div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote">2013/8/23 Maik Kulbe <span dir="ltr"><<a href="mailto:info@linux-web-development.de" target="_blank">info@linux-web-development.de</a>></span><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
-----Original Mail-----<br>
From: Jung Young Seok [<a href="mailto:jung.youngseok@gmail.com" target="_blank">jung.youngseok@gmail.com</a>]<br>
Sent: 22.08.13 - 17:19:32<br>
To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
<br>
Subject: [Gluster-users] Writing performance gets slower when write threadsare many<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Dear gluster users,<br>
<br>
We have set up glusterfs 3.4 on cent 6. 2 Gluster FS servers are<br>
replicated each (2brick x 2).<br>
Glusterfs is mounted on WAS as fuse.<br>
<br>
We realized when there are more thread to write files on gluster fs, it<br>
gets slower on writing.<br>
<br>
We've set 250 thread to write 1M file each repeatedly. In this case we get<br>
under 30 TPS. No NW bandwidth problem.<br>
Is there any way to avoid performance degradation when there are lots of<br>
write threads?<br>
</blockquote>
<br></div>
That depends on a lot of factors - what interconnect are you using? 1GE? 10GE? RDMA? What are your bricks built on? SATA? SSD? RAID?<br>
<br>
My first guess would be that interconnect latency and a kernel maximum for some connection count could be the problem.</blockquote><div> </div><div>Thank you for the comments,<br></div><div><br></div><div>We use 1GE for storage network. When I checked the net stream using iftop, it never reached 1G. (only was under 500Mbps. Since we have 2 glusterfs servers, 250Mb for first brick 250Mb for second brick)</div>
<div>When I scp a file to Gluster servers from WAS directly , it reached 80MB/s each. (used less encrypted option)</div><div><br></div><div>About HDD, </div><div>First server is on SAS 10k RAID1.</div><div>Second server is on SATA 10k RAID1.</div>
<div>Those are for test. We'll use SATA RAID1 both finally to reduce cost.</div><div><br></div><div>We are using CentOS 6.4. Since we use vanilla installation, most of kernel parameters are defaults.</div><div>If you give me any option that I can test, it would be great.</div>
<div><br></div><div>Thank you.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="">
<div class="h5">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Look forward any help.<br>
Thanks.<br>
<br>
Regards,<br>
Youngseok jung<br>
</blockquote>
</div></div></blockquote></div><br></div></div>