<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Could you give your 'gluster volume info' output?<br>
<br>
Pranith<br>
<div class="moz-cite-prefix">On 10/13/2014 09:36 PM, Roman wrote:<br>
</div>
<blockquote
cite="mid:CAFR=TBr6HHK_Qs7dDO=X1GnzsrBrOocZcJMJFJhuMwakLfuiRA@mail.gmail.com"
type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>I've got this kind of setup (servers run replica)</div>
<div><br>
</div>
<div><br>
</div>
<div>@ 10G backend</div>
<div>gluster storage1</div>
<div>gluster storage2</div>
<div>gluster client1</div>
<div><br>
</div>
<div>@1g backend</div>
<div>other gluster clients</div>
<div><br>
</div>
<div>Servers got HW RAID5 with SAS disks.</div>
<div><br>
</div>
<div>So today I've desided to create a 900GB file for iscsi
target that will be located @ glusterfs separate volume, using
dd (just a dummy file filled with zeros, bs=1G count 900)</div>
<div>For the first of all the process took pretty lots of time,
the writing speed was 130 MB/sec (client port was 2 gbps,
servers ports were running @ 1gbps).</div>
<div>Then it reported something like "endpoint is not connected"
and all of my VMs on the other volume started to give me IO
errors.</div>
<div>Servers load was around 4,6 (total 12 cores)</div>
<div><br>
</div>
<div>Maybe it was due to timeout of 2 secs, so I've made it a
big higher, 10 sec.</div>
<div><br>
</div>
<div>Also during the dd image creation time, VMs very often
reported me that their disks are slow like</div>
<div>
<p class="">WARNINGs: Read IO Wait time is -0.02 (outside
range
[0:1]).</p>
<p class="">Is 130MB /sec is the maximum bandwidth for all of
the volumes in total? That why would we need 10g backends?</p>
<p class="">HW Raid local speed is 300 MB/sec, so it should
not be an issue. any ideas or mby any advices?</p>
<p class=""><br>
</p>
<p class="">Maybe some1 got optimized sysctl.conf for 10G
backend?</p>
<p class="">mine is pretty simple, which can be found from
googling.</p>
<p class=""><br>
</p>
<p class="">just to mention: those VM-s were connected using
separate 1gbps intraface, which means, they should not be
affected by the client with 10g backend.</p>
<p class=""><br>
</p>
<p class="">logs are pretty useless, they just say this
during the outage</p>
<p class=""><br>
</p>
<p class="">[2014-10-13 12:09:18.392910] W
[client-handshake.c:276:client_ping_cbk]
0-HA-2TB-TT-Proxmox-cluster-client-0: timer must have
expired</p>
<p class="">[2014-10-13 12:10:08.389708] C
[client-handshake.c:127:rpc_client_ping_timer_expired]
0-HA-2TB-TT-Proxmox-cluster-client-0: server <a
moz-do-not-send="true" href="http://10.250.0.1:49159">10.250.0.1:49159</a>
has not responded in the last 2 seconds, disconnecting.</p>
<p class="">[2014-10-13 12:10:08.390312] W
[client-handshake.c:276:client_ping_cbk]
0-HA-2TB-TT-Proxmox-cluster-client-0: timer must have
expired</p>
</div>
<div>so I decided to set the timout a bit higher.</div>
<div>
<div><br>
</div>
<div>So it seems to me, that under high load GlusterFS is not
useable? 130 MB/s is not that much to get some kind of
timeouts or makeing the systme so slow, that VM-s feeling
themselves bad.</div>
<div><br>
</div>
<div>Of course, after the disconnection, healing process was
started, but as VM-s lost connection to both of servers, it
was pretty useless, they could not run anymore. and BTW,
when u load the server with such huge job (dd of 900GB),
healing process goes soooooo slow :)</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
-- <br>
Best regards,<br>
Roman.
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>