<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Here is the output of volume profile
info after 1.1Gb file is copied over mounted glusterfs on client:<br>
<br>
[root@gfs1 data]# gluster volume profile test info<br>
Brick: 93.123.32.41:/data<br>
-------------------------<br>
Cumulative Stats:<br>
Block Size: 131072b+ <br>
No. of Reads: 0 <br>
No. of Writes: 8192 <br>
%-latency Avg-latency Min-Latency Max-Latency No. of
calls Fop<br>
--------- ----------- ----------- -----------
------------ ----<br>
0.00 0.00 us 0.00 us 0.00 us
1 RELEASE<br>
0.02 114.00 us 35.00 us 208.00 us
5 LOOKUP<br>
0.26 9306.00 us 9306.00 us 9306.00 us
1 CREATE<br>
0.75 26260.00 us 26260.00 us 26260.00 us
1 FLUSH<br>
98.97 424.91 us 165.00 us 14056.00 us
8192 WRITE<br>
<br>
Duration: 18352 seconds<br>
Data Read: 0 bytes<br>
Data Written: 1073741824 bytes<br>
<br>
Interval 14 Stats:<br>
Block Size: 131072b+ <br>
No. of Reads: 0 <br>
No. of Writes: 5353 <br>
%-latency Avg-latency Min-Latency Max-Latency No. of
calls Fop<br>
--------- ----------- ----------- -----------
------------ ----<br>
0.00 0.00 us 0.00 us 0.00 us
1 RELEASE<br>
1.22 26260.00 us 26260.00 us 26260.00 us
1 FLUSH<br>
98.78 397.60 us 169.00 us 13423.00 us
5353 WRITE<br>
<br>
Duration: 172 seconds<br>
Data Read: 0 bytes<br>
Data Written: 701628416 bytes<br>
<br>
[root@gfs1 data]# <br>
<br>
<br>
<div class="moz-signature">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
---<br>
<a
href="http://cloudware.bg/?utm_source=email&utm_medium=signature&utm_content=link&utm_campaign=newwebsite"
style="font-family:Verdana; color:#00ACE5; font-size:12px;
text-decoration:none;">Find out about our new Cloud service -
Cloudware.bg</a><br>
<div style="font-family:Verdana; color:#A0A0A0;font-size:12px;">Access
anywhere. Manage it yourself. Pay as you go.</div>
<hr color="#103E81" noshade="noshade" size="1" width="400px"
align="left">
<div style="font-family:Verdana; color:#103E81; font-size:12px;">
<strong>Yavor Marinov</strong><br>
System Administrator<br>
<br>
Neterra Ltd.<br>
Telephone: +359 2 975 16 16<br>
Fax: +359 2 975 34 36<br>
Mobile: +359 888 610 048<br>
</div>
<a href="http://www.neterra.net" style="font-size:12px;
font-family:Verdana; color:#00ACE5; text-decoration:none;">www.neterra.net</a>
<br>
<br>
<br>
</div>
On 05/23/2013 04:54 PM, Michael Brown wrote:<br>
</div>
<blockquote cite="mid:519E1F90.40400@netdirect.ca" type="cite">
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<div class="moz-cite-prefix">That's only a single data point (one
LOOKUP call) - this tells you nothing.<br>
<br>
You need to check after it's been running (and processing
traffic) for a while.<br>
<br>
For example, here's the stats off one of my bricks (an SSD):<br>
<tt>Brick: fearless2:/export/bricks/500117310007a84c/glusterdata</tt><tt><br>
</tt><tt>------------------------------------------------------------</tt><tt><br>
</tt><tt>Cumulative Stats:</tt><tt><br>
</tt><tt> Block Size: 32b+
64b+ 128b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
0 1 </tt><tt><br>
</tt><tt>No. of Writes: 1
5634 4252 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 256b+
512b+ 2048b+ </tt><tt><br>
</tt><tt> No. of Reads: 0
1 0 </tt><tt><br>
</tt><tt>No. of Writes: 343
24 1 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 4096b+
8192b+ 16384b+ </tt><tt><br>
</tt><tt> No. of Reads: 8
7 10 </tt><tt><br>
</tt><tt>No. of Writes: 4
0 0 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> Block Size: 32768b+
65536b+ 131072b+ </tt><tt><br>
</tt><tt> No. of Reads: 25
165 436 </tt><tt><br>
</tt><tt>No. of Writes: 2
7 36 </tt><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt> %-latency Avg-latency Min-Latency Max-Latency
No. of calls Fop</tt><tt><br>
</tt><tt> --------- ----------- ----------- -----------
------------ ----</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 120 FORGET</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 2940 RELEASE</tt><tt><br>
</tt><tt> 0.00 0.00 us 0.00 us 0.00
us 4554 RELEASEDIR</tt><tt><br>
</tt><tt> 0.00 104.00 us 104.00 us 104.00
us 1 TRUNCATE</tt><tt><br>
</tt><tt> 0.00 125.00 us 110.00 us 140.00
us 2 XATTROP</tt><tt><br>
</tt><tt> 0.01 12.62 us 8.00 us 25.00
us 146 ACCESS</tt><tt><br>
</tt><tt> 0.04 81.77 us 65.00 us 112.00
us 60 SETXATTR</tt><tt><br>
</tt><tt> 0.05 30.99 us 25.00 us 57.00
us 212 SETATTR</tt><tt><br>
</tt><tt> 0.06 12.73 us 8.00 us 59.00
us 574 INODELK</tt><tt><br>
</tt><tt> 0.09 188.43 us 140.00 us 244.00
us 60 CREATE</tt><tt><br>
</tt><tt> 0.11 25.24 us 15.00 us 149.00
us 533 STATFS</tt><tt><br>
</tt><tt> 0.12 260.72 us 206.00 us 430.00
us 60 MKDIR</tt><tt><br>
</tt><tt> 0.26 11.34 us 5.00 us 127.00
us 2925 FLUSH</tt><tt><br>
</tt><tt> 0.27 15.14 us 7.00 us 90.00
us 2274 ENTRYLK</tt><tt><br>
</tt><tt> 0.36 102.52 us 81.00 us 161.00
us 442 RMDIR</tt><tt><br>
</tt><tt> 0.63 27.61 us 17.00 us 606.00
us 2880 OPEN</tt><tt><br>
</tt><tt> 0.76 171.58 us 91.00 us 5691.00
us 555 UNLINK</tt><tt><br>
</tt><tt> 0.87 22.66 us 8.00 us 469.00
us 4812 READDIR</tt><tt><br>
</tt><tt> 0.87 24.37 us 10.00 us 1302.00
us 4506 STAT</tt><tt><br>
</tt><tt> 0.94 61.67 us 16.00 us 194.00
us 1917 GETXATTR</tt><tt><br>
</tt><tt> 1.06 51.20 us 10.00 us 224.00
us 2600 FSTAT</tt><tt><br>
</tt><tt> 1.14 31.46 us 18.00 us 1016.00
us 4554 OPENDIR</tt><tt><br>
</tt><tt> 2.56 31.19 us 18.00 us 4373.00
us 10304 WRITE</tt><tt><br>
</tt><tt> 2.58 417.28 us 15.00 us 1860.00
us 776 READ</tt><tt><br>
</tt><tt> 3.64 17.26 us 6.00 us 4824.00
us 26507 FINODELK</tt><tt><br>
</tt><tt> 24.03 146.42 us 49.00 us 9854.00
us 20622 FXATTROP</tt><tt><br>
</tt><tt> 26.67 652.70 us 42.00 us 89705.00
us 5134 READDIRP</tt><tt><br>
</tt><tt> 32.86 128.19 us 9.00 us 4617.00
us 32204 LOOKUP</tt><br>
<br>
<br>
On 13-05-23 09:03 AM, Yavor Marinov wrote:<br>
</div>
<blockquote cite="mid:519E13BB.50406@neterra.net" type="cite">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
<div class="moz-cite-prefix">I've just enabled profiling of the
volume and this is the information from the profile info
printed:<br>
<br>
[root@gfs1 ~]# gluster volume profile test info<br>
Brick: 93.123.32.41:/data<br>
-------------------------<br>
Cumulative Stats:<br>
%-latency Avg-latency Min-Latency Max-Latency No. of
calls Fop<br>
--------- ----------- ----------- -----------
------------ ----<br>
100.00 148.00 us 148.00 us 148.00
us 1 LOOKUP<br>
<br>
Duration: 13950 seconds<br>
Data Read: 0 bytes<br>
Data Written: 0 bytes<br>
<br>
Interval 4 Stats:<br>
<br>
Duration: 7910 seconds<br>
Data Read: 0 bytes<br>
Data Written: 0 bytes<br>
<br>
[root@gfs1 ~]# <br>
<br>
Anything here that might be useful ?<br>
<br>
<br>
<div class="moz-signature">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
---<br>
<a moz-do-not-send="true"
href="http://cloudware.bg/?utm_source=email&utm_medium=signature&utm_content=link&utm_campaign=newwebsite"
style="font-family:Verdana; color:#00ACE5; font-size:12px;
text-decoration:none;">Find out about our new Cloud
service - Cloudware.bg</a><br>
<div style="font-family:Verdana;
color:#A0A0A0;font-size:12px;">Access anywhere. Manage it
yourself. Pay as you go.</div>
<hr color="#103E81" noshade="noshade" size="1" width="400px"
align="left">
<div style="font-family:Verdana; color:#103E81;
font-size:12px;"> <strong>Yavor Marinov</strong><br>
System Administrator<br>
<br>
Neterra Ltd.<br>
Telephone: +359 2 975 16 16<br>
Fax: +359 2 975 34 36<br>
Mobile: +359 888 610 048<br>
</div>
<a moz-do-not-send="true" href="http://www.neterra.net"
style="font-size:12px; font-family:Verdana; color:#00ACE5;
text-decoration:none;">www.neterra.net</a> <br>
<br>
<br>
</div>
On 05/23/2013 01:10 PM, Явор Маринов wrote:<br>
</div>
<blockquote cite="mid:519DEAFC.9060403@neterra.net" type="cite">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
<div class="moz-cite-prefix">I've made a mistake we are using
30Mbit connectivity on all of the nodes. Below is a iperf
test between the node and the client<br>
<br>
[root@gfs4 ~]# iperf -c 93.123.32.41<br>
------------------------------------------------------------<br>
Client connecting to 93.123.32.41, TCP port 5001<br>
TCP window size: 23.2 KByte (default)<br>
------------------------------------------------------------<br>
[ 3] local 93.123.32.44 port 49838 connected with
93.123.32.41 port 5001<br>
[ ID] Interval Transfer Bandwidth<br>
[ 3] 0.0-10.1 sec 49.9 MBytes 41.5 Mbits/sec<br>
[root@gfs4 ~]# <br>
<br>
But when trying to copy a 1Gb file on the client's mounted
volume the speed between the client and the node is ~500kb/s<br>
<br>
<br>
<div class="moz-signature">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
---<br>
<a moz-do-not-send="true"
href="http://cloudware.bg/?utm_source=email&utm_medium=signature&utm_content=link&utm_campaign=newwebsite"
style="font-family:Verdana; color:#00ACE5;
font-size:12px; text-decoration:none;">Find out about
our new Cloud service - Cloudware.bg</a><br>
<div style="font-family:Verdana;
color:#A0A0A0;font-size:12px;">Access anywhere. Manage
it yourself. Pay as you go.</div>
<hr color="#103E81" noshade="noshade" size="1"
width="400px" align="left">
<div style="font-family:Verdana; color:#103E81;
font-size:12px;"> <strong>Yavor Marinov</strong><br>
System Administrator<br>
<br>
Neterra Ltd.<br>
Telephone: +359 2 975 16 16<br>
Fax: +359 2 975 34 36<br>
Mobile: +359 888 610 048<br>
</div>
<a moz-do-not-send="true" href="http://www.neterra.net"
style="font-size:12px; font-family:Verdana;
color:#00ACE5; text-decoration:none;">www.neterra.net</a>
<br>
<br>
<br>
</div>
On 05/23/2013 12:16 PM, Nux! wrote:<br>
</div>
<blockquote
cite="mid:9042c5cc7890c50b1dd1260591ef9368@li.nux.ro"
type="cite">On 23.05.2013 09:41, Явор Маринов wrote: <br>
<blockquote type="cite">Thanks for your reply. <br>
<br>
No matter how many nodes (currently the volume is only
with its own <br>
node) the speed is really slow. For testing purposes, i
made a volume <br>
with only one node, without any replication - however the
speed is <br>
still ~500kb/s. The cloud servers are limited to 30Gbit/s
but still <br>
the traffic when writing to the node is ~500kb/s <br>
<br>
i'm using 3.3.1 glusterfsd with kernel 2.6.18-348.el5xen
and i need <br>
to know if the the problem is within the kernel. <br>
</blockquote>
<br>
I don't think it is a problem with gluster; I never used el5
for this, but I doubt there's an inherent problem with it
either. That speed limit looks odd to me and I think it's
somewhere in your setup. <br>
Have you done any actual speed tests in the VMs? <br>
<br>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Michael Brown | `One of the main causes of the fall of
Systems Consultant | the Roman Empire was that, lacking zero,
Net Direct Inc. | they had no way to indicate successful
☎: +1 519 883 1172 x5106 | termination of their C programs.' - Firth
</pre>
</blockquote>
<br>
</body>
</html>