<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Sorry I am in Australia Time Zone so had to call it a night.<div><br></div><div>I checked the PIDs and none of them were valid.</div><div><br></div><div>The UID 48 is Apache.</div><div><div style="margin: 0px; font-size: 10px; font-family: Monaco; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">apache:x:48:48:Apache:/var/www/html:/bin/bash</div><div><br></div><div>A bit detail as to what I am trying to achieve with gluster. We have 2 web servers which has a folder image that has images that are uploaded by our customers. We are doing master-master replication on mysql, which is working fine. As we have LB configured, customer would land on any of the servers. If they add a product, the database syncs the information, its only the image folder that they upload images to needs to be synced between the servers, thus we looked into glusterfs. Gluster works fine, but the only thing that caught my eye was the bandwidth utilisation bumped up a little bit even when there were no file changes made on either of the server. I understand keep alive, but this looked a bit odd that we are seeing lookup calls for random files on the wire.</div><div>First I thought it was just Gluster using the 200KB/s throughput, but last night I found out that its a combination of Gluster, mysql, LB. I switched off LB & mysql and found that Gluster is using around 30-40KB/s. At the moment we don’t have many files, but this raises scalability questions, what will happen when we reach 1000s of files in that directory.</div><div><br></div><div>I don’t know how Apache is causing this. I have Ccd my dev team on this as well.</div><div><br></div><div>Thanks for all your help.</div><div><br></div><div>Regards,</div><div>Gurdeep.</div><div><br></div><div><div>On 8 Jul 2014, at 12:45 am, Pranith Kumar Karampuri <<a href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF">
hi Gurdeep,<br>
Niels extracted the pids of applications which are causing
traffic here:<br>
<a class="moz-txt-link-freetext" href="http://paste.fedoraproject.org/116010/74385214">http://paste.fedoraproject.org/116010/74385214</a><br>
Please check if any of those pids make sense. Apparently all those
applications are from UID 48. You may want to check who that user as
well?<br>
<br>
Pranith<br>
<div class="moz-cite-prefix">On 07/07/2014 07:52 PM, Gurdeep Singh
(Guru) wrote:<br>
</div>
<blockquote cite="mid:41A8673B-C11E-49A1-A22E-7D281D812A7B@bazaari.com.au" type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Hello Pranith,
<div><br>
</div>
<div>That capture was taken in the morning. There is no pid 14927.</div>
<div><br>
</div>
<div>Please see attached capture that I just took from the server.</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 10px; font-family: Monaco;
color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 14927</div>
<div style="margin: 0px; font-size: 10px; font-family: Monaco;
color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">Warning:
bad syntax, perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px; font-size: 10px; font-family: Monaco;
color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND</div>
<div><br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div>
<div><br>
</div>
<div>Thanks,</div>
<div>Gurdeep.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>On 8 Jul 2014, at 12:12 am, Pranith Kumar Karampuri <<a moz-do-not-send="true" href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<div class="moz-cite-prefix">On 07/07/2014 07:39 PM,
Gurdeep Singh (Guru) wrote:<br>
</div>
<blockquote cite="mid:F94DEE44-1473-43DD-9908-C71D91BB27FC@bazaari.com.au" type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Hello Pranith,
<div><br>
</div>
<div>Process 18629 is not sending any traffic across the
servers. </div>
<div><br>
</div>
<div>The process that are constantly sending packet
across is 1055 & 18611.</div>
</blockquote>
In that case it is the application which is sending the
traffic. Niels just looked into the pcap file and even he
found the process with pid 14927 to be the one sending the
traffic. Could you check what process it is?<br>
<br>
Pranith.<br>
<blockquote cite="mid:F94DEE44-1473-43DD-9908-C71D91BB27FC@bazaari.com.au" type="cite">
<div><br>
</div>
<div>If any, what is the interval of RPC lookup on one
file? can we somehow control the lookup frequency?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Gurdeep.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div apple-content-edited="true"> </div>
<br>
<div>
<div>On 7 Jul 2014, at 11:59 pm, Pranith Kumar
Karampuri <<a moz-do-not-send="true" href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<div class="moz-cite-prefix">On 07/07/2014 07:22
PM, Gurdeep Singh (Guru) wrote:<br>
</div>
<blockquote cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au" type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<div style="margin: 0px; font-size: 10px;
font-family: Monaco; color: rgb(245, 245,
245); background-color: rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 1055</div>
<div style="margin: 0px; font-size: 10px;
font-family: Monaco; color: rgb(245, 245,
245); background-color: rgb(0, 0, 0);">Warning:
bad syntax, perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px; font-size: 10px;
font-family: Monaco; color: rgb(245, 245,
245); background-color: rgb(0, 0, 0);"> PID
TTY STAT TIME MAJFL TRS DRS
RSS %MEM COMMAND</div>
<div style="margin: 0px; font-size: 10px;
font-family: Monaco; color: rgb(245, 245,
245); background-color: rgb(0, 0, 0);"> 1055
? Ssl 86:01 31 0 319148
33092 3.2 /usr/sbin/glusterfs
--volfile-server=srv2 --volfile-id=/gv0
/var/www/html/image/</div>
<div style="margin: 0px; font-size: 10px;
font-family: Monaco; color: rgb(245, 245,
245); background-color: rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div><br>
</div>
</blockquote>
Gurdeep,<br>
Don't see anything odd here :-(. Mount is
looking up files and brick is serving it. Why
don't you keep a watch on the process '18629'
and similar processes in the cluster. Do a ps
aux | grep glustershd to get the pids. There
will be one such process per machine in the
cluster. Check how much it consumes. That is the
only process which does operations without any
operations on mounts.<br>
<br>
Pranith<br>
<br>
<blockquote cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au" type="cite">
<div><br>
</div>
<div><br>
</div>
<div>
<div>On 7 Jul 2014, at 11:49 pm, Pranith
Kumar Karampuri <<a moz-do-not-send="true" href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html;
charset=windows-1252" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<div class="moz-cite-prefix">On
07/07/2014 07:03 PM, Gurdeep Singh
(Guru) wrote:<br>
</div>
<blockquote cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au" type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Hello Niels,
<div><br>
</div>
<div>I did a net hogs on the interface
to see what process might be using
the bandwidth,</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">NetHogs
version 0.8.0</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco;
background-color: rgb(245, 245,
245);"> PID USER PROGRAM
DEV
SENT RECEIVED </div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">18611
root /usr/sbin/glusterfsd
tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
16.307 17.547 KB/sec</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">1055
root /usr/sbin/glusterfs
tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
17.249 16.259 KB/sec</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">13439
guru sshd: guru@pts/0
tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
0.966 0.051 KB/sec</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">18625
root /usr/sbin/glusterfs
tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
0.000 0.000 KB/sec</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">18629
root /usr/sbin/glusterfs
tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
0.000 0.000 KB/sec</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">9636
root /usr/sbin/glusterd
tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
0.000 0.000 KB/sec</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">?
root unknown TCP
0.000
0.000 KB/sec</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco;
background-color: rgb(245, 245,
245);"> TOTAL
34.523 33.856 KB/sec </div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
</blockquote>
Which process corresponds to '1055'?<br>
<br>
Pranith<br>
<blockquote cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au" type="cite">
<div>
<div>
<div>Its glusterfs and glusterfsd
process.</div>
<div><br>
</div>
<div>I looked at the capture file
and see that the lookup is being
made on random files.</div>
<div><br>
</div>
<div>For PID information, please
see this:</div>
<div><br>
</div>
<div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ sudo
netstat -tpn | grep 49152</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
127.0.0.1:49152
127.0.0.1:1012
ESTABLISHED 18611/glusterfsd
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
127.0.0.1:49152
127.0.0.1:1016
ESTABLISHED 18611/glusterfsd
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
127.0.0.1:1016
127.0.0.1:49152
ESTABLISHED 18625/glusterfs
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
10.8.0.6:1021
10.8.0.1:49152
ESTABLISHED 1055/glusterfs
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
10.8.0.6:49152
10.8.0.1:1017
ESTABLISHED 18611/glusterfsd
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
10.8.0.6:1020
10.8.0.1:49152
ESTABLISHED 18629/glusterfs
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
127.0.0.1:1023
127.0.0.1:49152
ESTABLISHED 18629/glusterfs
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
10.8.0.6:49152
10.8.0.1:1022
ESTABLISHED 18611/glusterfsd
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
10.8.0.6:49152
10.8.0.1:1021
ESTABLISHED 18611/glusterfsd
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
127.0.0.1:49152
127.0.0.1:1023
ESTABLISHED 18611/glusterfsd
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
127.0.0.1:1012
127.0.0.1:49152
ESTABLISHED 1055/glusterfs
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">tcp 0 0
10.8.0.6:1019
10.8.0.1:49152
ESTABLISHED 18625/glusterfs
</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ ps -v
18611</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">Warning: bad syntax,
perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);"> PID TTY STAT
TIME MAJFL TRS DRS RSS
%MEM COMMAND</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">18611 ? Ssl
14:12 0 0 650068
20404 2.0
/usr/sbin/glusterfsd -s srv2
--volfile-id
gv0.srv2.root-gluster-vol0 -p
/var/lib/glusterd/vols/gv0</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ ps -v
18629</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">Warning: bad syntax,
perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);"> PID TTY STAT
TIME MAJFL TRS DRS RSS
%MEM COMMAND</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">18629 ? Ssl
0:04 0 0 333296
17380 1.7 /usr/sbin/glusterfs
-s localhost --volfile-id
gluster/glustershd -p
/var/lib/glusterd/glustershd/r</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ ps -v
18629</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">Warning: bad syntax,
perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);"> PID TTY STAT
TIME MAJFL TRS DRS RSS
%MEM COMMAND</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">18629 ? Ssl
0:04 0 0 333296
17380 1.7 /usr/sbin/glusterfs
-s localhost --volfile-id
gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid
-l
/var/log/glusterfs/glustershd.log
-S
/var/run/823fa3197e2d1841be888</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ ps -v
18629</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">Warning: bad syntax,
perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);"> PID TTY STAT
TIME MAJFL TRS DRS RSS
%MEM COMMAND</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">18629 ? Ssl
0:04 0 0 333296
17380 1.7 /usr/sbin/glusterfs
-s localhost --volfile-id
gluster/glustershd -p
/var/lib/glusterd/glustershd/run/glustershd.pid
-l
/var/log/glusterfs/glustershd.log
-S
/var/run/823fa3197e2d1841be8881500723b063.socket
--xlator-option
*replicate*.node-uuid=84af83c9-0a29-</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ ps -v
18625</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">Warning: bad syntax,
perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);"> PID TTY STAT
TIME MAJFL TRS DRS RSS
%MEM COMMAND</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">18625 ? Ssl
0:03 0 0 239528
41040 4.0 /usr/sbin/glusterfs
-s localhost --volfile-id
gluster/nfs -p
/var/lib/glusterd/nfs/run/nfs.pid
-l /var/log/glusterfs/nfs.log
-S
/var/run/5ad5b036fd636cc5dddffa73593e4089.socket</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ sudo
nethogs tun0</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">Waiting for first
packet to arrive (see <a moz-do-not-send="true" href="http://sourceforge.net/">sourceforge.net</a>
bug 1019381)</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ rpm -qa
| grep gluster</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">glusterfs-3.5.1-1.el6.x86_64</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">glusterfs-cli-3.5.1-1.el6.x86_64</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">glusterfs-libs-3.5.1-1.el6.x86_64</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">glusterfs-fuse-3.5.1-1.el6.x86_64</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">glusterfs-server-3.5.1-1.el6.x86_64</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">glusterfs-api-3.5.1-1.el6.x86_64</div>
<div style="margin: 0px;
font-size: 10px; font-family:
Monaco; color: rgb(245, 245,
245); background-color: rgb(0,
0, 0);">[guru@srv2 ~]$ </div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>I don’t see anything odd
here. Please suggest.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Gurdeep.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>On 7 Jul 2014, at 9:06 pm,
Niels de Vos <<a moz-do-not-send="true" href="mailto:ndevos@redhat.com">ndevos@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">On Sun,
Jul 06, 2014 at 11:28:51PM
+1000, Gurdeep Singh (Guru)
wrote:<br>
<blockquote type="cite">Hello,<br>
<br>
I have setup gluster in
replicate type and its working
fine.<br>
<br>
I am seeing a constant
chatting between the hosts for
lookup call and <br>
lookup reply. I am trying to
understand as to why this
traffic is <br>
being initiated constantly.
Please look at the attached
image. This <br>
traffic is using around
200KB/s of constant bandwidth
and is <br>
exhausting our allocated
monthly bandwidth on our 2
VPS.<br>
</blockquote>
<br>
You can use Wireshark to
identify which process does the
LOOKUP calls. <br>
For this, do the following:<br>
<br>
1. select a LOOKUP Call<br>
2. enable the 'packet details'
pane (found in the main menu,
'view')<br>
3. expand the 'Transmission
Control Protocol' tree<br>
4. check the 'Source port' of
the LOOKUP Call<br>
<br>
Together with the 'Source' and
the 'Source port' you can go to
the <br>
server that matches the 'Source'
address. A command like this
would give <br>
you the PID of the process in
the right column:<br>
<br>
# netstat -tpn | grep
$SOURCE_PORT<br>
<br>
And with 'ps -v $PID' you can
check which process is
responsible for the <br>
LOOKUP. This process can be a
fuse-mount, self-heal-daemon or
any other <br>
glusterfs-client. Depending on
the type of client, you maybe
can tune <br>
the workload or other options a
little.<br>
<br>
In Wireshark you can also check
what filename is LOOKUP'd, just
expand <br>
the 'GlusterFS' part in the
'packet details' and check the
'Basename'. <br>
Maybe this filename (without
directory structure) does give
you any <br>
ideas of which activity is
causing the LOOKUPs.<br>
<br>
HTH,<br>
Niels<br>
<br>
<blockquote type="cite"><br>
The configuration I have for
Gluster is:<br>
<br>
[guru@srv1 ~]$ sudo gluster
volume info<br>
[sudo] password for guru: <br>
<br>
Volume Name: gv0<br>
Type: Replicate<br>
Volume ID:
dc8dc3f2-f5bd-4047-9101-acad04695442<br>
Status: Started<br>
Number of Bricks: 1 x 2 = 2<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1:
srv1:/root/gluster-vol0<br>
Brick2:
srv2:/root/gluster-vol0<br>
Options Reconfigured:<br>
cluster.lookup-unhashed: on<br>
performance.cache-refresh-timeout:
60<br>
performance.cache-size: 1GB<br>
storage.health-check-interval:
30<br>
<br>
<br>
<br>
Please suggest how to fine
tune the RPC calls/reply. <br>
</blockquote>
<br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</blockquote></div><br></div></body></html>