<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<div class="moz-cite-prefix">On 07/08/2014 09:12 AM, Gurdeep Singh
(Guru) wrote:<br>
</div>
<blockquote
cite="mid:50BE6F51-EBAB-4F5B-8012-6EF6049D3AC0@bazaari.com.au"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Sorry I am in Australia Time Zone so had to call it a night.
<div><br>
</div>
<div>I checked the PIDs and none of them were valid.</div>
<div><br>
</div>
<div>The UID 48 is Apache.</div>
<div>
<div style="margin: 0px; font-size: 10px; font-family: Monaco;
color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">apache:x:48:48:Apache:/var/www/html:/bin/bash</div>
<div><br>
</div>
<div>A bit detail as to what I am trying to achieve with
gluster. We have 2 web servers which has a folder image that
has images that are uploaded by our customers. We are doing
master-master replication on mysql, which is working fine. As
we have LB configured, customer would land on any of the
servers. If they add a product, the database syncs the
information, its only the image folder that they upload images
to needs to be synced between the servers, thus we looked into
glusterfs. Gluster works fine, but the only thing that caught
my eye was the bandwidth utilisation bumped up a little bit
even when there were no file changes made on either of the
server. I understand keep alive, but this looked a bit odd
that we are seeing lookup calls for random files on the wire.</div>
<div>First I thought it was just Gluster using the 200KB/s
throughput, but last night I found out that its a combination
of Gluster, mysql, LB. I switched off LB & mysql and found
that Gluster is using around 30-40KB/s. At the moment we don’t
have many files, but this raises scalability questions, what
will happen when we reach 1000s of files in that directory.</div>
<div><br>
</div>
<div>I don’t know how Apache is causing this. I have Ccd my dev
team on this as well.</div>
</div>
</blockquote>
Thanks Gurdeep. At least for now it seems to be traffic from apache.
Probably we need to find root cause for what can cause crawls if any
by apache on gluster mount.<br>
<br>
Pranith<br>
<blockquote
cite="mid:50BE6F51-EBAB-4F5B-8012-6EF6049D3AC0@bazaari.com.au"
type="cite">
<div>
<div><br>
</div>
<div>Thanks for all your help.</div>
<div><br>
</div>
<div>Regards,</div>
<div>Gurdeep.</div>
<div><br>
</div>
<div>
<div>On 8 Jul 2014, at 12:45 am, Pranith Kumar Karampuri <<a
moz-do-not-send="true" href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> hi Gurdeep,<br>
Niels extracted the pids of applications which are
causing traffic here:<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://paste.fedoraproject.org/116010/74385214">http://paste.fedoraproject.org/116010/74385214</a><br>
Please check if any of those pids make sense. Apparently
all those applications are from UID 48. You may want to
check who that user as well?<br>
<br>
Pranith<br>
<div class="moz-cite-prefix">On 07/07/2014 07:52 PM,
Gurdeep Singh (Guru) wrote:<br>
</div>
<blockquote
cite="mid:41A8673B-C11E-49A1-A22E-7D281D812A7B@bazaari.com.au"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Hello Pranith,
<div><br>
</div>
<div>That capture was taken in the morning. There is no
pid 14927.</div>
<div><br>
</div>
<div>Please see attached capture that I just took from
the server.</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 10px; font-family:
Monaco; color: rgb(245, 245, 245); background-color:
rgb(0, 0, 0);">[guru@srv2 ~]$ ps -v 14927</div>
<div style="margin: 0px; font-size: 10px; font-family:
Monaco; color: rgb(245, 245, 245); background-color:
rgb(0, 0, 0);">Warning: bad syntax, perhaps a bogus
'-'? See /usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px; font-size: 10px; font-family:
Monaco; color: rgb(245, 245, 245); background-color:
rgb(0, 0, 0);"> PID TTY STAT TIME MAJFL
TRS DRS RSS %MEM COMMAND</div>
<div><br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div>
<div><br>
</div>
<div>Thanks,</div>
<div>Gurdeep.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>On 8 Jul 2014, at 12:12 am, Pranith Kumar
Karampuri <<a moz-do-not-send="true"
href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<div class="moz-cite-prefix">On 07/07/2014 07:39
PM, Gurdeep Singh (Guru) wrote:<br>
</div>
<blockquote
cite="mid:F94DEE44-1473-43DD-9908-C71D91BB27FC@bazaari.com.au"
type="cite">
<meta http-equiv="Content-Type"
content="text/html; charset=windows-1252">
Hello Pranith,
<div><br>
</div>
<div>Process 18629 is not sending any traffic
across the servers. </div>
<div><br>
</div>
<div>The process that are constantly sending
packet across is 1055 & 18611.</div>
</blockquote>
In that case it is the application which is
sending the traffic. Niels just looked into the
pcap file and even he found the process with pid
14927 to be the one sending the traffic. Could
you check what process it is?<br>
<br>
Pranith.<br>
<blockquote
cite="mid:F94DEE44-1473-43DD-9908-C71D91BB27FC@bazaari.com.au"
type="cite">
<div><br>
</div>
<div>If any, what is the interval of RPC
lookup on one file? can we somehow control
the lookup frequency?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Gurdeep.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div apple-content-edited="true"> </div>
<br>
<div>
<div>On 7 Jul 2014, at 11:59 pm, Pranith
Kumar Karampuri <<a
moz-do-not-send="true"
href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html;
charset=windows-1252"
http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<div class="moz-cite-prefix">On
07/07/2014 07:22 PM, Gurdeep Singh
(Guru) wrote:<br>
</div>
<blockquote
cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au"
type="cite">
<meta http-equiv="Content-Type"
content="text/html;
charset=windows-1252">
<meta http-equiv="Content-Type"
content="text/html;
charset=windows-1252">
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 1055</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">Warning:
bad syntax, perhaps a bogus '-'?
See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">
PID TTY STAT TIME MAJFL
TRS DRS RSS %MEM COMMAND</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);"> 1055
? Ssl 86:01 31 0
319148 33092 3.2
/usr/sbin/glusterfs
--volfile-server=srv2
--volfile-id=/gv0
/var/www/html/image/</div>
<div style="margin: 0px; font-size:
10px; font-family: Monaco; color:
rgb(245, 245, 245);
background-color: rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div><br>
</div>
</blockquote>
Gurdeep,<br>
Don't see anything odd here
:-(. Mount is looking up files and
brick is serving it. Why don't you
keep a watch on the process '18629'
and similar processes in the cluster.
Do a ps aux | grep glustershd to get
the pids. There will be one such
process per machine in the cluster.
Check how much it consumes. That is
the only process which does operations
without any operations on mounts.<br>
<br>
Pranith<br>
<br>
<blockquote
cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au"
type="cite">
<div><br>
</div>
<div><br>
</div>
<div>
<div>On 7 Jul 2014, at 11:49 pm,
Pranith Kumar Karampuri <<a
moz-do-not-send="true"
href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
wrote:</div>
<br
class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html;
charset=windows-1252"
http-equiv="Content-Type">
<div text="#000000"
bgcolor="#FFFFFF"> <br>
<div class="moz-cite-prefix">On
07/07/2014 07:03 PM, Gurdeep
Singh (Guru) wrote:<br>
</div>
<blockquote
cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au"
type="cite">
<meta
http-equiv="Content-Type"
content="text/html;
charset=windows-1252">
Hello Niels,
<div><br>
</div>
<div>I did a net hogs on the
interface to see what
process might be using the
bandwidth,</div>
<div><br>
</div>
<div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">NetHogs
version 0.8.0</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
background-color:
rgb(245, 245, 245);">
PID USER PROGRAM
DEV SENT
RECEIVED </div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">18611
root
/usr/sbin/glusterfsd
tun0<span
class="Apple-tab-span"
style="white-space:pre"> </span> 16.307 17.547 KB/sec</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">1055
root
/usr/sbin/glusterfs
tun0<span
class="Apple-tab-span"
style="white-space:pre"> </span> 17.249 16.259 KB/sec</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">13439
guru sshd:
guru@pts/0
tun0<span
class="Apple-tab-span"
style="white-space:pre"> </span> 0.966 0.051 KB/sec</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">18625
root
/usr/sbin/glusterfs
tun0<span
class="Apple-tab-span"
style="white-space:pre"> </span> 0.000 0.000 KB/sec</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">18629
root
/usr/sbin/glusterfs
tun0<span
class="Apple-tab-span"
style="white-space:pre"> </span> 0.000 0.000 KB/sec</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">9636
root
/usr/sbin/glusterd
tun0<span
class="Apple-tab-span"
style="white-space:pre"> </span> 0.000 0.000 KB/sec</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);">?
root unknown TCP
0.000 0.000
KB/sec</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
background-color:
rgb(245, 245, 245);">
TOTAL
34.523
33.856 KB/sec </div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div style="margin: 0px;
font-size: 10px;
font-family: Monaco;
color: rgb(245, 245,
245); background-color:
rgb(0, 0, 0);
min-height: 14px;"><br>
</div>
<div>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
</blockquote>
Which process corresponds to
'1055'?<br>
<br>
Pranith<br>
<blockquote
cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au"
type="cite">
<div>
<div>
<div>Its glusterfs and
glusterfsd process.</div>
<div><br>
</div>
<div>I looked at the
capture file and see
that the lookup is
being made on random
files.</div>
<div><br>
</div>
<div>For PID
information, please
see this:</div>
<div><br>
</div>
<div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ sudo netstat
-tpn | grep 49152</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
127.0.0.1:49152
127.0.0.1:1012
ESTABLISHED
18611/glusterfsd </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
127.0.0.1:49152
127.0.0.1:1016
ESTABLISHED
18611/glusterfsd </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
127.0.0.1:1016
127.0.0.1:49152
ESTABLISHED
18625/glusterfs </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
10.8.0.6:1021
10.8.0.1:49152
ESTABLISHED
1055/glusterfs </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
10.8.0.6:49152
10.8.0.1:1017
ESTABLISHED
18611/glusterfsd </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
10.8.0.6:1020
10.8.0.1:49152
ESTABLISHED
18629/glusterfs </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
127.0.0.1:1023
127.0.0.1:49152
ESTABLISHED
18629/glusterfs </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
10.8.0.6:49152
10.8.0.1:1022
ESTABLISHED
18611/glusterfsd </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
10.8.0.6:49152
10.8.0.1:1021
ESTABLISHED
18611/glusterfsd </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
127.0.0.1:49152
127.0.0.1:1023
ESTABLISHED
18611/glusterfsd </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
127.0.0.1:1012
127.0.0.1:49152
ESTABLISHED
1055/glusterfs </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">tcp
0 0
10.8.0.6:1019
10.8.0.1:49152
ESTABLISHED
18625/glusterfs </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 18611</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">Warning:
bad syntax, perhaps
a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);"> PID
TTY STAT
TIME MAJFL TRS
DRS RSS %MEM
COMMAND</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">18611
? Ssl
14:12 0 0
650068 20404 2.0
/usr/sbin/glusterfsd
-s srv2 --volfile-id
gv0.srv2.root-gluster-vol0
-p
/var/lib/glusterd/vols/gv0</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 18629</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">Warning:
bad syntax, perhaps
a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);"> PID
TTY STAT
TIME MAJFL TRS
DRS RSS %MEM
COMMAND</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">18629
? Ssl
0:04 0 0
333296 17380 1.7
/usr/sbin/glusterfs
-s localhost
--volfile-id
gluster/glustershd
-p
/var/lib/glusterd/glustershd/r</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 18629</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">Warning:
bad syntax, perhaps
a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);"> PID
TTY STAT
TIME MAJFL TRS
DRS RSS %MEM
COMMAND</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">18629
? Ssl
0:04 0 0
333296 17380 1.7
/usr/sbin/glusterfs
-s localhost
--volfile-id
gluster/glustershd
-p
/var/lib/glusterd/glustershd/run/glustershd.pid
-l
/var/log/glusterfs/glustershd.log
-S
/var/run/823fa3197e2d1841be888</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 18629</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">Warning:
bad syntax, perhaps
a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);"> PID
TTY STAT
TIME MAJFL TRS
DRS RSS %MEM
COMMAND</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">18629
? Ssl
0:04 0 0
333296 17380 1.7
/usr/sbin/glusterfs
-s localhost
--volfile-id
gluster/glustershd
-p
/var/lib/glusterd/glustershd/run/glustershd.pid
-l
/var/log/glusterfs/glustershd.log
-S
/var/run/823fa3197e2d1841be8881500723b063.socket
--xlator-option
*replicate*.node-uuid=84af83c9-0a29-</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ ps -v 18625</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">Warning:
bad syntax, perhaps
a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);"> PID
TTY STAT
TIME MAJFL TRS
DRS RSS %MEM
COMMAND</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">18625
? Ssl
0:03 0 0
239528 41040 4.0
/usr/sbin/glusterfs
-s localhost
--volfile-id
gluster/nfs -p
/var/lib/glusterd/nfs/run/nfs.pid
-l
/var/log/glusterfs/nfs.log
-S
/var/run/5ad5b036fd636cc5dddffa73593e4089.socket</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ sudo nethogs
tun0</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">Waiting
for first packet to
arrive (see <a
moz-do-not-send="true"
href="http://sourceforge.net/">sourceforge.net</a> bug 1019381)</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ rpm -qa | grep
gluster</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">glusterfs-3.5.1-1.el6.x86_64</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">glusterfs-cli-3.5.1-1.el6.x86_64</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">glusterfs-libs-3.5.1-1.el6.x86_64</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">glusterfs-fuse-3.5.1-1.el6.x86_64</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">glusterfs-server-3.5.1-1.el6.x86_64</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">glusterfs-api-3.5.1-1.el6.x86_64</div>
<div style="margin:
0px; font-size:
10px; font-family:
Monaco; color:
rgb(245, 245, 245);
background-color:
rgb(0, 0, 0);">[guru@srv2
~]$ </div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>I don’t see
anything odd here.
Please suggest.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Gurdeep.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>On 7 Jul 2014, at
9:06 pm, Niels de Vos
<<a
moz-do-not-send="true"
href="mailto:ndevos@redhat.com">ndevos@redhat.com</a>> wrote:</div>
<br
class="Apple-interchange-newline">
<blockquote type="cite">On
Sun, Jul 06, 2014 at
11:28:51PM +1000,
Gurdeep Singh (Guru)
wrote:<br>
<blockquote
type="cite">Hello,<br>
<br>
I have setup gluster
in replicate type
and its working
fine.<br>
<br>
I am seeing a
constant chatting
between the hosts
for lookup call and
<br>
lookup reply. I am
trying to understand
as to why this
traffic is <br>
being initiated
constantly. Please
look at the attached
image. This <br>
traffic is using
around 200KB/s of
constant bandwidth
and is <br>
exhausting our
allocated monthly
bandwidth on our 2
VPS.<br>
</blockquote>
<br>
You can use Wireshark
to identify which
process does the
LOOKUP calls. <br>
For this, do the
following:<br>
<br>
1. select a LOOKUP
Call<br>
2. enable the 'packet
details' pane (found
in the main menu,
'view')<br>
3. expand the
'Transmission Control
Protocol' tree<br>
4. check the 'Source
port' of the LOOKUP
Call<br>
<br>
Together with the
'Source' and the
'Source port' you can
go to the <br>
server that matches
the 'Source' address.
A command like this
would give <br>
you the PID of the
process in the right
column:<br>
<br>
# netstat -tpn | grep
$SOURCE_PORT<br>
<br>
And with 'ps -v $PID'
you can check which
process is responsible
for the <br>
LOOKUP. This process
can be a fuse-mount,
self-heal-daemon or
any other <br>
glusterfs-client.
Depending on the type
of client, you maybe
can tune <br>
the workload or other
options a little.<br>
<br>
In Wireshark you can
also check what
filename is LOOKUP'd,
just expand <br>
the 'GlusterFS' part
in the 'packet
details' and check the
'Basename'. <br>
Maybe this filename
(without directory
structure) does give
you any <br>
ideas of which
activity is causing
the LOOKUPs.<br>
<br>
HTH,<br>
Niels<br>
<br>
<blockquote
type="cite"><br>
The configuration I
have for Gluster is:<br>
<br>
[guru@srv1 ~]$ sudo
gluster volume info<br>
[sudo] password for
guru: <br>
<br>
Volume Name: gv0<br>
Type: Replicate<br>
Volume ID:
dc8dc3f2-f5bd-4047-9101-acad04695442<br>
Status: Started<br>
Number of Bricks: 1
x 2 = 2<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1:
srv1:/root/gluster-vol0<br>
Brick2:
srv2:/root/gluster-vol0<br>
Options
Reconfigured:<br>
cluster.lookup-unhashed:
on<br>
performance.cache-refresh-timeout:
60<br>
performance.cache-size:
1GB<br>
storage.health-check-interval:
30<br>
<br>
<br>
<br>
Please suggest how
to fine tune the RPC
calls/reply. <br>
</blockquote>
<br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset
class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>