<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hello Pranith,<div><br></div><div>Process 18629 is not sending any traffic across the servers.&nbsp;</div><div><br></div><div>The process that are constantly sending packet across is 1055 &amp; 18611.</div><div><br></div><div>If any, what is the interval of RPC lookup on one file? can we somehow control the lookup frequency?</div><div><br></div><div>Thanks,</div><div>Gurdeep.</div><div><br></div><div><br></div><div><div apple-content-edited="true">

</div>
<br><div><div>On 7 Jul 2014, at 11:59 pm, Pranith Kumar Karampuri &lt;<a href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>&gt; wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
  
    <meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
  
  <div text="#000000" bgcolor="#FFFFFF">
    <br>
    <div class="moz-cite-prefix">On 07/07/2014 07:22 PM, Gurdeep Singh
      (Guru) wrote:<br>
    </div>
    <blockquote cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au" type="cite">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      <div style="margin: 0px; font-size: 10px; font-family: Monaco;
        color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">[guru@srv2
        ~]$ ps -v 1055</div>
      <div style="margin: 0px; font-size: 10px; font-family: Monaco;
        color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">Warning:
        bad syntax, perhaps a bogus '-'? See
        /usr/share/doc/procps-3.2.8/FAQ</div>
      <div style="margin: 0px; font-size: 10px; font-family: Monaco;
        color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">&nbsp;
        PID TTY&nbsp; &nbsp; &nbsp; STAT &nbsp; TIME&nbsp; MAJFL &nbsp; TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
      <div style="margin: 0px; font-size: 10px; font-family: Monaco;
        color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">&nbsp;1055
        ?&nbsp; &nbsp; &nbsp; &nbsp; Ssl &nbsp; 86:01 &nbsp; &nbsp; 31 &nbsp; &nbsp; 0 319148 33092&nbsp; 3.2
        /usr/sbin/glusterfs --volfile-server=srv2 --volfile-id=/gv0
        /var/www/html/image/</div>
      <div style="margin: 0px; font-size: 10px; font-family: Monaco;
        color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);">[guru@srv2
        ~]$&nbsp;</div>
      <div><br>
      </div>
    </blockquote>
    Gurdeep,<br>
    &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Don't see anything odd here :-(. Mount is looking up files
    and brick is serving it. Why don't you keep a watch on the process
    '18629' and similar processes in the cluster. Do a ps aux | grep
    glustershd to get the pids. There will be one such process per
    machine in the cluster. Check how much it consumes. That is the only
    process which does operations without any operations on mounts.<br>
    <br>
    Pranith<br>
    <br>
    <blockquote cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au" type="cite">
      <div><br>
      </div>
      <div><br>
      </div>
      <div>
        <div>On 7 Jul 2014, at 11:49 pm, Pranith Kumar Karampuri &lt;<a moz-do-not-send="true" href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>&gt;
          wrote:</div>
        <br class="Apple-interchange-newline">
        <blockquote type="cite">
          <meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
          <div text="#000000" bgcolor="#FFFFFF"> <br>
            <div class="moz-cite-prefix">On 07/07/2014 07:03 PM, Gurdeep
              Singh (Guru) wrote:<br>
            </div>
            <blockquote cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au" type="cite">
              <meta http-equiv="Content-Type" content="text/html;
                charset=windows-1252">
              Hello Niels,
              <div><br>
              </div>
              <div>I did a net hogs on the interface to see what process
                might be using the bandwidth,</div>
              <div><br>
              </div>
              <div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">NetHogs version 0.8.0</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0); min-height: 14px;"><br>
                </div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; background-color: rgb(245, 245, 245);">&nbsp; PID
                  USER &nbsp; &nbsp; PROGRAM&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; DEV&nbsp; &nbsp; &nbsp; &nbsp; SENT&nbsp; &nbsp; &nbsp; RECEIVED&nbsp; &nbsp; &nbsp; &nbsp;</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">18611 root &nbsp; &nbsp; /usr/sbin/glusterfsd &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>&nbsp;
                  16.307&nbsp; &nbsp; &nbsp; 17.547 KB/sec</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">1055&nbsp; root &nbsp; &nbsp; /usr/sbin/glusterfs&nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>&nbsp;
                  17.249&nbsp; &nbsp; &nbsp; 16.259 KB/sec</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">13439 guru &nbsp; &nbsp; sshd: guru@pts/0 &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
                  &nbsp; 0.966 &nbsp; &nbsp; &nbsp; 0.051 KB/sec</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">18625 root &nbsp; &nbsp; /usr/sbin/glusterfs&nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
                  &nbsp; 0.000 &nbsp; &nbsp; &nbsp; 0.000 KB/sec</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">18629 root &nbsp; &nbsp; /usr/sbin/glusterfs&nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
                  &nbsp; 0.000 &nbsp; &nbsp; &nbsp; 0.000 KB/sec</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">9636&nbsp; root &nbsp; &nbsp; /usr/sbin/glusterd &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>
                  &nbsp; 0.000 &nbsp; &nbsp; &nbsp; 0.000 KB/sec</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">? &nbsp; &nbsp; root &nbsp; &nbsp; unknown TCP &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.000 &nbsp; &nbsp; &nbsp;
                  0.000 KB/sec</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0); min-height: 14px;"><br>
                </div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; background-color: rgb(245, 245, 245);">&nbsp; TOTAL
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 34.523&nbsp; &nbsp; &nbsp; 33.856 KB/sec&nbsp;</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0); min-height: 14px;"><br>
                </div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0); min-height: 14px;"><br>
                </div>
                <div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                </div>
              </div>
            </blockquote>
            Which process corresponds to '1055'?<br>
            <br>
            Pranith<br>
            <blockquote cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au" type="cite">
              <div>
                <div>
                  <div>Its glusterfs and glusterfsd process.</div>
                  <div><br>
                  </div>
                  <div>I looked at the capture file and see that the
                    lookup is being made on random files.</div>
                  <div><br>
                  </div>
                  <div>For PID information, please see this:</div>
                  <div><br>
                  </div>
                  <div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$
                      sudo netstat -tpn | grep 49152</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 127.0.0.1:49152 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:1012&nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 127.0.0.1:49152 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:1016&nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 127.0.0.1:1016&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:49152 &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18625/glusterfs&nbsp; &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 10.8.0.6:1021 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:49152&nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 1055/glusterfs &nbsp; &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 10.8.0.6:49152&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:1017 &nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 10.8.0.6:1020 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:49152&nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18629/glusterfs&nbsp; &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 127.0.0.1:1023&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:49152 &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18629/glusterfs&nbsp; &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 10.8.0.6:49152&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:1022 &nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 10.8.0.6:49152&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:1021 &nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 127.0.0.1:49152 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:1023&nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 127.0.0.1:1012&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:49152 &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 1055/glusterfs &nbsp; &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp;
                      0 10.8.0.6:1019 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:49152&nbsp; &nbsp; &nbsp;
                      &nbsp; &nbsp; &nbsp; &nbsp; ESTABLISHED 18625/glusterfs&nbsp; &nbsp; &nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$ ps
                      -v 18611</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">Warning: bad
                      syntax, perhaps a bogus '-'? See
                      /usr/share/doc/procps-3.2.8/FAQ</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp;
                      STAT &nbsp; TIME&nbsp; MAJFL &nbsp; TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">18611 ?&nbsp; &nbsp; &nbsp; &nbsp;
                      Ssl &nbsp; 14:12&nbsp; &nbsp; &nbsp; 0 &nbsp; &nbsp; 0 650068 20404&nbsp; 2.0
                      /usr/sbin/glusterfsd -s srv2 --volfile-id
                      gv0.srv2.root-gluster-vol0 -p
                      /var/lib/glusterd/vols/gv0</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$ ps
                      -v 18629</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">Warning: bad
                      syntax, perhaps a bogus '-'? See
                      /usr/share/doc/procps-3.2.8/FAQ</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp;
                      STAT &nbsp; TIME&nbsp; MAJFL &nbsp; TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">18629 ?&nbsp; &nbsp; &nbsp; &nbsp;
                      Ssl&nbsp; &nbsp; 0:04&nbsp; &nbsp; &nbsp; 0 &nbsp; &nbsp; 0 333296 17380&nbsp; 1.7
                      /usr/sbin/glusterfs -s localhost --volfile-id
                      gluster/glustershd -p
                      /var/lib/glusterd/glustershd/r</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$ ps
                      -v 18629</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">Warning: bad
                      syntax, perhaps a bogus '-'? See
                      /usr/share/doc/procps-3.2.8/FAQ</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp;
                      STAT &nbsp; TIME&nbsp; MAJFL &nbsp; TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">18629 ?&nbsp; &nbsp; &nbsp; &nbsp;
                      Ssl&nbsp; &nbsp; 0:04&nbsp; &nbsp; &nbsp; 0 &nbsp; &nbsp; 0 333296 17380&nbsp; 1.7
                      /usr/sbin/glusterfs -s localhost --volfile-id
                      gluster/glustershd -p
                      /var/lib/glusterd/glustershd/run/glustershd.pid -l
                      /var/log/glusterfs/glustershd.log -S
                      /var/run/823fa3197e2d1841be888</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$ ps
                      -v 18629</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">Warning: bad
                      syntax, perhaps a bogus '-'? See
                      /usr/share/doc/procps-3.2.8/FAQ</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp;
                      STAT &nbsp; TIME&nbsp; MAJFL &nbsp; TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">18629 ?&nbsp; &nbsp; &nbsp; &nbsp;
                      Ssl&nbsp; &nbsp; 0:04&nbsp; &nbsp; &nbsp; 0 &nbsp; &nbsp; 0 333296 17380&nbsp; 1.7
                      /usr/sbin/glusterfs -s localhost --volfile-id
                      gluster/glustershd -p
                      /var/lib/glusterd/glustershd/run/glustershd.pid -l
                      /var/log/glusterfs/glustershd.log -S
                      /var/run/823fa3197e2d1841be8881500723b063.socket
                      --xlator-option
                      *replicate*.node-uuid=84af83c9-0a29-</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$ ps
                      -v 18625</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">Warning: bad
                      syntax, perhaps a bogus '-'? See
                      /usr/share/doc/procps-3.2.8/FAQ</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp;
                      STAT &nbsp; TIME&nbsp; MAJFL &nbsp; TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">18625 ?&nbsp; &nbsp; &nbsp; &nbsp;
                      Ssl&nbsp; &nbsp; 0:03&nbsp; &nbsp; &nbsp; 0 &nbsp; &nbsp; 0 239528 41040&nbsp; 4.0
                      /usr/sbin/glusterfs -s localhost --volfile-id
                      gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid
                      -l /var/log/glusterfs/nfs.log -S
                      /var/run/5ad5b036fd636cc5dddffa73593e4089.socket</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$
                      sudo nethogs tun0</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">Waiting for first
                      packet to arrive (see <a moz-do-not-send="true" href="http://sourceforge.net/">sourceforge.net</a>
                      bug 1019381)</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$
                      rpm -qa | grep gluster</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">glusterfs-3.5.1-1.el6.x86_64</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">glusterfs-cli-3.5.1-1.el6.x86_64</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">glusterfs-libs-3.5.1-1.el6.x86_64</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">glusterfs-fuse-3.5.1-1.el6.x86_64</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">glusterfs-server-3.5.1-1.el6.x86_64</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">glusterfs-api-3.5.1-1.el6.x86_64</div>
                    <div style="margin: 0px; font-size: 10px;
                      font-family: Monaco; color: rgb(245, 245, 245);
                      background-color: rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div>I don’t see anything odd here. Please suggest.</div>
                  <div><br>
                  </div>
                  <div>Thanks,</div>
                  <div>Gurdeep.</div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div>On 7 Jul 2014, at 9:06 pm, Niels de Vos &lt;<a moz-do-not-send="true" href="mailto:ndevos@redhat.com">ndevos@redhat.com</a>&gt;

                    wrote:</div>
                  <br class="Apple-interchange-newline">
                  <blockquote type="cite">On Sun, Jul 06, 2014 at
                    11:28:51PM +1000, Gurdeep Singh (Guru) wrote:<br>
                    <blockquote type="cite">Hello,<br>
                      <br>
                      I have setup gluster in replicate type and its
                      working fine.<br>
                      <br>
                      I am seeing a constant chatting between the hosts
                      for lookup call and <br>
                      lookup reply. I am trying to understand as to why
                      this traffic is <br>
                      being initiated constantly. Please look at the
                      attached image. This <br>
                      traffic is using around 200KB/s of constant
                      bandwidth and is <br>
                      exhausting our allocated monthly bandwidth on our
                      2 VPS.<br>
                    </blockquote>
                    <br>
                    You can use Wireshark to identify which process does
                    the LOOKUP calls. &nbsp;<br>
                    For this, do the following:<br>
                    <br>
                    1. select a LOOKUP Call<br>
                    2. enable the 'packet details' pane (found in the
                    main menu, 'view')<br>
                    3. expand the 'Transmission Control Protocol' tree<br>
                    4. check the 'Source port' of the LOOKUP Call<br>
                    <br>
                    Together with the 'Source' and the 'Source port' you
                    can go to the <br>
                    server that matches the 'Source' address. A command
                    like this would give <br>
                    you the PID of the process in the right column:<br>
                    <br>
                    &nbsp;# netstat -tpn | grep $SOURCE_PORT<br>
                    <br>
                    And with 'ps -v $PID' you can check which process is
                    responsible for the <br>
                    LOOKUP. This process can be a fuse-mount,
                    self-heal-daemon or any other <br>
                    glusterfs-client. Depending on the type of client,
                    you maybe can tune <br>
                    the workload or other options a little.<br>
                    <br>
                    In Wireshark you can also check what filename is
                    LOOKUP'd, just expand <br>
                    the 'GlusterFS' part in the 'packet details' and
                    check the 'Basename'. &nbsp;<br>
                    Maybe this filename (without directory structure)
                    does give you any <br>
                    ideas of which activity is causing the LOOKUPs.<br>
                    <br>
                    HTH,<br>
                    Niels<br>
                    <br>
                    <blockquote type="cite"><br>
                      The configuration I have for Gluster is:<br>
                      <br>
                      [guru@srv1 ~]$ sudo gluster volume info<br>
                      [sudo] password for guru: <br>
                      <br>
                      Volume Name: gv0<br>
                      Type: Replicate<br>
                      Volume ID: dc8dc3f2-f5bd-4047-9101-acad04695442<br>
                      Status: Started<br>
                      Number of Bricks: 1 x 2 = 2<br>
                      Transport-type: tcp<br>
                      Bricks:<br>
                      Brick1: srv1:/root/gluster-vol0<br>
                      Brick2: srv2:/root/gluster-vol0<br>
                      Options Reconfigured:<br>
                      cluster.lookup-unhashed: on<br>
                      performance.cache-refresh-timeout: 60<br>
                      performance.cache-size: 1GB<br>
                      storage.health-check-interval: 30<br>
                      <br>
                      <br>
                      <br>
                      Please suggest how to fine tune the RPC
                      calls/reply. <br>
                    </blockquote>
                    <br>
                    <br>
                  </blockquote>
                </div>
                <br>
              </div>
              <br>
              <fieldset class="mimeAttachmentHeader"></fieldset>
              <br>
              <pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
            </blockquote>
            <br>
          </div>
        </blockquote>
      </div>
      <br>
    </blockquote>
    <br>
  </div>

</blockquote></div><br></div></body></html>