<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div><div></div><div><br></div><div>Thanks,</div><div>Gurdeep.</div><div><br></div><div><br></div><div><div>On 8 Jul 2014, at 12:12 am, Pranith Kumar Karampuri &lt;<a href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>&gt; wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
  
    <meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
  
  <div text="#000000" bgcolor="#FFFFFF">
    <br>
    <div class="moz-cite-prefix">On 07/07/2014 07:39 PM, Gurdeep Singh
      (Guru) wrote:<br>
    </div>
    <blockquote cite="mid:F94DEE44-1473-43DD-9908-C71D91BB27FC@bazaari.com.au" type="cite">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      Hello Pranith,
      <div><br>
      </div>
      <div>Process 18629 is not sending any traffic across the servers.&nbsp;</div>
      <div><br>
      </div>
      <div>The process that are constantly sending packet across is 1055
        &amp; 18611.</div>
    </blockquote>
    In that case it is the application which is sending the traffic.
    Niels just looked into the pcap file and even he found the process
    with pid 14927 to be the one sending the traffic. Could you check
    what process it is?<br>
    <br>
    Pranith.<br>
    <blockquote cite="mid:F94DEE44-1473-43DD-9908-C71D91BB27FC@bazaari.com.au" type="cite">
      <div><br>
      </div>
      <div>If any, what is the interval of RPC lookup on one file? can
        we somehow control the lookup frequency?</div>
      <div><br>
      </div>
      <div>Thanks,</div>
      <div>Gurdeep.</div>
      <div><br>
      </div>
      <div><br>
      </div>
      <div>
        <div apple-content-edited="true">
        </div>
        <br>
        <div>
          <div>On 7 Jul 2014, at 11:59 pm, Pranith Kumar Karampuri &lt;<a moz-do-not-send="true" href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>&gt;
            wrote:</div>
          <br class="Apple-interchange-newline">
          <blockquote type="cite">
            <meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
            <div text="#000000" bgcolor="#FFFFFF"> <br>
              <div class="moz-cite-prefix">On 07/07/2014 07:22 PM,
                Gurdeep Singh (Guru) wrote:<br>
              </div>
              <blockquote cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au" type="cite">
                <meta http-equiv="Content-Type" content="text/html;
                  charset=windows-1252">
                <meta http-equiv="Content-Type" content="text/html;
                  charset=windows-1252">
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">[guru@srv2 ~]$ ps -v 1055</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">Warning: bad syntax, perhaps a bogus
                  '-'? See /usr/share/doc/procps-3.2.8/FAQ</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp; STAT &nbsp; TIME&nbsp; MAJFL &nbsp; TRS
                  &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">&nbsp;1055 ?&nbsp; &nbsp; &nbsp; &nbsp; Ssl &nbsp; 86:01 &nbsp; &nbsp; 31 &nbsp; &nbsp; 0
                  319148 33092&nbsp; 3.2 /usr/sbin/glusterfs
                  --volfile-server=srv2 --volfile-id=/gv0
                  /var/www/html/image/</div>
                <div style="margin: 0px; font-size: 10px; font-family:
                  Monaco; color: rgb(245, 245, 245); background-color:
                  rgb(0, 0, 0);">[guru@srv2 ~]$&nbsp;</div>
                <div><br>
                </div>
              </blockquote>
              Gurdeep,<br>
              &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Don't see anything odd here :-(. Mount is looking
              up files and brick is serving it. Why don't you keep a
              watch on the process '18629' and similar processes in the
              cluster. Do a ps aux | grep glustershd to get the pids.
              There will be one such process per machine in the cluster.
              Check how much it consumes. That is the only process which
              does operations without any operations on mounts.<br>
              <br>
              Pranith<br>
              <br>
              <blockquote cite="mid:32AD0B84-7C38-4F5F-BEFE-76F72F7F2A43@bazaari.com.au" type="cite">
                <div><br>
                </div>
                <div><br>
                </div>
                <div>
                  <div>On 7 Jul 2014, at 11:49 pm, Pranith Kumar
                    Karampuri &lt;<a moz-do-not-send="true" href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>&gt;

                    wrote:</div>
                  <br class="Apple-interchange-newline">
                  <blockquote type="cite">
                    <meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
                    <div text="#000000" bgcolor="#FFFFFF"> <br>
                      <div class="moz-cite-prefix">On 07/07/2014 07:03
                        PM, Gurdeep Singh (Guru) wrote:<br>
                      </div>
                      <blockquote cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au" type="cite">
                        <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
                        Hello Niels,
                        <div><br>
                        </div>
                        <div>I did a net hogs on the interface to see
                          what process might be using the bandwidth,</div>
                        <div><br>
                        </div>
                        <div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">NetHogs
                            version 0.8.0</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);
                            min-height: 14px;"><br>
                          </div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; background-color:
                            rgb(245, 245, 245);">&nbsp; PID USER &nbsp; &nbsp; PROGRAM&nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; DEV&nbsp; &nbsp; &nbsp; &nbsp; SENT&nbsp; &nbsp; &nbsp; RECEIVED&nbsp; &nbsp; &nbsp; &nbsp;</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">18611
                            root &nbsp; &nbsp; /usr/sbin/glusterfsd &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>&nbsp; 16.307&nbsp;
                            &nbsp; &nbsp; 17.547 KB/sec</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">1055&nbsp;
                            root &nbsp; &nbsp; /usr/sbin/glusterfs&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span>&nbsp; 17.249&nbsp;
                            &nbsp; &nbsp; 16.259 KB/sec</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">13439
                            guru &nbsp; &nbsp; sshd: guru@pts/0 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span> &nbsp; 0.966
                            &nbsp; &nbsp; &nbsp; 0.051 KB/sec</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">18625
                            root &nbsp; &nbsp; /usr/sbin/glusterfs&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span> &nbsp; 0.000
                            &nbsp; &nbsp; &nbsp; 0.000 KB/sec</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">18629
                            root &nbsp; &nbsp; /usr/sbin/glusterfs&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span> &nbsp; 0.000
                            &nbsp; &nbsp; &nbsp; 0.000 KB/sec</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">9636&nbsp;
                            root &nbsp; &nbsp; /usr/sbin/glusterd &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; tun0<span class="Apple-tab-span" style="white-space:pre"> </span> &nbsp; 0.000
                            &nbsp; &nbsp; &nbsp; 0.000 KB/sec</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);">? &nbsp; &nbsp;
                            root &nbsp; &nbsp; unknown TCP &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.000 &nbsp;
                            &nbsp; &nbsp; 0.000 KB/sec</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);
                            min-height: 14px;"><br>
                          </div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; background-color:
                            rgb(245, 245, 245);">&nbsp; TOTAL &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                            &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 34.523&nbsp; &nbsp; &nbsp; 33.856 KB/sec&nbsp;</div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);
                            min-height: 14px;"><br>
                          </div>
                          <div style="margin: 0px; font-size: 10px;
                            font-family: Monaco; color: rgb(245, 245,
                            245); background-color: rgb(0, 0, 0);
                            min-height: 14px;"><br>
                          </div>
                          <div>
                            <div><br>
                            </div>
                            <div><br>
                            </div>
                          </div>
                        </div>
                      </blockquote>
                      Which process corresponds to '1055'?<br>
                      <br>
                      Pranith<br>
                      <blockquote cite="mid:19E47DE9-3A91-47EC-8083-3E8E2574A045@bazaari.com.au" type="cite">
                        <div>
                          <div>
                            <div>Its glusterfs and glusterfsd process.</div>
                            <div><br>
                            </div>
                            <div>I looked at the capture file and see
                              that the lookup is being made on random
                              files.</div>
                            <div><br>
                            </div>
                            <div>For PID information, please see this:</div>
                            <div><br>
                            </div>
                            <div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ sudo netstat -tpn |
                                grep 49152</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 127.0.0.1:49152
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:1012&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 127.0.0.1:49152
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:1016&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 127.0.0.1:1016&nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:49152 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18625/glusterfs&nbsp; &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 10.8.0.6:1021 &nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:49152&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 1055/glusterfs &nbsp; &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 10.8.0.6:49152&nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:1017 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 10.8.0.6:1020 &nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:49152&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18629/glusterfs&nbsp; &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 127.0.0.1:1023&nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:49152 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18629/glusterfs&nbsp; &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 10.8.0.6:49152&nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:1022 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 10.8.0.6:49152&nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:1021 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 127.0.0.1:49152
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:1023&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18611/glusterfsd &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 127.0.0.1:1012&nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 127.0.0.1:49152 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 1055/glusterfs &nbsp; &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">tcp&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; 0 10.8.0.6:1019 &nbsp;
                                &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 10.8.0.1:49152&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                ESTABLISHED 18625/glusterfs&nbsp; &nbsp; &nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ ps -v 18611</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">Warning: bad syntax, perhaps a
                                bogus '-'? See
                                /usr/share/doc/procps-3.2.8/FAQ</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp; STAT &nbsp; TIME&nbsp; MAJFL &nbsp;
                                TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">18611 ?&nbsp; &nbsp; &nbsp; &nbsp; Ssl &nbsp; 14:12&nbsp; &nbsp; &nbsp; 0 &nbsp;
                                &nbsp; 0 650068 20404&nbsp; 2.0
                                /usr/sbin/glusterfsd -s srv2
                                --volfile-id gv0.srv2.root-gluster-vol0
                                -p /var/lib/glusterd/vols/gv0</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ ps -v 18629</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">Warning: bad syntax, perhaps a
                                bogus '-'? See
                                /usr/share/doc/procps-3.2.8/FAQ</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp; STAT &nbsp; TIME&nbsp; MAJFL &nbsp;
                                TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">18629 ?&nbsp; &nbsp; &nbsp; &nbsp; Ssl&nbsp; &nbsp; 0:04&nbsp; &nbsp; &nbsp; 0 &nbsp;
                                &nbsp; 0 333296 17380&nbsp; 1.7
                                /usr/sbin/glusterfs -s localhost
                                --volfile-id gluster/glustershd -p
                                /var/lib/glusterd/glustershd/r</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ ps -v 18629</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">Warning: bad syntax, perhaps a
                                bogus '-'? See
                                /usr/share/doc/procps-3.2.8/FAQ</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp; STAT &nbsp; TIME&nbsp; MAJFL &nbsp;
                                TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">18629 ?&nbsp; &nbsp; &nbsp; &nbsp; Ssl&nbsp; &nbsp; 0:04&nbsp; &nbsp; &nbsp; 0 &nbsp;
                                &nbsp; 0 333296 17380&nbsp; 1.7
                                /usr/sbin/glusterfs -s localhost
                                --volfile-id gluster/glustershd -p
                                /var/lib/glusterd/glustershd/run/glustershd.pid
                                -l /var/log/glusterfs/glustershd.log -S
                                /var/run/823fa3197e2d1841be888</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ ps -v 18629</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">Warning: bad syntax, perhaps a
                                bogus '-'? See
                                /usr/share/doc/procps-3.2.8/FAQ</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp; STAT &nbsp; TIME&nbsp; MAJFL &nbsp;
                                TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">18629 ?&nbsp; &nbsp; &nbsp; &nbsp; Ssl&nbsp; &nbsp; 0:04&nbsp; &nbsp; &nbsp; 0 &nbsp;
                                &nbsp; 0 333296 17380&nbsp; 1.7
                                /usr/sbin/glusterfs -s localhost
                                --volfile-id gluster/glustershd -p
                                /var/lib/glusterd/glustershd/run/glustershd.pid
                                -l /var/log/glusterfs/glustershd.log -S
                                /var/run/823fa3197e2d1841be8881500723b063.socket

                                --xlator-option
                                *replicate*.node-uuid=84af83c9-0a29-</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ ps -v 18625</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">Warning: bad syntax, perhaps a
                                bogus '-'? See
                                /usr/share/doc/procps-3.2.8/FAQ</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">&nbsp; PID TTY&nbsp; &nbsp; &nbsp; STAT &nbsp; TIME&nbsp; MAJFL &nbsp;
                                TRS &nbsp; DRS &nbsp; RSS %MEM COMMAND</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">18625 ?&nbsp; &nbsp; &nbsp; &nbsp; Ssl&nbsp; &nbsp; 0:03&nbsp; &nbsp; &nbsp; 0 &nbsp;
                                &nbsp; 0 239528 41040&nbsp; 4.0
                                /usr/sbin/glusterfs -s localhost
                                --volfile-id gluster/nfs -p
                                /var/lib/glusterd/nfs/run/nfs.pid -l
                                /var/log/glusterfs/nfs.log -S
                                /var/run/5ad5b036fd636cc5dddffa73593e4089.socket</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ sudo nethogs tun0</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">Waiting for first packet to arrive
                                (see <a moz-do-not-send="true" href="http://sourceforge.net/">sourceforge.net</a>
                                bug 1019381)</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$ rpm -qa | grep
                                gluster</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">glusterfs-3.5.1-1.el6.x86_64</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">glusterfs-cli-3.5.1-1.el6.x86_64</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">glusterfs-libs-3.5.1-1.el6.x86_64</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">glusterfs-fuse-3.5.1-1.el6.x86_64</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">glusterfs-server-3.5.1-1.el6.x86_64</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">glusterfs-api-3.5.1-1.el6.x86_64</div>
                              <div style="margin: 0px; font-size: 10px;
                                font-family: Monaco; color: rgb(245,
                                245, 245); background-color: rgb(0, 0,
                                0);">[guru@srv2 ~]$&nbsp;</div>
                            </div>
                            <div><br>
                            </div>
                            <div><br>
                            </div>
                            <div>I don’t see anything odd here. Please
                              suggest.</div>
                            <div><br>
                            </div>
                            <div>Thanks,</div>
                            <div>Gurdeep.</div>
                            <div><br>
                            </div>
                            <div><br>
                            </div>
                            <div><br>
                            </div>
                            <div><br>
                            </div>
                            <div><br>
                            </div>
                            <div>On 7 Jul 2014, at 9:06 pm, Niels de Vos
                              &lt;<a moz-do-not-send="true" href="mailto:ndevos@redhat.com">ndevos@redhat.com</a>&gt;


                              wrote:</div>
                            <br class="Apple-interchange-newline">
                            <blockquote type="cite">On Sun, Jul 06, 2014
                              at 11:28:51PM +1000, Gurdeep Singh (Guru)
                              wrote:<br>
                              <blockquote type="cite">Hello,<br>
                                <br>
                                I have setup gluster in replicate type
                                and its working fine.<br>
                                <br>
                                I am seeing a constant chatting between
                                the hosts for lookup call and <br>
                                lookup reply. I am trying to understand
                                as to why this traffic is <br>
                                being initiated constantly. Please look
                                at the attached image. This <br>
                                traffic is using around 200KB/s of
                                constant bandwidth and is <br>
                                exhausting our allocated monthly
                                bandwidth on our 2 VPS.<br>
                              </blockquote>
                              <br>
                              You can use Wireshark to identify which
                              process does the LOOKUP calls. &nbsp;<br>
                              For this, do the following:<br>
                              <br>
                              1. select a LOOKUP Call<br>
                              2. enable the 'packet details' pane (found
                              in the main menu, 'view')<br>
                              3. expand the 'Transmission Control
                              Protocol' tree<br>
                              4. check the 'Source port' of the LOOKUP
                              Call<br>
                              <br>
                              Together with the 'Source' and the 'Source
                              port' you can go to the <br>
                              server that matches the 'Source' address.
                              A command like this would give <br>
                              you the PID of the process in the right
                              column:<br>
                              <br>
                              &nbsp;# netstat -tpn | grep $SOURCE_PORT<br>
                              <br>
                              And with 'ps -v $PID' you can check which
                              process is responsible for the <br>
                              LOOKUP. This process can be a fuse-mount,
                              self-heal-daemon or any other <br>
                              glusterfs-client. Depending on the type of
                              client, you maybe can tune <br>
                              the workload or other options a little.<br>
                              <br>
                              In Wireshark you can also check what
                              filename is LOOKUP'd, just expand <br>
                              the 'GlusterFS' part in the 'packet
                              details' and check the 'Basename'. &nbsp;<br>
                              Maybe this filename (without directory
                              structure) does give you any <br>
                              ideas of which activity is causing the
                              LOOKUPs.<br>
                              <br>
                              HTH,<br>
                              Niels<br>
                              <br>
                              <blockquote type="cite"><br>
                                The configuration I have for Gluster is:<br>
                                <br>
                                [guru@srv1 ~]$ sudo gluster volume info<br>
                                [sudo] password for guru: <br>
                                <br>
                                Volume Name: gv0<br>
                                Type: Replicate<br>
                                Volume ID:
                                dc8dc3f2-f5bd-4047-9101-acad04695442<br>
                                Status: Started<br>
                                Number of Bricks: 1 x 2 = 2<br>
                                Transport-type: tcp<br>
                                Bricks:<br>
                                Brick1: srv1:/root/gluster-vol0<br>
                                Brick2: srv2:/root/gluster-vol0<br>
                                Options Reconfigured:<br>
                                cluster.lookup-unhashed: on<br>
                                performance.cache-refresh-timeout: 60<br>
                                performance.cache-size: 1GB<br>
                                storage.health-check-interval: 30<br>
                                <br>
                                <br>
                                <br>
                                Please suggest how to fine tune the RPC
                                calls/reply. <br>
                              </blockquote>
                              <br>
                              <br>
                            </blockquote>
                          </div>
                          <br>
                        </div>
                        <br>
                        <fieldset class="mimeAttachmentHeader"></fieldset>
                        <br>
                        <pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
                      </blockquote>
                      <br>
                    </div>
                  </blockquote>
                </div>
                <br>
              </blockquote>
              <br>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
  </div>

</blockquote></div><br></div></body></html>