<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div apple-content-edited="true"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Verdana; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><a href="http://pastie.org/3528212">http://pastie.org/3528212</a></div><div><br></div><div>SELinux/iptables info is below. FYI I started up the standard RedHat nfs server (with gluster shutdown), and it started cleanly and correctly bound to portmap/rpcbind.</div><div><br></div><div>[root@dresproddns02 ~]# getenforce </div><div>Permissive</div><div>[root@dresproddns02 ~]# iptables-save </div><div># Generated by iptables-save v1.4.7 on Mon Mar 5 14:34:27 2012</div><div>*filter</div><div>:INPUT ACCEPT [51197275114:5822299813362]</div><div>:FORWARD ACCEPT [0:0]</div><div>:OUTPUT ACCEPT [51184812127:5800516863347]</div><div>COMMIT</div><div># Completed on Mon Mar 5 14:34:27 2012</div><div># Generated by iptables-save v1.4.7 on Mon Mar 5 14:34:27 2012</div><div>*nat</div><div>:PREROUTING ACCEPT [663610:53184995]</div><div>:POSTROUTING ACCEPT [4471677:292235853]</div><div>:OUTPUT ACCEPT [4471677:292235853]</div><div>-A PREROUTING -d 172.31.0.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination 169.254.202.2 </div><div>-A PREROUTING -d 172.31.0.2/32 -p udp -m udp --dport 53 -j DNAT --to-destination 169.254.202.2 </div><div>COMMIT</div><div># Completed on Mon Mar 5 14:34:27 2012</div><div># Generated by iptables-save v1.4.7 on Mon Mar 5 14:34:27 2012</div><div>*mangle</div><div>:PREROUTING ACCEPT [330516307:119149868984]</div><div>:INPUT ACCEPT [330469748:119144996058]</div><div>:FORWARD ACCEPT [0:0]</div><div>:OUTPUT ACCEPT [319783522:99640884653]</div><div>:POSTROUTING ACCEPT [320799566:99775260122]</div><div>-A PREROUTING -d 172.31.0.0/32 -i bond0 -p tcp -m tcp --dport 5222 -j MARK --set-xmark 0x2bc/0xffffffff </div><div>-A PREROUTING -d 172.31.0.0/32 -i bond0 -p tcp -m tcp --dport 5262 -j MARK --set-xmark 0x2bd/0xffffffff </div><div>-A PREROUTING -d 172.31.0.0/32 -i bond0 -p tcp -m multiport --dports 9091,9091 -j MARK --set-xmark 0x2be/0xffffffff </div><div>-A PREROUTING -d 172.31.0.0/32 -i bond0 -p tcp -m tcp --dport 5222 -j MARK --set-xmark 0x2bc/0xffffffff </div><div>-A PREROUTING -d 172.31.0.0/32 -i bond0 -p tcp -m tcp --dport 5262 -j MARK --set-xmark 0x2bd/0xffffffff </div><div>-A PREROUTING -d 172.31.0.0/32 -i bond0 -p tcp -m multiport --dports 9091,9091 -j MARK --set-xmark 0x2be/0xffffffff </div><div>-A PREROUTING -d 172.31.0.64/32 -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x103/0xffffffff </div><div>COMMIT</div><div># Completed on Mon Mar 5 14:34:27 2012</div><div>[root@dresproddns02 ~]# </div><div><br></div></span><br class="Apple-interchange-newline">
</div>
<br><div><div>On Mar 5, 2012, at 2:05 PM, Bryan Whitehead wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Is selinux running? iptables?<br><br>Can you <a href="http://pastie.org/">http://pastie.org/</a> the nfs.log in /var/log/glusterfs ? <br><br><div class="gmail_quote">On Mon, Mar 5, 2012 at 3:59 AM, David Coulson <span dir="ltr"><<a href="mailto:david@davidcoulson.net">david@davidcoulson.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Yep.<br>
<br>
[root@dresproddns01 ~]# service glusterd stop<br>
Stopping glusterd: [ OK ]<div class="im"><br>
[root@dresproddns01 ~]# ps ax | grep nfs<br></div>
120494 pts/0 S+ 0:00 grep nfs<div class="im"><br>
2167119 ? S 0:00 [nfsiod]<br></div>
[root@dresproddns01 ~]# service rpcbind stop<br>
Stopping rpcbind: [ OK ]<div class="im"><br>
[root@dresproddns01 ~]# rpcinfo -p<br></div>
rpcinfo: can't contact portmapper: RPC: Remote system error - No
such file or directory<br>
[root@dresproddns01 ~]# service rpcbind start<br>
Starting rpcbind: [ OK ]<br>
[root@dresproddns01 ~]# service glusterd start<br>
Starting glusterd: [ OK ]<div class="im"><br>
[root@dresproddns01 ~]# rpcinfo -p<br>
program vers proto port service<br>
100000 4 tcp 111 portmapper<br>
100000 3 tcp 111 portmapper<br>
100000 2 tcp 111 portmapper<br>
100000 4 udp 111 portmapper<br>
100000 3 udp 111 portmapper<br>
100000 2 udp 111 portmapper<br>
<br></div>
Note that I waited a short while between the last two steps. FYI,
this is RHEL6 (the two systems that work are RHEL6 too, so I'm not
sure it matters much).<div><div class="h5"><br>
<br>
On 3/5/12 3:27 AM, Bryan Whitehead wrote:
<blockquote type="cite">did you start portmap service before you started
gluster?<br>
<br>
<div class="gmail_quote">On Sun, Mar 4, 2012 at 11:53 AM, David
Coulson <span dir="ltr"><<a href="mailto:david@davidcoulson.net" target="_blank">david@davidcoulson.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <tt>I've four systems
with multiple 4-way replica volumes. I'm migrating a
number of volumes from Fuse to NFS for performance
reasons.<br>
<br>
My first two hosts seem to work nicely, but the other two
won't start the NFS services properly. I looked through
the nfs.log, but it doesn't give any indication of why it
did not register with rpcbind. I'm presuming I've got a
misconfiguration on two of the systems, but there isn't a
clear indication of what is not working.<br>
<br>
Here is an example from a host which does not work:<br>
<br>
[root@dresproddns01 ~]# rpcinfo -p<br>
program vers proto port service<br>
100000 4 tcp 111 portmapper<br>
100000 3 tcp 111 portmapper<br>
100000 2 tcp 111 portmapper<br>
100000 4 udp 111 portmapper<br>
100000 3 udp 111 portmapper<br>
100000 2 udp 111 portmapper<br>
[root@dresproddns01 ~]# ps ax | grep nfs<br>
2167119 ? S 0:00 [nfsiod]<br>
2738268 ? Ssl 0:00
/opt/glusterfs/3.2.5/sbin/glusterfs -f
/etc/glusterd/nfs/nfs-server.vol -p
/etc/glusterd/nfs/run/nfs.pid -l
/var/log/glusterfs/nfs.log<br>
2934228 pts/0 S+ 0:00 grep nfs<br>
[root@dresproddns01 ~]# netstat -ntlp | grep 2738268<br>
tcp 0 0 <a href="http://0.0.0.0:38465/" target="_blank">0.0.0.0:38465</a>
0.0.0.0:* LISTEN
2738268/glusterfs <br>
tcp 0 0 <a href="http://0.0.0.0:38466/" target="_blank">0.0.0.0:38466</a>
0.0.0.0:* LISTEN
2738268/glusterfs <br>
tcp 0 0 <a href="http://0.0.0.0:38467/" target="_blank">0.0.0.0:38467</a>
0.0.0.0:* LISTEN 2738268/glusterfs
<br>
<br>
[root@dresproddns01 ~]# gluster volume info svn<br>
<br>
Volume Name: svn<br>
Type: Replicate<br>
Status: Started<br>
Number of Bricks: 4<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: rhesproddns01:/gluster/svn<br>
Brick2: rhesproddns02:/gluster/svn<br>
Brick3: dresproddns01:/gluster/svn<br>
Brick4: dresproddns02:/gluster/svn<br>
Options Reconfigured:<br>
performance.client-io-threads: 1<br>
performance.flush-behind: on<br>
network.ping-timeout: 5<br>
performance.stat-prefetch: 1<br>
nfs.disable: off<br>
nfs.register-with-portmap: on<br>
auth.allow: 10.250.53.*,10.252.248.*,169.254.*,127.0.0.1<br>
performance.cache-size: 256Mb<br>
performance.write-behind-window-size: 128Mb<br>
<br>
Only obvious difference with a host which does work is
this:<br>
<br>
[root@rhesproddns01 named]# rpcinfo -p<br>
program vers proto port service<br>
100000 4 tcp 111 portmapper<br>
100000 3 tcp 111 portmapper<br>
100000 2 tcp 111 portmapper<br>
100000 4 udp 111 portmapper<br>
100000 3 udp 111 portmapper<br>
100000 2 udp 111 portmapper<br>
100005 3 tcp 38465 mountd<br>
100005 1 tcp 38466 mountd<br>
100003 3 tcp 38467 nfs<br>
<br>
<br>
Any ideas where to look for errors?<br>
<br>
</tt> </div>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
<br>
</blockquote>
</div>
<br>
</blockquote>
</div></div></div>
</blockquote></div><br>
</blockquote></div><br></body></html>