<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I poke around at this today and figured it out - Sort of.<br>
<br>
When I did 'gluster volume set named nfs.register-with-portmap on'
originally, I only had rpcbind running on two of my four servers.
Gluster nfs started up on all four, but obviously only two correctly
connected with rpcbind/portmap. Seems that if rpcbind is not running
when you set 'register-with-portmap on', even when you cycle gluster
it still doesn't work.<br>
<br>
So, I started up rpcbind, did a 'register-with-portmap off',
followed by 'register-with-portmap on' and it works now.<br>
<br>
When I diffed the before and after nfs-server.vol files, I see this:<br>
<br>
[root@dresproddns01 ~]# diff nfs-vol-1
/etc/glusterd/nfs/nfs-server.vol <br>
143c143<br>
< option rpc.register-with-portmap off<br>
---<br>
> option rpc.register-with-portmap on<br>
<br>
<br>
Apparently if rpcbind is not running, the option does not get
enabled properly. There is an error in nfs.log, but it's hard to
find especially if the node you manage the cluster from isn't the
node with the issue. It isn't clear either that it's broken even if
you cycle gluster (and even though the gluster volume configuration
says 'register-with-portmap on'. Does the 'gluster volume set'
command have the ability to get success/fail information back from
each node? It also appears that 'register-with-portmap' is applied
to all volumes, even if you just enable it on one - Is there a
cluster-wide place to 'set' options?<br>
<br>
[2012-03-05 19:51:22.517368] E
[rpcsvc.c:2771:nfs_rpcsvc_program_register_portmap] 0-nfsrpc: Could
not register with portmap<br>
[2012-03-05 19:51:22.517420] E
[rpcsvc.c:2861:nfs_rpcsvc_program_register] 0-nfsrpc: portmap
registration of program failed<br>
[2012-03-05 19:51:22.517428] E
[rpcsvc.c:2874:nfs_rpcsvc_program_register] 0-nfsrpc: Program
registration failed: MOUNT3, Num: 100005, Ver: 3, Port: 38465<br>
<br>
<br>
David<br>
<br>
<br>
On 3/5/12 2:05 PM, Bryan Whitehead wrote:
<blockquote
cite="mid:CAA3XVTwaMkDTynajU_j+exyZxgffrbCbsONLJD+qJ1am9AUj0w@mail.gmail.com"
type="cite">Is selinux running? iptables?<br>
<br>
Can you <a moz-do-not-send="true" href="http://pastie.org/">http://pastie.org/</a>
the nfs.log in /var/log/glusterfs ? <br>
<br>
<div class="gmail_quote">On Mon, Mar 5, 2012 at 3:59 AM, David
Coulson <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:david@davidcoulson.net">david@davidcoulson.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Yep.<br>
<br>
[root@dresproddns01 ~]# service glusterd stop<br>
Stopping glusterd:
[ OK ]
<div class="im"><br>
[root@dresproddns01 ~]# ps ax | grep nfs<br>
</div>
120494 pts/0 S+ 0:00 grep nfs
<div class="im"><br>
2167119 ? S 0:00 [nfsiod]<br>
</div>
[root@dresproddns01 ~]# service rpcbind stop<br>
Stopping rpcbind:
[ OK ]
<div class="im"><br>
[root@dresproddns01 ~]# rpcinfo -p<br>
</div>
rpcinfo: can't contact portmapper: RPC: Remote system error
- No such file or directory<br>
[root@dresproddns01 ~]# service rpcbind start<br>
Starting rpcbind:
[ OK ]<br>
[root@dresproddns01 ~]# service glusterd start<br>
Starting glusterd:
[ OK ]
<div class="im"><br>
[root@dresproddns01 ~]# rpcinfo -p<br>
program vers proto port service<br>
100000 4 tcp 111 portmapper<br>
100000 3 tcp 111 portmapper<br>
100000 2 tcp 111 portmapper<br>
100000 4 udp 111 portmapper<br>
100000 3 udp 111 portmapper<br>
100000 2 udp 111 portmapper<br>
<br>
</div>
Note that I waited a short while between the last two steps.
FYI, this is RHEL6 (the two systems that work are RHEL6 too,
so I'm not sure it matters much).
<div>
<div class="h5"><br>
<br>
On 3/5/12 3:27 AM, Bryan Whitehead wrote:
<blockquote type="cite">did you start portmap service
before you started gluster?<br>
<br>
<div class="gmail_quote">On Sun, Mar 4, 2012 at 11:53
AM, David Coulson <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:david@davidcoulson.net"
target="_blank">david@davidcoulson.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <tt>I've
four systems with multiple 4-way replica
volumes. I'm migrating a number of volumes
from Fuse to NFS for performance reasons.<br>
<br>
My first two hosts seem to work nicely, but
the other two won't start the NFS services
properly. I looked through the nfs.log, but it
doesn't give any indication of why it did not
register with rpcbind. I'm presuming I've got
a misconfiguration on two of the systems, but
there isn't a clear indication of what is not
working.<br>
<br>
Here is an example from a host which does not
work:<br>
<br>
[root@dresproddns01 ~]# rpcinfo -p<br>
program vers proto port service<br>
100000 4 tcp 111 portmapper<br>
100000 3 tcp 111 portmapper<br>
100000 2 tcp 111 portmapper<br>
100000 4 udp 111 portmapper<br>
100000 3 udp 111 portmapper<br>
100000 2 udp 111 portmapper<br>
[root@dresproddns01 ~]# ps ax | grep nfs<br>
2167119 ? S 0:00 [nfsiod]<br>
2738268 ? Ssl 0:00
/opt/glusterfs/3.2.5/sbin/glusterfs -f
/etc/glusterd/nfs/nfs-server.vol -p
/etc/glusterd/nfs/run/nfs.pid -l
/var/log/glusterfs/nfs.log<br>
2934228 pts/0 S+ 0:00 grep nfs<br>
[root@dresproddns01 ~]# netstat -ntlp | grep
2738268<br>
tcp 0 0 <a moz-do-not-send="true"
href="http://0.0.0.0:38465" target="_blank">0.0.0.0:38465</a>
0.0.0.0:* LISTEN
2738268/glusterfs <br>
tcp 0 0 <a moz-do-not-send="true"
href="http://0.0.0.0:38466" target="_blank">0.0.0.0:38466</a>
0.0.0.0:* LISTEN
2738268/glusterfs <br>
tcp 0 0 <a moz-do-not-send="true"
href="http://0.0.0.0:38467" target="_blank">0.0.0.0:38467</a>
0.0.0.0:* LISTEN
2738268/glusterfs <br>
<br>
[root@dresproddns01 ~]# gluster volume info
svn<br>
<br>
Volume Name: svn<br>
Type: Replicate<br>
Status: Started<br>
Number of Bricks: 4<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: rhesproddns01:/gluster/svn<br>
Brick2: rhesproddns02:/gluster/svn<br>
Brick3: dresproddns01:/gluster/svn<br>
Brick4: dresproddns02:/gluster/svn<br>
Options Reconfigured:<br>
performance.client-io-threads: 1<br>
performance.flush-behind: on<br>
network.ping-timeout: 5<br>
performance.stat-prefetch: 1<br>
nfs.disable: off<br>
nfs.register-with-portmap: on<br>
auth.allow:
10.250.53.*,10.252.248.*,169.254.*,127.0.0.1<br>
performance.cache-size: 256Mb<br>
performance.write-behind-window-size: 128Mb<br>
<br>
Only obvious difference with a host which does
work is this:<br>
<br>
[root@rhesproddns01 named]# rpcinfo -p<br>
program vers proto port service<br>
100000 4 tcp 111 portmapper<br>
100000 3 tcp 111 portmapper<br>
100000 2 tcp 111 portmapper<br>
100000 4 udp 111 portmapper<br>
100000 3 udp 111 portmapper<br>
100000 2 udp 111 portmapper<br>
100005 3 tcp 38465 mountd<br>
100005 1 tcp 38466 mountd<br>
100003 3 tcp 38467 nfs<br>
<br>
<br>
Any ideas where to look for errors?<br>
<br>
</tt> </div>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org"
target="_blank">Gluster-users@gluster.org</a><br>
<a moz-do-not-send="true"
href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users"
target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
<br>
</blockquote>
</div>
<br>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
</body>
</html>