<font size=2 face="sans-serif">Hi,</font>
<br>
<br><font size=2 face="sans-serif">we got a strange security issue when
connecting a Solaris NFS client to Gluster volumes</font>
<br><font size=2 face="sans-serif">Initially, we tried to share a volume
between a Linux Client (10.1.99.200) and a Solaris Client (10.1.99.201)</font>
<br>
<br><font size=2 face="sans-serif">We create this volume </font>
<br>
<br><font size=2 face="sans-serif">[root@llsmagfs001a glusterfs]# gluster
volume info vol1</font>
<br>
<br><font size=2 face="sans-serif">Volume Name: vol1</font>
<br><font size=2 face="sans-serif">Type: Distribute</font>
<br><font size=2 face="sans-serif">Volume ID: 4abcee08-6172-441a-851b-53becb77c281</font>
<br><font size=2 face="sans-serif">Status: Started</font>
<br><font size=2 face="sans-serif">Number of Bricks: 1</font>
<br><font size=2 face="sans-serif">Transport-type: tcp</font>
<br><font size=2 face="sans-serif">Bricks:</font>
<br><font size=2 face="sans-serif">Brick1: llsmagfs001a.cloud.testsc.sc:/export/vol1</font>
<br><font size=2 face="sans-serif">Options Reconfigured:</font>
<br><font size=2 face="sans-serif">diagnostics.client-log-level: DEBUG</font>
<br><font size=2 face="sans-serif">diagnostics.brick-log-level: DEBUG</font>
<br><font size=2 face="sans-serif">auth.allow: 10.1.99.200</font>
<br><font size=2 face="sans-serif">nfs.rpc-auth-allow: 10.1.99.201</font>
<br><font size=2 face="sans-serif">diagnostics.client-sys-log-level: WARNING</font>
<br><font size=2 face="sans-serif">diagnostics.brick-sys-log-level: WARNING</font>
<br>
<br>
<br><font size=2 face="sans-serif">The volume is exported only for the
Solaris client (via nfs.rpc-auth-allow)</font>
<br>
<br><font size=2 face="sans-serif">[root@llsmagfs001a glusterfs]# showmount
-e 10.1.99.202</font>
<br><font size=2 face="sans-serif">Export list for 10.1.99.202:</font>
<br><font size=2 face="sans-serif">/vol1 10.1.99.201</font>
<br>
<br><font size=2 face="sans-serif">If we try to mount this volume via NFS
from the Linux client, we receive an access denied as expected</font>
<br>
<br><font size=2 face="sans-serif">[root@llsmaofr001a mnt]# ifconfig eth0
| grep "inet addr"</font>
<br><font size=2 face="sans-serif"> inet
addr:10.1.99.200 Bcast:10.1.99.255 Mask:255.255.254.0</font>
<br><font size=2 face="sans-serif">[root@llsmaofr001a mnt]# mount -t nfs
-o vers=3 10.1.99.202:/vol1 /mnt/vol1</font>
<br><font size=2 face="sans-serif">mount.nfs: access denied by server while
mounting 10.1.99.202:/vol1</font>
<br>
<br><font size=2 face="sans-serif">But if we try to mount this volume from
another Solaris Client (10.1.98.66), we do not receive an access denied</font>
<br>
<br><font size=2 face="sans-serif"># ifconfig -a</font>
<br><font size=2 face="sans-serif">lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL>
mtu 8232 index 1</font>
<br><font size=2 face="sans-serif"> inet 127.0.0.1
netmask ff000000</font>
<br><font size=2 face="sans-serif">bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4>
mtu 1500 index 2</font>
<br><font size=2 face="sans-serif"> inet 10.1.98.66
netmask fffffe00 broadcast 10.1.99.255</font>
<br><font size=2 face="sans-serif"> ether 0:14:4f:5e:32:aa</font>
<br><font size=2 face="sans-serif"># mount -o vers=3 nfs://10.1.99.202/vol1
/mnt</font>
<br><font size=2 face="sans-serif"># mount | grep nfs</font>
<br><font size=2 face="sans-serif">/mnt on nfs://10.1.99.202/vol1 remote/read/write/setuid/devices/vers=3/xattr/dev=594001d
on Thu Oct 10 11:48:15 2013</font>
<br><font size=2 face="sans-serif"># echo "test from solaris"
> /mnt/test.solaris</font>
<br><font size=2 face="sans-serif"># ls /mnt</font>
<br><font size=2 face="sans-serif">test.solaris</font>
<br>
<br><font size=2 face="sans-serif">Tested with</font>
<br><font size=2 face="sans-serif">- Solaris 10 and Solaris 11</font>
<br><font size=2 face="sans-serif">- RHEL6</font>
<br><font size=2 face="sans-serif">- GlusterFS 3.3.1-1, GlusterFS 3.4.0-2
and GlusterFS 3.4.1-2</font>
<br>
<br><font size=2 face="sans-serif">Do we have to set another option to
enforce rpc auth for Solaris Client ?</font>
<br>
<br>
<br><font size=2 face="sans-serif">Debug message (when trying to mount
the volume from the linux client via NFS)</font>
<br>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.578302] D [socket.c:463:__socket_rwv]
0-socket.nfs-server: would have passed zero length to read/write</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.579045] D [socket.c:486:__socket_rwv]
0-socket.nfs-server: EOF on socket</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.579078] D [socket.c:2236:socket_event_handler]
0-transport: disconnecting now</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.587459] D [socket.c:463:__socket_rwv]
0-socket.nfs-server: would have passed zero length to read/write</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.588021] D [socket.c:486:__socket_rwv]
0-socket.nfs-server: EOF on socket</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.588076] D [socket.c:2236:socket_event_handler]
0-transport: disconnecting now</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.589570] D [socket.c:463:__socket_rwv]
0-socket.nfs-server: would have passed zero length to read/write</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.590260] D [mount3.c:912:mnt3svc_mnt]
0-nfs-mount: dirpath: /vol1</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.590293] D [mount3.c:855:mnt3_find_export]
0-nfs-mount: dirpath: /vol1</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.590309] D [mount3.c:749:mnt3_mntpath_to_export]
0-nfs-mount: Found export volume: vol1</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.590339] I [mount3.c:787:mnt3_check_client_net]
0-nfs-mount: Peer 10.1.99.200:860 not allowed</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.590353] D [mount3.c:934:mnt3svc_mnt]
0-nfs-mount: Client mount not allowed</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.591104] D [socket.c:486:__socket_rwv]
0-socket.nfs-server: EOF on socket</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:14:07.591171] D [socket.c:2236:socket_event_handler]
0-transport: disconnecting now</font>
<br>
<br>
<br><font size=2 face="sans-serif">Debug message (when trying to mount
the volume from the solaris client via NFS)</font>
<br>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.444951] D [nfs3-helpers.c:1641:nfs3_log_fh_entry_call]
0-nfs-nfsv3: XID: 5250f479, LOOKUP: args: FH: exportid 00000000-0000-0000-0000-000000000000,
gfid 00000000-0000-0000-0000-000000000000, name: vol1</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.446010] D [nfs3-helpers.c:3458:nfs3_log_newfh_res]
0-nfs-nfsv3: XID: 5250f479, LOOKUP: NFS: 0(Call completed successfully.),
POSIX: 117(Structure needs cleaning), FH: exportid 4abcee08-6172-441a-851b-53becb77c281,
gfid 00000000-0000-0000-0000-000000000001</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.446539] D [nfs3-helpers.c:1641:nfs3_log_fh_entry_call]
0-nfs-nfsv3: XID: 5250f478, LOOKUP: args: FH: exportid 00000000-0000-0000-0000-000000000000,
gfid 00000000-0000-0000-0000-000000000000, name: vol1</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.447234] D [nfs3-helpers.c:3458:nfs3_log_newfh_res]
0-nfs-nfsv3: XID: 5250f478, LOOKUP: NFS: 0(Call completed successfully.),
POSIX: 117(Structure needs cleaning), FH: exportid 4abcee08-6172-441a-851b-53becb77c281,
gfid 00000000-0000-0000-0000-000000000001</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.448077] D [socket.c:486:__socket_rwv]
0-socket.nfs-server: EOF on socket</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.448133] D [socket.c:2236:socket_event_handler]
0-transport: disconnecting now</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.469271] D [nfs3-helpers.c:1627:nfs3_log_common_call]
0-nfs-nfsv3: XID: 5ed48474, FSINFO: args: FH: exportid 4abcee08-6172-441a-851b-53becb77c281,
gfid 00000000-0000-0000-0000-000000000001</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.469601] D [nfs3-helpers.c:3389:nfs3_log_common_res]
0-nfs-nfsv3: XID: 5ed48474, FSINFO: NFS: 0(Call completed successfully.),
POSIX: 117(Structure needs cleaning)</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.470341] D [nfs3-helpers.c:1627:nfs3_log_common_call]
0-nfs-nfsv3: XID: 5ed48475, FSSTAT: args: FH: exportid 4abcee08-6172-441a-851b-53becb77c281,
gfid 00000000-0000-0000-0000-000000000001</font>
<br><font size=2 face="sans-serif">[2013-10-10 10:17:15.471159] D [nfs3-helpers.c:3389:nfs3_log_common_res]
0-nfs-nfsv3: XID: 5ed48475, FSSTAT: NFS: 0(Call completed successfully.),
POSIX: 117(Structure needs cleaning)</font>
<br>
<br><font size=2 face="sans-serif">Regards,</font>
<br>
<br><font size=2 face="sans-serif">Olivier</font>
<br>
<br>