<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<font size="-1">Hi all!<br>
I'm rather new to glusterfs, trying it out for redundant storage
for my very small company. <br>
I have a minimal setup of glusterfs, 2 servers (storage1 &
storage2) with one brick each, both added to volume "testvol1". I
then mount "testvol1" on a third server (app1). This is fine as
long as I use any one servers ip and filesystem glusterfs, but
when trying to mount the volume using a file
"/owndata/conf/glusterfs/storage.vol", it does not work. Also I
can't use nfs - don't need it but maybe it's related. <br>
<br>
So: <br>
"mount -t glusterfs 192.168.160.21:/testvol1 /mnt/tmp/" works. I
write a file to /mnt/tmp/filename, then mount
192.168.12.210:/testvol1 and the newly created file is there.<br>
<br>
Trying to mount using config file storage.vol:<br>
</font>
<blockquote><font size="-1">mount -t glusterfs
/owndata/conf/glusterfs/storage.vol /mnt/tmp</font><br>
<font size="-1">Mount failed. Please check the log file for more
details.</font><br>
</blockquote>
<font size="-1">The main error in the log is<br>
</font>
<blockquote><font size="-1"> <i>E
[client-handshake.c:1778:client_query_portmap_cbk] 0-remote1:
failed to get the port number for remote subvolume. Please run
'gluster volume status' on server to see if brick process is
running</i>.</font><br>
</blockquote>
<font size="-1">There are lots & lots of pages on the 'net about
this error message, none of the solutions I've found has worked.<br>
</font><br>
<font size="-1"><font size="-1"><font size="-1">CentOS 6.5 on all
servers, they are all kvm hosts under oVirt (this is just the
testing stage, will be on real iron in production).<br>
</font>No firewall anywhere, selinux is permissive.<br>
</font><br>
<b>File storage.vol:</b><br>
</font>
<blockquote><tt><font size="-1">volume remote1</font></tt><tt><br>
</tt><tt><font size="-1"> type protocol/client</font></tt><tt><br>
</tt><tt><font size="-1"> option transport-type tcp</font></tt><tt><br>
</tt><tt><font size="-1"> option remote-host 192.168.12.210</font></tt><tt><br>
</tt><tt><font size="-1"> option remote-subvolume testvol1</font></tt><tt><br>
</tt><tt><font size="-1">end-volume</font></tt><tt><br>
</tt><tt><font size="-1"> </font></tt><tt><br>
</tt><tt><font size="-1">volume remote2</font></tt><tt><br>
</tt><tt><font size="-1"> type protocol/client</font></tt><tt><br>
</tt><tt><font size="-1"> option transport-type tcp</font></tt><tt><br>
</tt><tt><font size="-1"> option remote-host 192.168.160.21</font></tt><tt><br>
</tt><tt><font size="-1"> option remote-subvolume testvol1</font></tt><tt><br>
</tt><tt><font size="-1">end-volume</font></tt><tt><br>
</tt><tt><font size="-1"> </font></tt><tt><br>
</tt><tt><font size="-1">volume replicate</font></tt><tt><br>
</tt><tt><font size="-1"> type cluster/replicate</font></tt><tt><br>
</tt><tt><font size="-1"> subvolumes remote1 remote2</font></tt><tt><br>
</tt><tt><font size="-1">end-volume</font></tt><tt><br>
</tt><tt><font size="-1"> </font></tt><tt><br>
</tt><tt><font size="-1">volume writebehind</font></tt><tt><br>
</tt><tt><font size="-1"> type performance/write-behind</font></tt><tt><br>
</tt><tt><font size="-1"> option window-size 1MB</font></tt><tt><br>
</tt><tt><font size="-1"> subvolumes replicate</font></tt><tt><br>
</tt><tt><font size="-1">end-volume</font></tt><tt><br>
</tt><tt><font size="-1"> </font></tt><tt><br>
</tt><tt><font size="-1">volume cache</font></tt><tt><br>
</tt><tt><font size="-1"> type performance/io-cache</font></tt><tt><br>
</tt><tt><font size="-1"> option cache-size 256MB</font></tt><tt><br>
</tt><tt><font size="-1"> subvolumes writebehind</font></tt><tt><br>
</tt><font size="-1"><tt>end-volume</tt></font><br>
</blockquote>
<font size="-1"><b># gluster volume info</b><br>
</font>
<blockquote><tt><font size="-1">Volume Name: testvol1</font></tt><tt><br>
</tt><tt><font size="-1">Type: Replicate</font></tt><tt><br>
</tt><tt><font size="-1">Volume ID:
bcca4aa2-46c0-44a2-8175-1305faa8b4f9</font></tt><tt><br>
</tt><tt><font size="-1">Status: Started</font></tt><tt><br>
</tt><tt><font size="-1">Number of Bricks: 1 x 2 = 2</font></tt><tt><br>
</tt><tt><font size="-1">Transport-type: tcp</font></tt><tt><br>
</tt><tt><font size="-1">Bricks:</font></tt><tt><br>
</tt><tt><font size="-1">Brick1: 192.168.12.210:/export/vdb/brick1</font></tt><tt><br>
</tt><tt><font size="-1">Brick2: 192.168.160.21:/export/vdb/brick1</font></tt><br>
</blockquote>
<font size="-1"><b># gluster volume status</b><br>
</font>
<blockquote><tt><font size="-1">Status of volume: testvol1</font></tt><tt><br>
</tt><tt><font size="-1">Gluster
process Port
Online Pid</font></tt><tt><br>
</tt><tt><font size="-1">-------------------------------------------------------------------------</font></tt><tt><br>
</tt><tt><font size="-1">Brick
192.168.12.210:/export/vdb/brick1 49152
Y 1656</font></tt><tt><br>
</tt><tt><font size="-1">Brick
192.168.160.21:/export/vdb/brick1 49152
Y 139090</font></tt><tt><br>
</tt><tt><font size="-1">NFS Server on
localhost 2049 Y 1670</font></tt><tt><br>
</tt><tt><font size="-1">Self-heal Daemon on
localhost N/A Y 1674</font></tt><tt><br>
</tt><tt><font size="-1">NFS Server on
192.168.160.21 2049 Y 1481</font></tt><tt><br>
</tt><tt><font size="-1">Self-heal Daemon on
192.168.160.21 N/A Y 139105</font></tt><tt><br>
</tt><tt><font size="-1"> </font></tt><tt><br>
</tt><tt><font size="-1">Task Status of Volume testvol1</font></tt><tt><br>
</tt><tt><font size="-1">------------------------------------------------------------------------------</font></tt><tt><br>
</tt><tt><font size="-1">There are no active volume tasks</font></tt><br>
</blockquote>
<b>Complete log after fail:<br>
</b>
<blockquote><tt>[2014-10-02 14:38:22.252235] I
[glusterfsd.c:2026:main] 0-/usr/sbin/glusterfs: Started running
/usr/sbin/glusterfs version 3.4.0.57rhs (/usr/sbin/glusterfs
--fuse-mountopts=allow_other,default_permissions,max_read=131072
--volfile=/owndata/conf/glusterfs/storage.vol
--fuse-mountopts=allow_other,default_permissions,max_read=131072
/mnt/glust)</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.284438] W
[options.c:848:xl_opt_validate] 0-writebehind: option
'window-size' is deprecated, preferred is 'cache-size',
continuing with correction</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.284476] W [io-cache.c:1672:init]
0-cache: dangling volume. check volfile </tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.294306] I
[socket.c:3505:socket_init] 0-remote2: SSL support is NOT
enabled</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.294339] I
[socket.c:3520:socket_init] 0-remote2: using system polling
thread</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.294832] I
[socket.c:3505:socket_init] 0-remote1: SSL support is NOT
enabled</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.294848] I
[socket.c:3520:socket_init] 0-remote1: using system polling
thread</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.294870] I [client.c:2171:notify]
0-remote1: parent translators are ready, attempting connect on
transport</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.306697] I [client.c:2171:notify]
0-remote2: parent translators are ready, attempting connect on
transport</tt><tt><br>
</tt><tt>Final graph:</tt><tt><br>
</tt><tt>+------------------------------------------------------------------------------+</tt><tt><br>
</tt><tt> 1: volume remote1</tt><tt><br>
</tt><tt> 2: type protocol/client</tt><tt><br>
</tt><tt> 3: option remote-subvolume testvol1</tt><tt><br>
</tt><tt> 4: option remote-host 192.168.12.210</tt><tt><br>
</tt><tt> 5: option transport-type socket</tt><tt><br>
</tt><tt> 6: end-volume</tt><tt><br>
</tt><tt> 7: </tt><tt><br>
</tt><tt> 8: volume remote2</tt><tt><br>
</tt><tt> 9: type protocol/client</tt><tt><br>
</tt><tt> 10: option remote-subvolume testvol1</tt><tt><br>
</tt><tt> 11: option remote-host 192.168.160.21</tt><tt><br>
</tt><tt> 12: option transport-type socket</tt><tt><br>
</tt><tt> 13: end-volume</tt><tt><br>
</tt><tt> 14: </tt><tt><br>
</tt><tt> 15: volume replicate</tt><tt><br>
</tt><tt> 16: type cluster/replicate</tt><tt><br>
</tt><tt> 17: subvolumes remote1 remote2</tt><tt><br>
</tt><tt> 18: end-volume</tt><tt><br>
</tt><tt> 19: </tt><tt><br>
</tt><tt> 20: volume writebehind</tt><tt><br>
</tt><tt> 21: type performance/write-behind</tt><tt><br>
</tt><tt> 22: option cache-size 1MB</tt><tt><br>
</tt><tt> 23: subvolumes replicate</tt><tt><br>
</tt><tt> 24: end-volume</tt><tt><br>
</tt><tt> 25: </tt><tt><br>
</tt><tt> 26: volume cache</tt><tt><br>
</tt><tt> 27: type performance/io-cache</tt><tt><br>
</tt><tt> 28: option cache-size 256MB</tt><tt><br>
</tt><tt> 29: subvolumes writebehind</tt><tt><br>
</tt><tt> 30: end-volume</tt><tt><br>
</tt><tt> 31: </tt><tt><br>
</tt><tt>+------------------------------------------------------------------------------+</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.310830] E
[client-handshake.c:1778:client_query_portmap_cbk] 0-remote1:
failed to get the port number for remote subvolume. Please run
'gluster volume status' on server to see if brick process is
running.</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.310887] I
[client.c:2103:client_rpc_notify] 0-remote1: disconnected from
192.168.12.210:24007. Client process will keep trying to connect
to glusterd until brick's port is available. </tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.311031] E
[client-handshake.c:1778:client_query_portmap_cbk] 0-remote2:
failed to get the port number for remote subvolume. Please run
'gluster volume status' on server to see if brick process is
running.</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.311059] I
[client.c:2103:client_rpc_notify] 0-remote2: disconnected from
192.168.160.21:24007. Client process will keep trying to connect
to glusterd until brick's port is available. </tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.311070] E
[afr-common.c:4025:afr_notify] 0-replicate: All subvolumes are
down. Going offline until atleast one of them comes back up.</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.314827] I
[fuse-bridge.c:5874:fuse_graph_setup] 0-fuse: switched to graph
0</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.316140] I
[fuse-bridge.c:4811:fuse_init] 0-glusterfs-fuse: FUSE inited
with protocol versions: glusterfs 7.13 kernel 7.13</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.321404] W
[fuse-bridge.c:1134:fuse_attr_cbk] 0-glusterfs-fuse: 2: LOOKUP()
/ => -1 (Transport endpoint is not connected)</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.324731] I
[fuse-bridge.c:5715:fuse_thread_proc] 0-fuse: unmounting
/mnt/glust</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.324931] W
[glusterfsd.c:1099:cleanup_and_exit]
(-->/lib64/libc.so.6(clone+0x6d) [0x7f6e2ec5e86d]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f6e2f2f19d1]
(-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd)
[0x4052ad]))) 0-: received signum (15), shutting down</tt><tt><br>
</tt><tt>[2014-10-02 14:38:22.324946] I [fuse-bridge.c:6412:fini]
0-fuse: Unmounting '/mnt/glust'.</tt><br>
</blockquote>
<font size="-1"><br>
Regards<br>
Peter H<br>
</font><br>
</body>
</html>