<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10pt"><div id="yiv5028017569"><div><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10pt;"><div id="yiv5028017569yui_3_13_0_ym1_7_1389734926589_30" style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;background-color:transparent;font-style:normal;"><br clear="none"><span id="yiv5028017569yui_3_13_0_ym1_7_1389734926589_23"></span></div><div id="yiv5028017569yui_3_13_0_ym1_7_1389734926589_33" style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;background-color:transparent;font-style:normal;"><span id="yiv5028017569yui_3_13_0_ym1_7_1389734926589_23">Hi
Ben, <br><br>1. Port 24007 is open and all iptable rules have been flushed:<br><br>[root@gcvs4056 run]# telnet gcvs0139 24007<br>Trying 10.131.83.139...<br>Connected to gcvs0139.<br>Escape character is '^]'.<br><br><br>2. gluster peer status looks good from both boxes: <br><br>box1<br>[root@gcvs4056 run]# gluster peer status<br>Number of Peers: 1<br><br>Hostname: gcvs0139<br>Uuid: d40ba14d-cbb4-40e7-86a2-62afaa99af4d<br>State: Peer in Cluster (Connected)<br><br>box 2:<br><br><br># gluster peer status<br>Number of Peers: 1<br><br>Hostname: gcvs4056<br>Port: 24007<br>Uuid: b1aae40a-78be-4303-bf48-49fb41d6bb30<br>State: Peer in Cluster (Connected)<br><br>3. selinux is disabled on both boxes.<br><br> grep dis /etc/sysconfig/selinux <br># disabled - No SELinux policy is loaded.<br>SELINUX=disabled<br><br><br><br>Thanks for your help!<br><br clear="none"></span></div><div class="yiv5028017569yqt9247668738"
id="yiv5028017569yqt41448"><div class="yiv5028017569yahoo_quoted" id="yiv5028017569yui_3_13_0_ym1_7_1389734926589_17" style="display: block;"> <br clear="none"> <br clear="none"> <div class="yiv5028017569yui_3_13_0_ym1_1_1389734926589_5028" style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10pt;"> <div class="yiv5028017569yui_3_13_0_ym1_1_1389734926589_5029" style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:12pt;"> <div dir="ltr"> <font face="Arial" size="2"> On Tuesday, January 14, 2014 4:54 PM, Ben Turner <bturner@redhat.com> wrote:<br clear="none"> </font> </div> <div class="yiv5028017569y_msg_container">----- Original Message -----<br clear="none">> From: "Jeffrey Brewster" <<a rel="nofollow" shape="rect" ymailto="mailto:jab2805@yahoo.com" target="_blank" href="mailto:jab2805@yahoo.com">jab2805@yahoo.com</a>><br
clear="none">> To: "Ben Turner" <<a rel="nofollow" shape="rect" ymailto="mailto:bturner@redhat.com" target="_blank" href="mailto:bturner@redhat.com">bturner@redhat.com</a>><br clear="none">> Cc: <a rel="nofollow" shape="rect" ymailto="mailto:gluster-users@gluster.org" target="_blank" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br clear="none">> Sent: Tuesday, January 14, 2014 4:35:30 PM<br clear="none">>
Subject: Re: [Gluster-users] Unable to mount gfs gv0 volume Enterprise Linux Enterprise Linux Server release 5.6<br clear="none">> (Carthage)<br clear="none">> <br clear="none">> Hi Ben,<br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> I don't have any "E" (error I assume) lines in the mnt.log file. I check all<br clear="none">> the log files in the /var/log/glusterfs/ dir. I restarted glusterd to see<br clear="none">> if I could see any errors.<br clear="none">> <br clear="none"><br clear="none">Make sure SELinux is disabled and your firewall is open to allow gluster traffic. Have a look at:<br clear="none"><br clear="none"><a rel="nofollow" shape="rect" target="_blank"
href="http://www.gluster.org/community/documentation/index.php/Basic_Gluster_Troubleshooting">http://www.gluster.org/community/documentation/index.php/Basic_Gluster_Troubleshooting</a><br clear="none"><br clear="none">For what ports you need open. As a test I would just try disabling iptables and adding in the rules after you confirm it is working.<br clear="none"><br clear="none">-b<div class="yiv5028017569yqt2019339168" id="yiv5028017569yqtfd16132"><br clear="none"> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> Data:<br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> Warnings from mount log:<br clear="none">> -------------<br clear="none">> <br clear="none">> # grep W mnt.log | cat -n<br clear="none">> <br clear="none">> <br clear="none">> 1 [2014-01-14<br clear="none">>
19:32:22.920069] W [common-utils.c:2247:gf_get_reserved_ports]<br clear="none">> 0-glusterfs: could not open the file<br clear="none">> /proc/sys/net/ipv4/ip_local_reserved_ports for getting reserv<br clear="none">> <br clear="none">> 2 [2014-01-14 19:32:22.920108] W<br clear="none">> [common-utils.c:2280:gf_process_reserved_ports] 0-glusterfs: Not able to<br clear="none">> get reserved ports, hence there is a possibility that glusterfs may c<br clear="none">> <br clear="none">> 3 [2014-01-14 19:32:22.935611] W<br clear="none">> [common-utils.c:2247:gf_get_reserved_ports] 0-glusterfs: could not open<br clear="none">> the file /proc/sys/net/ipv4/ip_local_reserved_ports for getting reserv<br clear="none">> <br clear="none">> 4 [2014-01-14 19:32:22.935646] W<br clear="none">>
[common-utils.c:2280:gf_process_reserved_ports] 0-glusterfs: Not able to<br clear="none">> get reserved ports, hence there is a possibility that glusterfs may c<br clear="none">> <br clear="none">> 5 [2014-01-14 19:32:22.938783] W<br clear="none">> [common-utils.c:2247:gf_get_reserved_ports] 0-glusterfs: could not open<br clear="none">> the file /proc/sys/net/ipv4/ip_local_reserved_ports for getting reserv<br clear="none">> <br clear="none">> 6 [2014-01-14 19:32:22.938826] W<br clear="none">> [common-utils.c:2280:gf_process_reserved_ports] 0-glusterfs: Not able to<br clear="none">> get reserved ports, hence there is a possibility that glusterfs may c<br clear="none">> 7 [2014-01-14 19:32:22.941076] W [socket.c:514:__socket_rwv]<br clear="none">> 0-gv0-client-1: readv
failed (No data available)<br clear="none">> <br clear="none">> 8 [2014-01-14 19:32:22.945278] W<br clear="none">> [common-utils.c:2247:gf_get_reserved_ports] 0-glusterfs: could
not open<br clear="none">> the file /proc/sys/net/ipv4/ip_local_reserved_ports for getting reserv<br clear="none">> <br clear="none">> 9 [2014-01-14 19:32:22.945312] W<br clear="none">> [common-utils.c:2280:gf_process_reserved_ports] 0-glusterfs: Not able to<br clear="none">> get reserved ports, hence there is a possibility that glusterfs may c<br clear="none">> 10 [2014-01-14<br clear="none">> 19:32:22.946921] W [socket.c:514:__socket_rwv] 0-gv0-client-0: readv failed<br clear="none">> (No data available)<br clear="none">> <br clear="none">> 11 [2014-01-14 19:32:22.953383] W<br clear="none">> [common-utils.c:2247:gf_get_reserved_ports] 0-glusterfs: could not open<br clear="none">> the file /proc/sys/net/ipv4/ip_local_reserved_ports for getting reserv<br clear="none">> <br
clear="none">> 12 [2014-01-14 19:32:22.953423] W<br clear="none">> [common-utils.c:2280:gf_process_reserved_ports] 0-glusterfs: Not able to<br clear="none">> get reserved ports, hence there is a possibility that glusterfs may c<br clear="none">> <br clear="none">> 13 [2014-01-14 19:32:22.976633] W [glusterfsd.c:1002:cleanup_and_exit]<br clear="none">> (-->/lib64/libc.so.6(clone+0x6d) [0x31f6ad40cd]<br clear="none">> (-->/lib64/libpthread.so.0 [0x<br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> After restarting glusterd:<br clear="none">> -----------------------------<br clear="none">> <br clear="none">> <br clear="none">> # grep E * | grep 21:25| cat -n<br clear="none">> <br
clear="none">> <br clear="none">> 1 etc-glusterfs-glusterd.vol.log:[2014-01-14 21:25:47.637082] E<br clear="none">> [rpc-transport.c:253:rpc_transport_load] 0-rpc-transport:<br clear="none">> /usr/lib64/glusterfs/3.4.2/rpc-transport/rdma.so: cannot open shared<br clear="none">> object file: No such file or directory<br clear="none">> 2 etc-glusterfs-glusterd.vol.log:[2014-01-14 21:25:49.940650] E<br clear="none">> [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key:<br clear="none">> brick-0<br clear="none">> 3 etc-glusterfs-glusterd.vol.log:[2014-01-14 21:25:49.940698] E<br clear="none">> [glusterd-store.c:1858:glusterd_store_retrieve_volume] 0-: Unknown key:<br
clear="none">> brick-1<br clear="none">> 4 etc-glusterfs-glusterd.vol.log:[2014-01-14 21:25:52.075563] E<br clear="none">> [glusterd-utils.c:3801:glusterd_nodesvc_unlink_socket_file]<br clear="none">> 0-management: Failed to remove<br clear="none">> /var/run/3096dde11d292c28c8c2f97101c272e8.socket error: Resource<br clear="none">> temporarily unavailable<br clear="none">> 5 etc-glusterfs-glusterd.vol.log:[2014-01-14 21:25:53.084722] E<br clear="none">> [glusterd-utils.c:3801:glusterd_nodesvc_unlink_socket_file]<br clear="none">> 0-management: Failed to remove<br clear="none">> /var/run/15f2dcd004edbff6ab31364853d6b6b0.socket error: No such file or<br
clear="none">> directory<br clear="none">> 6 glustershd.log:[2014-01-14 21:25:42.392401] W<br clear="none">> [socket.c:1962:__socket_proto_state_machine] 0-glusterfs: reading from<br clear="none">> socket failed. Error (No data available), peer (127.0.0.1:24007)<br clear="none">> 7 glustershd.log:[2014-01-14 21:25:53.476026] E<br clear="none">> [afr-self-heald.c:1067:afr_find_child_position] 0-gv0-replicate-0:<br clear="none">> getxattr failed on gv0-client-0 - (Transport endpoint is not connected)<br clear="none">> 8 nfs.log:[2014-01-14 21:25:42.391560] W<br clear="none">> [socket.c:1962:__socket_proto_state_machine] 0-glusterfs: reading from<br clear="none">>
socket failed. Error (No data available), peer (127.0.0.1:24007)<br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> Procs After restrt:<br clear="none">> <br clear="none">> <br clear="none">> ps -ef | grep gluster<br clear="none">> root 6345 1 0 18:35 ? 00:00:00 /usr/sbin/glusterfsd -s<br clear="none">> gcvs4056 --volfile-id gv0.gcvs4056.data-gv0-brick1-app -p<br clear="none">> /var/lib/glusterd/vols/gv0/run/gcvs4056-data-gv0-brick1-app.pid -S<br clear="none">> /var/run/f2339d9fa145fd28662d8b970fbd4aab.socket --brick-name<br clear="none">> /data/gv0/brick1/app -l /var/log/glusterfs/bricks/data-gv0-brick1-app.log<br clear="none">> --xlator-option *-posix.glusterd-uuid=b1aae40a-78be-4303-bf48-49fb41d6bb30<br clear="none">> --brick-port 49153 --xlator-option
gv0-server.listen-port=49153<br clear="none">> root 7240 1 0 21:25 ? 00:00:00 /usr/sbin/glusterd<br clear="none">> --pid-file=/var/run/glusterd.pid<br clear="none">> root 7266 1 0 21:25 ? 00:00:00 /usr/sbin/glusterfs -s<br clear="none">> localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l<br clear="none">> /var/log/glusterfs/nfs.log -S<br clear="none">> /var/run/3096dde11d292c28c8c2f97101c272e8.socket<br clear="none">> root 7273 1 0 21:25 ? 00:00:00 /usr/sbin/glusterfs -s<br clear="none">> localhost --volfile-id gluster/glustershd -p<br clear="none">> /var/lib/glusterd/glustershd/run/glustershd.pid -l<br
clear="none">> /var/log/glusterfs/glustershd.log -S<br clear="none">> /var/run/15f2dcd004edbff6ab31364853d6b6b0.socket --xlator-option<br clear="none">> *replicate*.node-uuid=b1aae40a-78be-4303-bf48-49fb41d6bb30<br clear="none">> root 7331 5375 0 21:34 pts/1 00:00:00 grep gluster<br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> On Tuesday, January 14, 2014 4:08 PM, Ben Turner <<a rel="nofollow" shape="rect" ymailto="mailto:bturner@redhat.com" target="_blank" href="mailto:bturner@redhat.com">bturner@redhat.com</a>> wrote:<br clear="none">> <br clear="none">> ----- Original Message -----<br clear="none">> > From: "Jeffrey Brewster" <<a rel="nofollow" shape="rect" ymailto="mailto:jab2805@yahoo.com" target="_blank"
href="mailto:jab2805@yahoo.com">jab2805@yahoo.com</a>><br clear="none">> >
To: "Ben Turner" <<a rel="nofollow" shape="rect" ymailto="mailto:bturner@redhat.com" target="_blank" href="mailto:bturner@redhat.com">bturner@redhat.com</a>><br clear="none">> > Cc: <a rel="nofollow" shape="rect" ymailto="mailto:gluster-users@gluster.org" target="_blank" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br clear="none">> > Sent: Tuesday, January 14, 2014 3:57:24 PM<br clear="none">> > Subject: Re: [Gluster-users] Unable to mount gfs gv0 volume Enterprise<br clear="none">> > Linux Enterprise Linux Server release 5.6<br clear="none">> > (Carthage)<br clear="none">> > <br clear="none">> > Thanks Ben,<br clear="none">> > <br clear="none">> > <br clear="none">> > <br clear="none">> > I tried that it still failed.<br clear="none">> <br clear="none">> As Vijay suggested have a look at /var/log/glusterfs, there should be a
log<br clear="none">> there with the mountpoint name that
should give us a clue as to what is<br clear="none">> going on. To note if there is a problem with FUSE not being loaded you will<br clear="none">> see something like:<br clear="none">> <br clear="none">> [2013-01-12 01:58:22.213417] I [glusterfsd.c:1759:main]<br clear="none">> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version<br clear="none">> 3.3.0.5rhs<br clear="none">> [2013-01-12 01:58:22.213831] E [mount.c:596:gf_fuse_mount] 0-glusterfs-fuse:<br clear="none">> cannot open /dev/fuse (No such file or directory)<br clear="none">> [2013-01-12 01:58:22.213856] E [xlator.c:385:xlator_init] 0-fuse:<br clear="none">> Initialization of volume 'fuse' failed, review your volfile again<br clear="none">> <br clear="none">> If you can't tell the problem from the log shoot out the relevant line and<br clear="none">> I'll have a look.<br clear="none">> <br clear="none">> -b<br clear="none">>
<br clear="none">> <br clear="none">> > <br clear="none">> > <br clear="none">> > <br clear="none">> > On Tuesday, January 14, 2014 3:22 PM, Ben Turner <<a rel="nofollow" shape="rect" ymailto="mailto:bturner@redhat.com" target="_blank" href="mailto:bturner@redhat.com">bturner@redhat.com</a>><br clear="none">> > wrote:<br clear="none">> > <br clear="none">> > ----- Original Message -----<br clear="none">> > > From: "Jeffrey Brewster" <<a rel="nofollow" shape="rect" ymailto="mailto:jab2805@yahoo.com" target="_blank" href="mailto:jab2805@yahoo.com">jab2805@yahoo.com</a>><br clear="none">> > > To: <a rel="nofollow" shape="rect" ymailto="mailto:gluster-users@gluster.org" target="_blank" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br clear="none">> > > Sent: Tuesday, January 14, 2014 1:47:55 PM<br clear="none">> > > Subject:
[Gluster-users] Unable to mount gfs gv0 volume Enterprise Linux<br clear="none">> > >
Enterprise Linux Server release 5.6<br clear="none">> > > (Carthage)<br clear="none">> > > <br clear="none">> > > <br clear="none">> > > <br clear="none">> > > Hi all,<br clear="none">> > > <br clear="none">> > > I have been following the quick start guide as part of a POC. I created a<br clear="none">> > > 10GB brick to be mounted. I'm unable to mount the volume. I don't see any<br clear="none">> > > thing in the logs. has anyone had the same issues? I was thinking I need<br clear="none">> > > to<br clear="none">> > > install gluster-client but I don't see in the latest release rpms.<br clear="none">> > > <br clear="none">> > > Data:<br clear="none">> > > ===========<br clear="none">> > > <br clear="none">> > > OS Version:<br clear="none">> > > ------------<br clear="none">> >
> <br clear="none">> > > Description: Enterprise Linux Enterprise Linux Server release 5.6<br clear="none">> > > (Carthage<br clear="none">> > > <br clear="none">> > > <br clear="none">> > > Installed packages on both servers<br clear="none">> > > ------------<br clear="none">> > > <br clear="none">> > > # rpm -qa | grep gluster | cat -n<br clear="none">> > > 1 glusterfs-libs-3.4.2-1.el5<br clear="none">> > > 2 glusterfs-3.4.2-1.el5<br clear="none">> > > 3 glusterfs-cli-3.4.2-1.el5<br clear="none">> > > 4 glusterfs-geo-replication-3.4.2-1.el5<br clear="none">> > > 5 glusterfs-fuse-3.4.2-1.el5<br clear="none">> > > 6 glusterfs-server-3.4.2-1.el5<br clear="none">> > > <br clear="none">> > > <br clear="none">> > > gluster peer probe successful:<br clear="none">> > > -----------<br
clear="none">> > > peer probe: success: host gcvs0139 port 24007 already in peer list<br clear="none">> > > <br clear="none">> > > Gluster info:<br clear="none">> > > ---------<br clear="none">> > > gluster volume info | cat -n<br clear="none">> > > 1<br clear="none">> > > 2 Volume Name: gv0<br clear="none">> > > 3 Type: Replicate<br clear="none">> > > 4 Volume ID: 30a27041-ba1b-456f-b0bc-d8cdd2376c2f<br clear="none">> > > 5 Status: Started<br clear="none">> > > 6 Number of Bricks: 1 x 2 = 2<br clear="none">> > > 7 Transport-type: tcp<br clear="none">> > > 8 Bricks:<br clear="none">> > > 9 Brick1: gcvs0139:/data/gv0/brick1/app<br clear="none">> > > 10 Brick2: gcvs4056:/data/gv0/brick1/app<br clear="none">> > > <br clear="none">> > > <br clear="none">> > > Mount Failure:<br clear="none">>
> > ----------<br clear="none">> > > <br clear="none">> > > <br clear="none">> > > [<a rel="nofollow" shape="rect" ymailto="mailto:root@gcvs4056" target="_blank" href="mailto:root@gcvs4056">root@gcvs4056</a> jbrewster]# mount -t glusterfs gcvs4056:/gv0 /mnt<br clear="none">> > > Mount failed. Please check the log file for more details.<br clear="none">> > > <br clear="none">> > <br clear="none">> > I bet you need to modprobe the fuse module, in el5 its not loaded by<br clear="none">> > default.<br clear="none">> > <br clear="none">> > <br clear="none">> > -b<br clear="none">> > <br clear="none">> > > <br clear="none">> > > <br clear="none">> > > _______________________________________________<br clear="none">> > > Gluster-users mailing list<br clear="none">> > > <a rel="nofollow" shape="rect"
ymailto="mailto:Gluster-users@gluster.org" target="_blank" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br clear="none">> > > <a rel="nofollow" shape="rect" target="_blank" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></div><br clear="none"><br clear="none"></div> </div> </div> </div></div> </div></div></div></div></body></html>