<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Hello,
<div><br>
</div>
<div>I don't have a great deal of experience yet with Gluster, and I'm having some tremendous difficulties adding an additional (3rd) server to the pool I have set up. This is running Gluster 3.3.0 on RHEL 6.3, using RPMs:</div>
<div>
<div># rpm -qa | grep gluster</div>
<div>glusterfs-rdma-3.3.0-1.el6.x86_64</div>
<div>glusterfs-server-3.3.0-1.el6.x86_64</div>
<div>glusterfs-3.3.0-1.el6.x86_64</div>
<div>glusterfs-fuse-3.3.0-1.el6.x86_64</div>
</div>
<div><br>
</div>
<div>When I first set it up, I was able to add the second server without an issue using
<b>gluster peer probe vothgluster2</b>, however when I do the same here, it just hangs for a couple minutes, then returns to the prompt without successfully probing the server.</div>
<div><br>
</div>
<div>From the first server:</div>
<div>
<div># gluster volume info</div>
<div> </div>
<div>Volume Name: home</div>
<div>Type: Distribute</div>
<div>Volume ID: a6f62691-a876-431a-a941-d3b76e197eeb</div>
<div>Status: Started</div>
<div>Number of Bricks: 4</div>
<div>Transport-type: tcp,rdma</div>
<div>Bricks:</div>
<div>Brick1: vothgluster1:/export/brick1</div>
<div>Brick2: vothgluster1:/export/brick2</div>
<div>Brick3: vothgluster2:/export/brick1</div>
<div>Brick4: vothgluster2:/export/brick2</div>
<div>Options Reconfigured:</div>
<div>nfs.rpc-auth-allow: 128.135.10.*,205.208.111.*,128.135.233.*,128.135.186.*,10.135.10.*,205.208.76.203</div>
<div>nfs.disable: on</div>
<div>nfs.register-with-portmap: on</div>
<div>performance.io-cache: on</div>
<div>nfs.port: 2049</div>
<div>features.quota: off</div>
<div>performance.cache-size: 64MB</div>
<div>performance.read-ahead: off</div>
<div>performance.write-behind-window-size: 1MB</div>
<div>nfs.ports-insecure: on</div>
</div>
<div><br>
</div>
<div># gluster peer status</div>
<div>Number of Peers: 2</div>
<div><br>
</div>
<div>Hostname: vothgluster2</div>
<div>Uuid: 44033fcb-f6da-4491-b860-d05bb3ce23f7</div>
<div>State: Peer in Cluster (Connected)</div>
<div><br>
</div>
<div>Hostname: vothgluster3</div>
<div>Uuid: 924322ef-5e55-427b-a393-90903ccb7d16</div>
<div>State: Establishing Connection (Connected)</div>
<div><br>
</div>
<div>From the second server, gluster volume info reports the same.</div>
<div>
<div># gluster peer status</div>
<div>Number of Peers: 1</div>
<div><br>
</div>
<div>Hostname: 205.208.111.14</div>
<div>Uuid: 70ee8eed-224e-4031-8cfe-52004c5ccf47</div>
<div>State: Peer in Cluster (Connected)</div>
</div>
<div><br>
</div>
<div>From the new (problematic) server:</div>
<div>
<div># gluster volume info</div>
<div>No volumes present</div>
</div>
<div><br>
</div>
<div>
<div># gluster peer status</div>
<div>Number of Peers: 1</div>
<div><br>
</div>
<div>Hostname: 205.208.111.14</div>
<div>Uuid: 00000000-0000-0000-0000-000000000000</div>
<div>State: Connected to Peer (Connected)</div>
</div>
<div><br>
</div>
<div>Forward and Reverse DNS lookups between all 3 servers work fine. Ping works fine. IB connectivity works fine (using tcp,rdma transport type).</div>
<div><br>
</div>
<div>/var/log/glusterfs/etc-glusterfs-glusterd.vol.log on vothgluster3</div>
<div>
<div>[2012-09-14 15:16:27.528532] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.3.0</div>
<div>[2012-09-14 15:16:27.530778] I [glusterd.c:807:init] 0-management: Using /var/lib/glusterd as working directory</div>
<div>[2012-09-14 15:16:28.088422] I [glusterd.c:95:glusterd_uuid_init] 0-glusterd: retrieved UUID: 924322ef-5e55-427b-a393-90903ccb7d16</div>
<div>[2012-09-14 15:16:28.097455] I [glusterd.c:265:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system</div>
<div>Given volfile:</div>
<div>+------------------------------------------------------------------------------+</div>
<div> 1: volume management</div>
<div> 2: type mgmt/glusterd</div>
<div> 3: option working-directory /var/lib/glusterd</div>
<div> 4: option transport-type socket,rdma</div>
<div> 5: option transport.socket.keepalive-time 10</div>
<div> 6: option transport.socket.keepalive-interval 2</div>
<div> 7: option transport.socket.read-fail-log off</div>
<div> 8: end-volume</div>
<div><br>
</div>
<div>+------------------------------------------------------------------------------+</div>
<div>[2012-09-14 15:16:32.760625] I [glusterd-handler.c:1753:glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 70ee8eed-224e-4031-8cfe-52004c5ccf47</div>
<div>[2012-09-14 15:16:32.760707] I [glusterd-handler.c:411:glusterd_friend_find] 0-glusterd: Unable to find peer by uuid</div>
<div>[2012-09-14 15:16:32.760929] I [glusterd-handler.c:423:glusterd_friend_find] 0-glusterd: Unable to find hostname: 205.208.111.14</div>
<div>[2012-09-14 15:16:32.760950] I [glusterd-handler.c:1767:glusterd_handle_probe_query] 0-glusterd: Unable to find peerinfo for host: 205.208.111.14 (24007)</div>
<div>[2012-09-14 15:16:32.766007] I [glusterd-handler.c:2204:glusterd_friend_add] 0-management: connect returned 0</div>
<div>[2012-09-14 15:16:32.766067] I [glusterd-handler.c:1793:glusterd_handle_probe_query] 0-glusterd: Responded to 205.208.111.14, op_ret: 0, op_errno: 0, ret: 0</div>
<div>[2012-09-14 15:16:32.766572] I [glusterd-handshake.c:397:glusterd_set_clnt_mgmt_program] 0-: Using Program glusterd mgmt, Num (1238433), Version (2)</div>
<div>[2012-09-14 15:16:32.766620] I [glusterd-handshake.c:403:glusterd_set_clnt_mgmt_program] 0-: Using Program Peer mgmt, Num (1238437), Version (2)</div>
<div>[2012-09-14 15:18:44.916221] I [glusterd-handler.c:813:glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req</div>
<div>[2012-09-14 15:19:41.020394] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req</div>
<div>[2012-09-14 15:22:40.807457] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req</div>
<div>[2012-09-14 15:22:48.655202] I [glusterd-handler.c:813:glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req</div>
</div>
<div><br>
</div>
<div>-----</div>
<div><br>
</div>
<div>Please let me know any other information to provide.</div>
<div><br>
</div>
<div>Thanks in advance,</div>
<div>Tom</div>
<div><br>
</div>
<br>
<br>
<div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
<div style="font-size: medium; "><span style="font-size: 12px; ">--</span></div>
<div apple-content-edited="true" style="font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; font-size: 12px; border-spacing: 0px; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px; ">
<div>Tom Indelli</div>
<div><a href="mailto:tindelli@uchicago.edu">tindelli@uchicago.edu</a></div>
<div>Systems Administrator</div>
<div>Physical Sciences Division</div>
<div>The University of Chicago</div>
</span></div>
</span></div>
</span></div>
</span></div>
</span></div>
</span></div>
</span></div>
</span></div>
</div>
</body>
</html>