<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 12/14/2014 02:57 AM, wodel youchi
wrote:<br>
</div>
<blockquote
cite="mid:1719753123.302321.1418506032237.JavaMail.yahoo@jws11164.mail.ir2.yahoo.com"
type="cite">
<div style="color:#000; background-color:#fff;
font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial,
Lucida Grande, sans-serif;font-size:16px">Hi again,<br style=""
class="">
<br style="" class="">
I had a hard time to configure geo-replication. After creating
the session, the start gave me faulty state and I had to deal
with it, it was the /nonexistent/gsyncd in
/var/lib/glusterd/geo-replication/data1_node3.example_data2/gsyncd.conf
to change to /usr/libexec/gluster/gsyncd<br style="" class="">
</div>
</blockquote>
When you get /nonexistent/gsyncd, that means issue with pem keys
setup. For security reason while pushing the pem keys to slave, we
add command=<COMMAND TO RUN> before key in
/root/.ssh/authorized_keys.<br>
<br>
For example,<br>
<pre class="bz_comment_text
bz_wrap_comment_text" id="comment_text_7">command="/usr/local/libexec/glusterfs/gsyncd" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAqxqMiZ8dyXUQq0pLVOYpRSsC+aYFn6pbPQZ3LtRPKGYfA63SNoYni
fhnM2UR9fnZz3hisBUxIzcVrVux2y3ojI/vPFFi08tVtK8/r9yoqqh3HqlBnotY50H1/1qeco+71U9hy276fUONP64KoOZtme3MwYuoNz
4z1NvCQFcEbXtPfHO5A9P3C+NuMhgNK8N63RSCzZ6dtO+wZygbVJlbPNQxp8Y5E8rbIuzRy6bD/0nmEKc/nqvEYTYgkck
ES0Xy92JVxbcwCOnZFNi4rT6+HarDIuFRB835I5ss+QBrT9SM09qmFuQ== root@fedoravm1
If you have any other entry in /root/.ssh/authorized_keys file of slave machine with same key then you will get /nonexistent/gsyncd error.
Advantage of having "command" in authorized key is, if any master node is compromised,
then they can login to any slave node. But with this command option, logged in user is
only limited to use gsyncd or the command specified in authorized_keys.
</pre>
<br>
<blockquote
cite="mid:1719753123.302321.1418506032237.JavaMail.yahoo@jws11164.mail.ir2.yahoo.com"
type="cite">
<div style="color:#000; background-color:#fff;
font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial,
Lucida Grande, sans-serif;font-size:16px"><br style="" class="">
after that the geo-started, but nothing happened, no file or
directory was synced.<br style="" class="">
<br style="" class="">
the gluster volume geo-replication data1
<a class="moz-txt-link-abbreviated" href="mailto:geoaccount@node3.example.com::data2">geoaccount@node3.example.com::data2</a> status<br style="" class="">
<br style="" class="">
keeps saying: Changelog Crawl<br style="" class="">
<br style="" class="">
[2014-12-13 21:57:17.129836] W
[master(/mnt/srv1/brick1):1005:process] _GMaster: incomplete
sync, retrying changelogs: CHANGELOG.1418504186<br style=""
class="">
[2014-12-13 21:57:22.648163] W
[master(/mnt/srv1/brick1):294:regjob] _GMaster: Rsync:
.gfid/f066ca4a-2d31-4342-bc7e-a37da25b2253 [errcode: 23]<br
style="" class="">
[2014-12-13 21:57:22.648426] W
[master(/mnt/srv1/brick1):986:process] _GMaster: changelogs
CHANGELOG.1418504186 could not be processed - moving on...<br
style="" class="">
<br style="" class="">
but new files/directories are synced, so I deleted all data on
the master cluster and recreated them, and all was synced<br
style="" class="">
but the status command (above) keeps saying Changelog Crawl<br
style="" class="">
<br style="" class="">
On the slave node I have these logs<br style="" class="">
<br style="" class="">
[2014-12-13 21:11:12.149041] W
[client-rpc-fops.c:1210:client3_3_removexattr_cbk]
0-data2-client-0: remote operation failed: No data available<br
style="" class="">
[2014-12-13 21:11:12.149067] W [fuse-bridge.c:1261:fuse_err_cbk]
0-glusterfs-fuse: 3243: REMOVEXATTR()
/.gfid/cd26bcc2-9b9f-455a-a2b2-9b6358f24203 => -1 (No data
available)<br style="" class="">
[2014-12-13 21:11:12.516674] W
[client-rpc-fops.c:1210:client3_3_removexattr_cbk]
0-data2-client-0: remote operation failed: No data available<br
style="" class="">
[2014-12-13 21:11:12.516705] W [fuse-bridge.c:1261:fuse_err_cbk]
0-glusterfs-fuse: 3325: REMOVEXATTR()
/.gfid/e6142b37-2362-4c95-a291-f396a122b014 => -1 (No data
available)<br style="" class="">
[2014-12-13 21:11:12.517577] W
[client-rpc-fops.c:1210:client3_3_removexattr_cbk]
0-data2-client-0: remote operation failed: No data available<br
style="" class="">
[2014-12-13 21:11:12.517600] W [fuse-bridge.c:1261:fuse_err_cbk]
0-glusterfs-fuse: 3331: REMOVEXATTR()
/.gfid/e6142b37-2362-4c95-a291-f396a122b014 => -1 (No data
available)<br style="" class="">
<br style="" class="">
and<br style="" class="">
[2014-12-13 21:57:16.741321] W
[syncdutils(slave):480:errno_wrap] <top>: reached maximum
retries (['.gfid/ba9c75ef-d4f7-4a6b-923f-82a8c7be4443',
'glusterfs.gfid.newfile',
'\x00\x00\x00\x1b\x00\x00\x00\x1bcab3ae81-7b52-4c55-ac33-37814ff374c4\x00\x00\x00\x81\xb0glustercli1.lower-test\x00\x00\x00\x01\xb0\x00\x00\x00\x00\x00\x00\x00\x00'])...<br
style="" class="">
[2014-12-13 21:57:22.400269] W
[syncdutils(slave):480:errno_wrap] <top>: reached maximum
retries (['.gfid/ba9c75ef-d4f7-4a6b-923f-82a8c7be4443',
'glusterfs.gfid.newfile',
'\x00\x00\x00\x1b\x00\x00\x00\x1bcab3ae81-7b52-4c55-ac33-37814ff374c4\x00\x00\x00\x81\xb0glustercli1.lower-test\x00\x00\x00\x01\xb0\x00\x00\x00\x00\x00\x00\x00\x00'])...<br
style="" class="">
<br style="" class="">
I didn't find what does this mean<br style="" class="">
</div>
</blockquote>
Does your slave had data before Geo-replication session is created.
From the log what I can see is, geo-rep is failing to create a file
in slave(may be same file name exists with different GFID(GFID is
GlusterFS unique identifier for file))<br>
Rsync is failing since file is not created in slave, and it is
unable to sync.<br>
<br>
--<br>
regards<br>
Aravinda<br>
<br style="" class="">
<blockquote
cite="mid:1719753123.302321.1418506032237.JavaMail.yahoo@jws11164.mail.ir2.yahoo.com"
type="cite">
<div style="color:#000; background-color:#fff;
font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial,
Lucida Grande, sans-serif;font-size:16px">any idea.<br style=""
class="">
<br style="" class="">
Regards<br>
<div class="qtdSeparateBR"><br>
<br>
</div>
<div style="display: block;" class="yahoo_quoted">
<div style="font-family: HelveticaNeue, Helvetica Neue,
Helvetica, Arial, Lucida Grande, sans-serif; font-size:
16px;">
<div style="font-family: HelveticaNeue, Helvetica Neue,
Helvetica, Arial, Lucida Grande, sans-serif; font-size:
16px;">
<div dir="ltr"> <font face="Arial" size="2"> Le Vendredi
12 décembre 2014 19h35, wodel youchi
<a class="moz-txt-link-rfc2396E" href="mailto:wodel_doom@yahoo.fr"><wodel_doom@yahoo.fr></a> a écrit :<br>
</font> </div>
<br>
<br>
<div class="y_msg_container">
<div id="yiv7532134357">
<div>
<div
style="color:#000;background-color:#fff;font-family:HelveticaNeue,
Helvetica Neue, Helvetica, Arial, Lucida Grande,
sans-serif;font-size:16px;">
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_10804"><span>Thanks
for your reply,</span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_10803"><br
clear="none">
</div>
<div dir="ltr">When executing the gverify.sh
script I had these errors on slave.log</div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_14169">[2014-12-12
18:12:45.423669] I
[options.c:1163:xlator_option_init_double]
0-fuse: option attribute-timeout convertion
failed value 1.0<br class="yiv7532134357"
style="" clear="none">
[2014-12-12 18:12:45.423689] E
[xlator.c:425:xlator_init] 0-fuse:
Initialization of volume 'fuse' failed, review
your volfile again<br clear="none">
</div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_14168"><br
clear="none">
<span></span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_10802"><span
id="yiv7532134357yui_3_16_0_1_1418405629302_10801">I think that the
problem is linked to the locale variables,
mine were</span></div>
<div dir="ltr"><span
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"><br
clear="none">
</span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_11580"><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">LANG=fr_FR.UTF-8<br
class="yiv7532134357" style="" clear="none">
LC_CTYPE=</span><span
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">"fr_FR.UTF-8"</span><br
clear="none">
</span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_11590"><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">LC_NUMERIC=</span><span
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">"fr_FR.UTF-8"</span><br
clear="none">
</span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_11598"><span
id="yiv7532134357yui_3_16_0_1_1418405629302_10801">LC_TIME="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_COLLATE="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_MONETARY="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_MESSAGES="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_PAPER="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_NAME="fr_FR.UTF-8"<br class="yiv7532134357"
style="" clear="none">
LC_ADDRESS="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_TELEPHONE="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_MEASUREMENT="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_IDENTIFICATION="fr_FR.UTF-8"<br
class="yiv7532134357" style="" clear="none">
LC_ALL=<br class="yiv7532134357" style=""
clear="none">
</span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_11630"><br
clear="none">
<span
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"></span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_11631"><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">I changed </span><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">LC_CTYPE and LC_NUMERIC to C and then
executed the gverify.sh script again and it
worked, but the gluster vol geo-rep ...
failed.</span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_12658"><br
clear="none">
<span class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style=""></span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_12659"><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">I then changed the /etc/locale.conf
file and modified the LANG from </span><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style=""><span class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style="">fr_FR.UTF-8 </span>to C, rebooted
the VM and voila, the geo-replication session
was created successfuly.</span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_12662"><br
clear="none">
</div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_12663">but
I am not sure if my changes won't affect other
things.</div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_12664"><br
clear="none">
</div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_15716">Regards<br
clear="none">
</div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_14927"><br
clear="none">
<span class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style=""></span></div>
<div dir="ltr"
id="yiv7532134357yui_3_16_0_1_1418405629302_14928"><span
class="yiv7532134357"
id="yiv7532134357yui_3_16_0_1_1418405629302_10801"
style=""><br clear="none">
</span></div>
<div class="yiv7532134357qtdSeparateBR"><br
clear="none">
<br clear="none">
</div>
<div class="yiv7532134357yqt0598168449"
id="yiv7532134357yqt94053">
<div class="yiv7532134357yahoo_quoted"
style="display:block;">
<div style="font-family:HelveticaNeue,
Helvetica Neue, Helvetica, Arial, Lucida
Grande, sans-serif;font-size:16px;">
<div style="font-family:HelveticaNeue,
Helvetica Neue, Helvetica, Arial, Lucida
Grande, sans-serif;font-size:16px;">
<div dir="ltr"> <font face="Arial"
size="2"> Le Vendredi 12 décembre 2014
8h30, Kotresh Hiremath Ravishankar
<a class="moz-txt-link-rfc2396E" href="mailto:khiremat@redhat.com"><khiremat@redhat.com></a> a écrit :<br
clear="none">
</font> </div>
<br clear="none">
<br clear="none">
<div class="yiv7532134357y_msg_container">Hi,<br
clear="none">
<br clear="none">
The setup is failing while doing
compatibility test between master and
slave cluster.<br clear="none">
The gverify.sh script is failing to get
master volume details for the same.<br
clear="none">
<br clear="none">
Could you run the following and paste
the output here?<br clear="none">
<br clear="none">
bash -x
/usr/local/libexec/glusterfs/gverify.sh
<master-vol-name> root
<slave-host-name>
<slave-vol-name>
<temp-log-file><br clear="none">
<br clear="none">
If source installed gverify.sh is found
in above path where as if rpm install,<br
clear="none">
it is found in
/usr/libexec/glusterfs/gverify.sh<br
clear="none">
<br clear="none">
If you are sure the master and slave
gluster versions and size is fine, the
easy workaround<br clear="none">
is to use force.<br clear="none">
<br clear="none">
gluster vol geo-replication
<master-vol>
<slave-host>::<slave-vol>
create push-pem force<br clear="none">
<br clear="none">
<br clear="none">
<br clear="none">
Thanks and Regards,<br clear="none">
Kotresh H R<br clear="none">
<div class="yiv7532134357yqt3668363615"
id="yiv7532134357yqtfd06448"><br
clear="none">
----- Original Message -----<br
clear="none">
From: "wodel youchi" <<a
moz-do-not-send="true"
rel="nofollow" shape="rect"
ymailto="mailto:wodel_doom@yahoo.fr"
target="_blank"
href="mailto:wodel_doom@yahoo.fr">wodel_doom@yahoo.fr</a>><br
clear="none">
To: <a moz-do-not-send="true"
rel="nofollow" shape="rect"
ymailto="mailto:gluster-users@gluster.org"
target="_blank"
href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br
clear="none">
Sent: Friday, December 12, 2014
3:13:48 AM<br clear="none">
Subject: [Gluster-users] Porblem
creating Geo-replication<br
clear="none">
<br clear="none">
Hi, <br clear="none">
I am using Centos7x64 updates <br
clear="none">
GlusterFS 3.6 from <a
moz-do-not-send="true"
rel="nofollow" shape="rect"
target="_blank"
href="http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/epel-7Server/">http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/epel-7Server/
</a>repository. <br clear="none">
No firewall and No Selinux. <br
clear="none">
<br clear="none">
<br clear="none">
I've two nodes with distributed
replicated volume: data1 <br
clear="none">
and a third node with a distributed
volume: data2 <br clear="none">
the two volumes have the same size <br
clear="none">
<br clear="none">
I've trouble to configure
geo-replication to the third node,
I've been following the RedHat Storage
3 Admin Guide, but It does not work. <br
clear="none">
<br clear="none">
I've created the ssh-passwordless
connection between the nodes, and
followed these commands <br
clear="none">
<br clear="none">
On the master: <br clear="none">
# gluster system: : execute
gsec_create <br clear="none">
Common secret pub file present at
/var/lib/glusterd/geo-replication/common_secret.pem.pub
<br clear="none">
<br clear="none">
# gluster volume geo-replication data1
node3.example.com::data2 create
push-pem <br clear="none">
Unable to fetch master volume details.
Please check the master cluster and
master volume. <br clear="none">
geo-replication command failed <br
clear="none">
<br clear="none">
from
/var/log/glusterfs/etc-glusterfs-glusterd.vol.log
I get these error messges <br
clear="none">
[2014-12-11 21:34:47.152644] E
[glusterd-geo-rep.c:2012:glusterd_verify_slave]
0-: Not a valid slave <br
clear="none">
[2014-12-11 21:34:47.152750] E
[glusterd-geo-rep.c:2240:glusterd_op_stage_gsync_create]
0-: node3.example.com::data2 is not a
valid slave volume. Error: Unable to
fetch master volume details. Please
check the master cluster and master
volume. <br clear="none">
[2014-12-11 21:34:47.152764] E
[glusterd-syncop.c:1151:gd_stage_op_phase]
0-management: Staging of operation
'Volume Geo-replication Create' failed
on localhost : Unable to fetch master
volume details. Please check the
master cluster and master volume. <br
clear="none">
[2014-12-11 21:35:25.559144] E
[glusterd-handshake.c:914:gd_validate_mgmt_hndsk_req]
0-management: Rejecting management
handshake request from unknown peer
192.168.1.9:1005 <br clear="none">
<br clear="none">
the 192.168.1.9 is the IP address of
the 3rd node. <br clear="none">
<br clear="none">
any idea!!. <br clear="none">
<br clear="none">
thanks <br clear="none">
<br clear="none">
<br clear="none">
<br clear="none">
<br clear="none">
_______________________________________________<br clear="none">
Gluster-users mailing list<br
clear="none">
<a moz-do-not-send="true"
rel="nofollow" shape="rect"
ymailto="mailto:Gluster-users@gluster.org"
target="_blank"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br
clear="none">
<a moz-do-not-send="true"
rel="nofollow" shape="rect"
target="_blank"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br
clear="none">
</div>
<br clear="none">
<br clear="none">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="yqt0598168449" id="yqt83015">_______________________________________________<br
clear="none">
Gluster-users mailing list<br clear="none">
<a moz-do-not-send="true" shape="rect"
ymailto="mailto:Gluster-users@gluster.org"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br
clear="none">
<a moz-do-not-send="true" shape="rect"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></div>
<br>
<br>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>