<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">That should not happen. After a replica failover the "now" active node should continue where the "old" active node left off.</div>
<div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Could you provide geo-replication logs from master and slave after reproducing this (with changelog mode).</div>
<div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Thanks,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">
-venky</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 17, 2014 at 9:34 PM, CJ Beck <span dir="ltr"><<a href="mailto:chris.beck@workday.com" target="_blank">chris.beck@workday.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>I did set it intentionally because I found a case where files would be missed during geo-replication. Xsync seemed to handle the case better. The issue was when you bring the “Active” node down that is handling the geo-replication session, and it’s set
to ChangeLog as the change method. Any files that are written into the cluster while geo-replication is down (eg, while the geo-replication session is being failed to another node), are missed / skipped, and won’t ever be transferred to the other cluster.</div>
<div><br>
</div>
<div>Is this the expected behavior? If not, then I can open a bug on it.</div>
<div><br>
</div>
<div>-CJ</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>Venky Shankar <<a href="mailto:yknev.shankar@gmail.com" target="_blank">yknev.shankar@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Wednesday, April 16, 2014 at 4:43 PM<div><div class="h5"><br>
<span style="font-weight:bold">To: </span>CJ Beck <<a href="mailto:chris.beck@workday.com" target="_blank">chris.beck@workday.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Gluster-users] Question about geo-replication and deletes in 3.5 beta train<br>
</div></div></div><div><div class="h5">
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div class="gmail_extra">
<div class="gmail_quote">On Thu, Apr 17, 2014 at 3:01 AM, CJ Beck <span dir="ltr">
<<a href="mailto:chris.beck@workday.com" target="_blank">chris.beck@workday.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>I did have the “change_detector” set to xsync, which seems to be the issue (bypassing the changelog method). So I can fix that and see if the deletes are propagated.</div>
</div>
</blockquote>
<div><br>
</div>
<div>
<div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Was that set intentionally? Setting this as the main change detection mechanism would crawl the filesystem every 60 seconds to replicate the changes. Changelog mode handles live changes,
so any deletes that were performed before this option was set would not be propagated.</div>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div><br>
</div>
<div>Also, is there a way to tell the geo-replication to go ahead and walk the filesystems to do a “sync” so the remote side files are deleted, if they are not on the source?</div>
</div>
</blockquote>
<div><br>
</div>
<div>
<div class="gmail_default" style="font-family:arial,helvetica,sans-serif">As of now, no. With distributed geo-replication, the geo-rep daemon crawls the bricks (instead of the mount). Since the brick would have a subset of the file system entities (for e.g.
in a distributed volume), it's hard to find out purged entries without having to crawl the mount and comparing the entries b/w master and slave (which is slow). This is where changelog mode helps.</div>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div><br>
</div>
<div>Thanks for the quick reply!</div>
<div><br>
</div>
<div>
<div>[root@host ~]# gluster volume geo-replication test-poc 10.10.1.120::test-poc status detail</div>
<div><br>
</div>
<div>MASTER NODE MASTER VOL MASTER BRICK SLAVE STATUS CHECKPOINT STATUS CRAWL STATUS FILES SYNCD FILES PENDING BYTES PENDING DELETES PENDING FILES SKIPPED</div>
<div>----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------</div>
<div><a href="http://host1.com" target="_blank">host1.com</a> test-poc /data/test-poc 10.10.1.120::test-poc Passive N/A N/A 382 0 0 0 0</div>
<div><a href="http://host2.com" target="_blank">host2.com</a> test-poc /data/test-poc 10.10.1.122::test-poc Passive N/A N/A 0 0 0 0 0</div>
<div><a href="http://host3.com" target="_blank">host3.com</a> test-poc /data/test-poc 10.10.1.121::test-poc Active N/A Hybrid Crawl 10765 70 0 0 0</div>
</div>
<div><br>
</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>Venky Shankar <<a href="mailto:yknev.shankar@gmail.com" target="_blank">yknev.shankar@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Wednesday, April 16, 2014 at 1:54 PM<br>
<span style="font-weight:bold">To: </span>CJ Beck <<a href="mailto:chris.beck@workday.com" target="_blank">chris.beck@workday.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Gluster-users] Question about geo-replication and deletes in 3.5 beta train<br>
</div>
<div>
<div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div style="font-family:arial,helvetica,sans-serif">"ignore-deletes" is only valid in the initial crawl mode[1] where it does not propagate deletes to the slave (changelog mode does). Was the session restarted by any chance?</div>
<div style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif">[1] Geo-replication now has two internal operations modes: a one shot filesystem crawl mode (used to replicate data already present in a volume) and the changelog mode (for replicating live changes).</div>
<div style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div style="font-family:arial,helvetica,sans-serif">Thanks,<br>
</div>
<div style="font-family:arial,helvetica,sans-serif">-venky</div>
<div style="font-family:arial,helvetica,sans-serif"><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu, Apr 17, 2014 at 1:25 AM, CJ Beck <span dir="ltr">
<<a href="mailto:chris.beck@workday.com" target="_blank">chris.beck@workday.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;word-wrap:break-word">
<div style="font-family:Calibri,sans-serif">I have an issue where deletes are not being propagated to the slave cluster in a geo-replicated environment. I’ve looked through the code, and it appears as though this is something that might have been changed to
be hard coded?</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">When I try to change it via a config option on the command line, it replies with a “reserved option” error:</div>
<div>
<div><font face="Courier">[root@host ~]# gluster volume geo-replication test-poc 10.10.1.120::test-poc config ignore_deletes 1</font></div>
<div><font face="Courier">Reserved option</font></div>
<div><font face="Courier">geo-replication command failed</font></div>
<div><font face="Courier">[root@host ~]# gluster volume geo-replication test-poc 10.10.1.120::test-poc config ignore-deletes 1</font></div>
<div><font face="Courier">Reserved option</font></div>
<div><font face="Courier">geo-replication command failed</font></div>
<div><font face="Courier">[root@host ~]#</font></div>
</div>
<br>
<div style="font-family:Calibri,sans-serif">Looking at the source code (although, I’m not a C expert by any means), it seems as though it’s hard-coded to be “true” all the time?</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">(from glusterd-geo-rep.c):</div>
<div>
<div><font face="Courier">4285 /* ignore-deletes */</font></div>
<div><font face="Courier">4286 runinit_gsyncd_setrx (&runner, conf_path);</font></div>
<div><font face="Courier">4287 runner_add_args (&runner, "ignore-deletes", "true", ".", ".", NULL);</font></div>
<div><font face="Courier">4288 RUN_GSYNCD_CMD;</font></div>
</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Any ideas how to get deletes propagated to the slave cluster?</div>
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">Thanks!</div>
<span><font color="#888888">
<div style="font-family:Calibri,sans-serif"><br>
</div>
<div style="font-family:Calibri,sans-serif">-CJ</div>
</font></span></div>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</span></div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div></div></span>
</div>
</blockquote></div><br></div>