Hi Pranith. Sure, here is a pastebin sampling of logs from one of the hosts: <a href="http://pastebin.com/1U1ziwjC">http://pastebin.com/1U1ziwjC</a><br><br><div class="gmail_quote">On Mon, May 16, 2011 at 20:48, Pranith Kumar. Karampuri <span dir="ltr"><<a href="mailto:pranithk@gluster.com">pranithk@gluster.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">hi Remi,<br>
Would it be possible to post the logs on the client, so that we can find what issue you are running into.<br>
<br>
Pranith<br>
<div><div></div><div class="h5">----- Original Message -----<br>
From: "Remi Broemeling" <<a href="mailto:remi@goclio.com">remi@goclio.com</a>><br>
To: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Sent: Monday, May 16, 2011 10:47:33 PM<br>
Subject: [Gluster-users] Rebuild Distributed/Replicated Setup<br>
<br>
<br>
Hi,<br>
<br>
I've got a distributed/replicated GlusterFS v3.1.2 (installed via RPM) setup across two servers (web01 and web02) with the following vol config:<br>
<br>
volume shared-application-data-client-0<br>
type protocol/client<br>
option remote-host web01<br>
option remote-subvolume /var/glusterfs/bricks/shared<br>
option transport-type tcp<br>
option ping-timeout 5<br>
end-volume<br>
<br>
volume shared-application-data-client-1<br>
type protocol/client<br>
option remote-host web02<br>
option remote-subvolume /var/glusterfs/bricks/shared<br>
option transport-type tcp<br>
option ping-timeout 5<br>
end-volume<br>
<br>
volume shared-application-data-replicate-0<br>
type cluster/replicate<br>
subvolumes shared-application-data-client-0 shared-application-data-client-1<br>
end-volume<br>
<br>
volume shared-application-data-write-behind<br>
type performance/write-behind<br>
subvolumes shared-application-data-replicate-0<br>
end-volume<br>
<br>
volume shared-application-data-read-ahead<br>
type performance/read-ahead<br>
subvolumes shared-application-data-write-behind<br>
end-volume<br>
<br>
volume shared-application-data-io-cache<br>
type performance/io-cache<br>
subvolumes shared-application-data-read-ahead<br>
end-volume<br>
<br>
volume shared-application-data-quick-read<br>
type performance/quick-read<br>
subvolumes shared-application-data-io-cache<br>
end-volume<br>
<br>
volume shared-application-data-stat-prefetch<br>
type performance/stat-prefetch<br>
subvolumes shared-application-data-quick-read<br>
end-volume<br>
<br>
volume shared-application-data<br>
type debug/io-stats<br>
subvolumes shared-application-data-stat-prefetch<br>
end-volume<br>
<br>
In total, four servers mount this via GlusterFS FUSE. For whatever reason (I'm really not sure why), the GlusterFS filesystem has run into a bit of split-brain nightmare (although to my knowledge an actual split brain situation has never occurred in this environment), and I have been getting solidly corrupted issues across the filesystem as well as complaints that the filesystem cannot be self-healed.<br>
<br>
What I would like to do is completely empty one of the two servers (here I am trying to empty server web01), making the other one (in this case web02) the authoritative source for the data; and then have web01 completely rebuild it's mirror directly from web02.<br>
<br>
What's the easiest/safest way to do this? Is there a command that I can run that will force web01 to re-initialize it's mirror directly from web02 (and thus completely eradicate all of the split-brain errors and data inconsistencies)?<br>
<br>
Thanks!<br>
--<br>
<br>
Remi Broemeling<br>
System Administrator<br>
Clio - Practice Management Simplified<br>
<a href="tel:1-888-858-2546" value="+18888582546">1-888-858-2546</a> x(2^5) | <a href="mailto:remi@goclio.com">remi@goclio.com</a><br>
<a href="http://www.goclio.com" target="_blank">www.goclio.com</a> | blog | twitter | facebook<br>
<br>
</div></div>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br><br clear="all"><br>-- <br><div>Remi Broemeling</div><div>System Administrator</div><span style="font-family:arial, sans-serif;font-size:13px;border-collapse:collapse"><div><font face="arial, helvetica, sans-serif">Clio - Practice Management Simplified</font></div>
<div><font face="arial, helvetica, sans-serif">1-888-858-2546 x(2^5) | <a href="mailto:remi@goclio.com" target="_blank">remi@goclio.com</a></font></div><div><font face="arial, helvetica, sans-serif"><a href="http://www.goclio.com/" target="_blank">www.goclio.com</a> | <a href="http://www.goclio.com/blog" target="_blank">blog</a> | <a href="http://www.twitter.com/goclio" target="_blank">twitter</a> | <a href="http://www.facebook.com/goclio" target="_blank">facebook</a></font></div>
</span><br>