<div class="gmail_quote">2009/9/15 Stephan von Krawczynski <span dir="ltr"><<a href="mailto:skraw@ithnet.com">skraw@ithnet.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div></div><div class="h5">On Mon, 14 Sep 2009 21:20:49 +0200<br>
"Steve" <<a href="mailto:steeeeeveee@gmx.net">steeeeeveee@gmx.net</a>> wrote:<br>
<br>
><br>
> -------- Original-Nachricht --------<br>
> > Datum: Mon, 14 Sep 2009 21:14:32 +0200<br>
> > Von: Stephan von Krawczynski <<a href="mailto:skraw@ithnet.com">skraw@ithnet.com</a>><br>
> > An: Anand Avati <<a href="mailto:avati@gluster.com">avati@gluster.com</a>><br>
> > CC: <a href="mailto:gluster-devel@nongnu.org">gluster-devel@nongnu.org</a><br>
> > Betreff: Re: [Gluster-devel] solutions for split brain situation<br>
><br>
> > On Mon, 14 Sep 2009 21:44:12 +0530<br>
> > Anand Avati <<a href="mailto:avati@gluster.com">avati@gluster.com</a>> wrote:<br>
> ><br>
> > > > Our "split brain" is no real split brain and looks like this: Logfiles<br>
> > are<br>
> > > > written every 5 mins. If you add a secondary server that has 14 days<br>
> > old<br>
> > > > logfiles on it you notice that about half of your data vanishes while<br>
> > not<br>
> > > > successful self heal is performed, because the old logfiles read from<br>
> > the<br>
> > > > secondary server overwrite the new logfiles on your primary while new<br>
> > data is<br>
> > > > added to them.<br>
> > ><br>
> > > Have you been using favorite-child option?<br>
> ><br>
> > No, the option was not used.<br>
> ><br>
> > > Auto resolving of<br>
> > > split-brain is bound to make you lose data of one of the subvolumes.<br>
> > > If you had indeed specified favorite-child option, and the<br>
> > > favorite-child option happens to be the server which had 14day old<br>
> > > logs, what just happened was exactly what was in the elaborate warning<br>
> > > log.<br>
> > ><br>
> > > Now what is more interesting for me is, the sequence of taking down<br>
> > > and bringing up the servers you followed to split brain? Was is really<br>
> > > just taking one server (any of them) down and bringing it back up? Did<br>
> > > you face a split brain with just this? Can you please describe the<br>
> > > minimal steps necessary to reproduce your issue?<br>
> ><br>
> > Take 2 servers and one client. Use a minimal replicate setup but do _not_<br>
> > add<br>
> > the second server. Copy some data on the first server via glusterfs, then<br>
> > rsync that data on the second server directly from the first server<br>
> > (glusterfsd not yet active there). Now change some of the data to have<br>
> > files<br>
> > that are really newer as your rsync cycle. Then start glusterfsd on the<br>
> > second<br>
> > server. Your client will add it. Then open the newer files r/w on the<br>
> > client.<br>
> > You will notice the split brain messages in the client logs and find that<br>
> > every<br>
> > other file gets indeed read in from the second (outdated) server fileset.<br>
> > Write it back and your newer files on the first server are gone.<br>
> > As said, no favorite child option set.<br>
> ><br>
> You just rsynced but did you synced the extended attributes as well?<br>
<br>
</div></div>No, we explicitly did not sync the extended attributes. But your question<br>
should be placed more general: if I have a working glusterfs server, must all<br>
data be backuped including extended attributes?<br>
Why should it be lethal not to backup them, when I can get data online by<br>
simply starting to export it via glusterfsd that has not been touched by<br>
glusterfsd before? (think of a first-time export, you have some data and<br>
install glusterfs for the very first time. Your data is of course exported<br>
without any troubles. Where is the difference to a rsync backup with no<br>
extended attributes?<br></blockquote><div> <br>Can we all read this in relation to extended attributes and the cluster/replicate translator. <a href="http://gluster.com/community/documentation/index.php/Understanding_AFR_Translator">Understanding AFR translator</a><br>
Also, if you want more reliable restores directly to a single storage brick (rather than restoring onto a replicate translator) I would suggest you have a backup system that handles extended attributes. I am using bacula for this purpose, but you may find other solutions that fit.<br>
<br>Regards,<br>Michael Cassaniti<br></div></div>