<span style>Here's another ZooKeeper management framework that may be useful. It's called Curator, developed by Netflix, and recently released as open source. It probably has a bit more inertia than ZkFarmer too.</span><div style>
<br></div><div style><a href="http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html" target="_blank" style="color:rgb(17,85,204)">http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html</a></div>
<div style><br></div><div style><a href="https://github.com/Netflix/curator" target="_blank" style="color:rgb(17,85,204)">https://github.com/Netflix/curator</a></div><div style><br></div><div style>HTH</div><div style><br>
</div><div style>-louis</div><div style><br></div><br><div class="gmail_quote">On Mon, May 7, 2012 at 10:43 AM, Jeff Darcy <span dir="ltr"><<a href="mailto:jdarcy@redhat.com" target="_blank">jdarcy@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I've long felt that our ways of dealing with cluster membership and staging of<br>
config changes is not quite as robust and scalable as we might want.<br>
Accordingly, I spent a bit of time a couple of weeks ago looking into the<br>
possibility of using ZooKeeper to do some of this stuff. Yeah, it brings in a<br>
heavy Java dependency, but when I looked at some lighter-weight alternatives<br>
they all seemed to be lacking in more important ways. Basically the idea was<br>
to do this:<br>
<br>
* Set up the first N (e.g. N=3) nodes in our cluster as ZooKeeper servers, or<br>
point everyone at an existing ZooKeeper cluster.<br>
<br>
* Use ZK ephemeral nodes as a way to track cluster membership ("peer probe"<br>
merely updates ZK, and "peer status" merely reads from it).<br>
<br>
* Store config information in ZK *once* instead of regenerating volfiles etc.<br>
on every node (and dealing with the ugly cases where a node was down when the<br>
config change happened).<br>
<br>
* Set watches on ZK nodes to be notified when config changes happen, and<br>
respond appropriately.<br>
<br>
I eventually ran out of time and moved on to other things, but this or<br>
something like it (e.g. using Riak Core) still seems like a better approach<br>
than what we have. In that context, it looks like ZkFarmer[1] might be a big<br>
help. AFAICT someone else was trying to solve almost exactly the same kind of<br>
server/config problem that we have, and wrapped their solution into a library.<br>
Is this a direction other devs might be interested in pursuing some day,<br>
if/when time allows?<br>
<br>
<br>
[1] <a href="https://github.com/rs/zkfarmer" target="_blank">https://github.com/rs/zkfarmer</a><br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br>