<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Jul 31, 2014 at 12:02 PM, Humble Devassy Chirammal <span dir="ltr"><<a href="mailto:humble.devassy@gmail.com" target="_blank">humble.devassy@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I second Jason, either the quorum=auto has to be disabled or just add one more server to the trusted pool and find the result .<span class="HOEnZb"><font color="#888888"><br>
<br></font></span></div><span class="HOEnZb"><font color="#888888">--Humble<br></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">
On Fri, Aug 1, 2014 at 12:22 AM, Jason Brooks <span dir="ltr"><<a href="mailto:jbrooks@redhat.com" target="_blank">jbrooks@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><br>
<br>
----- Original Message -----<br>
> From: "Vince Loschiavo" <<a href="mailto:vloschiavo@gmail.com" target="_blank">vloschiavo@gmail.com</a>><br>
> To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
> Sent: Thursday, July 31, 2014 9:22:16 AM<br>
> Subject: [Gluster-users] Virt-store use case - HA failure issue - suggestions needed<br>
><br>
> I'm currently testing Gluster 3.5.1 in a two server QEMU/KVM environment.<br>
> Centos 6.5:<br>
> Two servers (KVM07 & KVM08), Two brick (one brick per server) replicated<br>
> volume<br>
><br>
> I've tuned the volume per the documentation here:<br>
> <a href="http://gluster.org/documentation/use_cases/Virt-store-usecase/" target="_blank">http://gluster.org/documentation/use_cases/Virt-store-usecase/</a><br>
><br>
> I have the gluster volume fuse mounted on KVM07 and KVM08 and am using it<br>
> to store raw disk images.<br>
><br>
> KVM is using the fuse mounted volume as a "dir: Filesystem Directory:<br>
> storage pool.<br>
><br>
> With setting dynamic_ownership = 0 in /etc/libvirt/qemu.conf and chown-ing<br>
> the files to qemu:qemu, live migration works great.<br>
><br>
> Problem:<br>
> If I need to take down one of these servers for maintenance, I live migrate<br>
> the VMs to the other server.<br>
> service gluster stop<br>
> then kill all the remaining gluster and brick processes.<br>
<br>
</div>The guide says that quorum-type=auto sets a rule such that at least half<br>
of the bricks in the replica group should be UP and running. If not,<br>
the replica group becomes read-only. I think the rule is actually 51%,<br>
so bringing down one of the two servers makes your volume read-only.<br>
<br>
If you want two servers, you need to unset this rule. Better to add a<br>
third server and a third replica, though.<br>
<br>
Regards, Jason<br>
<div><br>
<br>
><br>
> At this point, the VMs die. The Fuse mount recovers and remains attached<br>
> to the volume via the other server, but the VIRT disk images are not fully<br>
> synced.<br>
><br>
> This causes the VMs to go into a read-only files system state, then kernel<br>
> panic. Reboots/restarts of the VMs just cause kernel panics. This<br>
> effectively brings down the two node cluster.<br>
><br>
> Bringing back up the gluster node / bricks /etc, prompts a self-heal. Once<br>
> self-heal is completed, the VMs can boot normally.<br>
><br>
> Question: is there a better way to accomplish HA with live/running Virt<br>
> images? The goal is to be able to bring down any one server in the pair<br>
> and perform maintenance without interrupting the VMs.<br>
><br>
> I assume my shutdown process is flawed but haven't been able to find a<br>
> better process.<br>
><br>
> Any suggestions are welcome.<br>
><br>
><br>
> --<br>
> -Vince Loschiavo<br>
><br>
</div>> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> <a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div>That was it. Thank you. I'm somewhat space constrained in my lab, so I chose to disable quorum and set server-quorum to 50%. I assume that was redundant, but it works for me.</div>
<div><br></div><div><br></div><div><br></div>-- <br>-Vince Loschiavo<br>
</div></div>