<div dir="ltr">How sure are we that this is having the intended effect?<div><br></div><div>I have two gluster nodes with bricks which are also ovirt hosts, and the replicated volume configured as such:</div><div><br></div>
<div><div>gluster volume info rep1</div><div> </div><div>Volume Name: rep1</div><div>Type: Replicate</div><div>Volume ID: 5876746d-5977-4fa9-a829-69f44b3364ec</div><div>Status: Started</div><div>Number of Bricks: 1 x 2 = 2</div>
<div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: 10.0.10.2:/mnt/storage/lv-storage-domain/rep1</div><div>Brick2: 10.0.10.3:/mnt/storage/lv-storage-domain/rep1</div><div>Options Reconfigured:</div><div>diagnostics.client-log-level: ERROR</div>
<div>storage.owner-gid: 36</div><div>storage.owner-uid: 36</div><div>server.allow-insecure: on</div><div>cluster.quorum-type: fixed</div><div>nfs.trusted-sync: off</div><div>nfs.trusted-write: on</div><div>performance.quick-read: off</div>
<div>performance.read-ahead: off</div><div>performance.io-cache: off</div><div>performance.stat-prefetch: off</div><div>cluster.eager-lock: enable</div><div>network.remote-dio: enable</div><div>cluster.server-quorum-type: server</div>
</div><div><br></div><div>I also set the bricks to 1 (<span style="font-family:arial,sans-serif;font-size:13px">cluster.quorum-count 1, although its not showing above) as this option seems related to storage bricks, not gluster peers.</span></div>
<div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">I peered with a 3rd gluster node which doesn't have any volumes configured. I've tested this setup, and the storage doesn't go down when one of the nodes is rebooted.</span></div>
<div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">There still seems to be a high potential for split brain here.</span></div><div>
<span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><font face="arial, sans-serif"><a href="http://gluster.org/community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options">http://gluster.org/community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options</a></font></div>
<div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><table border="1" cellpadding="5" cellspacing="0" style="color:rgb(0,0,0);font-family:arial,sans-serif;line-height:16.899999618530273px">
<tbody><tr><td><span id="cluster.quorum-type">cluster.quorum-type</span></td><td>Method used for quorum enforcement. "None" means no quorum enforcement, which is the historical behavior. "Auto" means quorum is set to be more than half of the <b>bricks</b> in a subvolume, or exactly half if that includes the first listed brick. "Fixed" means quorum is set to a value specified by <a href="http://gluster.org/community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options#cluster.quorum-count" style="color:rgb(0,87,158);text-decoration:none">cluster.quorum-count</a>. If quorum is not met, then modifing operations such as <i>write</i> will fail with EROFS. This prevents most cases of "split brain" which result from conflicting writes to different bricks.</td>
<td>None</td></tr><tr><td><span id="cluster.quorum-count">cluster.quorum-count</span></td><td>Number of subvolumes that must be present to achieve quorum, as described for <a href="http://gluster.org/community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options#cluster.quorum-type" style="color:rgb(0,87,158);text-decoration:none">cluster.quorum-type</a>. This value is not used unless <i>cluster.quorum-type</i> is "fixed".</td>
<td>0</td></tr></tbody></table></div><div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">We don't seem able to configure "allow 50% of the bricks to be active, as long as 51% of the gluster peers are connected".</span></div>
<div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">I also don't understand why 'subvolumes' are referred to here. Is this old terminology for 'volumes'?</span></div>
<div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">Thoughts?</span></div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr">
<span style="font-family:arial,sans-serif;font-size:16px"><strong>Steve Dainard </strong></span><span style="font-size:12px"></span><br>
<span style="font-family:arial,sans-serif;font-size:12px">IT Infrastructure Manager<br>
<a href="http://miovision.com/" target="_blank">Miovision</a> | <em>Rethink Traffic</em><br><br>
<strong style="font-family:arial,sans-serif;font-size:13px;color:#999999"><a href="http://miovision.com/blog" target="_blank">Blog</a> | </strong><font color="#999999" style="font-family:arial,sans-serif;font-size:13px"><strong><a href="https://www.linkedin.com/company/miovision-technologies" target="_blank">LinkedIn</a> | <a href="https://twitter.com/miovision" target="_blank">Twitter</a> | <a href="https://www.facebook.com/miovision" target="_blank">Facebook</a></strong></font> </span>
<hr style="font-family:arial,sans-serif;font-size:13px;color:#333333;clear:both">
<div style="color:#999999;font-family:arial,sans-serif;font-size:13px;padding-top:5px">
<span style="font-family:arial,sans-serif;font-size:12px">Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, Canada | N2C 1L3</span><br>
<span style="font-family:arial,sans-serif;font-size:12px">This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.</span></div>
</div></div>
<br><br><div class="gmail_quote">On Mon, Mar 10, 2014 at 7:58 AM, James <span dir="ltr"><<a href="mailto:purpleidea@gmail.com" target="_blank">purpleidea@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
A quick search reveals:<br>
<br>
# Sets the quorum percentage for the trusted storage pool.<br>
cluster.server-quorum-ratio<br>
<br>
# If set to server, enables the specified volume to<br>
participate in quorum.<br>
cluster.server-quorum-type<br>
<br>
# If quorum-type is "fixed" only allow writes if this many<br>
bricks or present. Other quorum types will OVERWRITE this value.<br>
cluster.quorum-count<br>
<br>
# If value is "fixed" only allow writes if quorum-count bricks<br>
are present. If value is "auto" only allow writes if more than half of<br>
bricks, or exactly half including the first, are present.<br>
cluster.quorum-type<br>
<br>
I took these from my previous "notes" (code) in:<br>
<a href="https://github.com/purpleidea/puppet-gluster/blob/master/manifests/volume/property/data.pp#L18" target="_blank">https://github.com/purpleidea/puppet-gluster/blob/master/manifests/volume/property/data.pp#L18</a><br>
<br>
You can get newer values or appropriate values for your version by<br>
running something like:<br>
<br>
gluster volume set help ( i think )<br>
<br>
Cheers,<br>
James<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
On Mon, Mar 10, 2014 at 2:43 AM, Andrew Lau <<a href="mailto:andrew@andrewklau.com">andrew@andrewklau.com</a>> wrote:<br>
> Thanks again James!<br>
><br>
> So far so good, I plan to test this a little more in a few days but so far<br>
> it seems the only volume setting I need is:<br>
> cluster.server-quorum-type: server<br>
><br>
> Default cluster.server-quorum-ratio >50%<br>
> So 2 is greater than 1.5.. which should allow writes.<br>
><br>
> On Thu, Mar 6, 2014 at 5:00 PM, James <<a href="mailto:purpleidea@gmail.com">purpleidea@gmail.com</a>> wrote:<br>
>><br>
>> Top posting sorry:<br>
>><br>
>> Yes, you can add a third "arbiter" node, that exists to help with the<br>
>> quorum issues. AFAICT, all you do is peer is with the cluster (as you<br>
>> did with the other hosts) but don't add any storage for example.<br>
>><br>
>> Then you set the cluster.quorum* style volume settings that you're<br>
>> interested. I don't have a list of exactly which ones off the top of<br>
>> my head, but if you make a list, let me know!<br>
>><br>
>> Cheers,<br>
>> James<br>
>><br>
>><br>
>> On Wed, Mar 5, 2014 at 10:51 PM, Andrew Lau <<a href="mailto:andrew@andrewklau.com">andrew@andrewklau.com</a>> wrote:<br>
>> > Hi,<br>
>> ><br>
>> > I'm looking for an option to add an arbiter node to the gluster<br>
>> > cluster, but the leads I've been following seem to lead to<br>
>> > inconclusive results.<br>
>> ><br>
>> > The scenario is, a 2 node replicated cluster. What I want to do is<br>
>> > introduce a fake host/arbiter node which would set the cluster to a 3<br>
>> > node meaning, we can meet the conditions of allow over 50% to write<br>
>> > (ie. 2 can write, 1 can not).<br>
>> ><br>
>> > elyograg from IRC gave me a few links [1], [2]<br>
>> > But these appear to be over a year old, and still under review.<br>
>> ><br>
>> > Gluster 3.2 volume options (I'm running 3.4, but there doesn't seem to<br>
>> > be an updated page) [3]<br>
>> > seem to state the that the cluster quorum is identified by active<br>
>> > peers. This also backs up the statement in [2] in regards to a patch<br>
>> > for active volumes rather than cluster peers.<br>
>> ><br>
>> > Has anyone gone down this path, or could they confirm any of these<br>
>> > leads? (ie. does a host w/o any volumes get considered as a peer<br>
>> > within the cluster)<br>
>> ><br>
>> > Thanks,<br>
>> > Andrew<br>
>> ><br>
>> > [1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=914804" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=914804</a><br>
>> > [2] <a href="http://review.gluster.org/#/c/4363/" target="_blank">http://review.gluster.org/#/c/4363/</a><br>
>> > [3]<br>
>> > <a href="http://gluster.org/community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options#cluster.quorum-type" target="_blank">http://gluster.org/community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options#cluster.quorum-type</a><br>
>> > _______________________________________________<br>
>> > Gluster-users mailing list<br>
>> > <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
>> > <a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
><br>
><br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</div></div></blockquote></div><br></div>