<div dir="ltr">James, can we agree to disagree on that ? <div><br></div><div>First, one of the main ideas of having a replicated/distributed filesystem is to have your data safe, and avoiding a single point of failure or, at the very least, minimizing the points of failure.</div>
<div><br></div><div>By adding multiple bricks per server, you are increasing your risk.</div><div><br></div><div>Regarding performance, glusterd will run on a single core. The more work you push to it (multiple volumes) the quicker it will saturate and cap your performance. (will, in all fairness, it will happen also with several connections, so, let's call this even).</div>
<div><br></div><div>Again, on performance, splitting volumes on a RAID6 will only make yo literally lose diskspace IF you divide you volumes on the controller. Remember: 6 disks on raid6 means having the available space of 4 disks, since the other two are "spares" (this is a simplification). On the other hand, if you use all of the 12 disks, you will have the available space of 10 disks, instead of 4x2=8.</div>
<div><br></div><div>If the alternative create a single volume on the RAID controller, and the create two logical volumes on the OS, STILL, you have one single RAID controller, thus, you might still lose, since the OS would have some overhead to control two volumes. </div>
<div><br></div><div>Now, REALLY important factors is, indeed, having "symmetrical" settings, like same processors, same disk configuration for bricks, same mount of ram, same NIC configurations, just to rule out all of those potential problems that would make one node wait for the other. </div>
<div><br></div><div>Remember: gluster write, as it would be expected, is only as fast as the slowest node, thus affecting performance greatly.</div><div><br></div><div>Hope this helps.</div><div><br></div><div><br></div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 21, 2014 at 7:20 PM, James <span dir="ltr"><<a href="mailto:purpleidea@gmail.com" target="_blank">purpleidea@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">On Fri, Mar 21, 2014 at 2:02 PM, Justin Dossey <<a href="mailto:jbd@podomatic.com">jbd@podomatic.com</a>> wrote:<br>
> The more bricks you allocate, the higher your operational complexity. One<br>
> brick per server is perfectly fine.<br>
<br>
</div>I don't agree that you necessarily have a "higher operational<br>
complexity" by adding more bricks per host. Especially if you're using<br>
Puppet-Gluster [1] to manage it all ;) I do think you'll have higher<br>
complexity if your cluster isn't homogenous or your bricks per host<br>
isn't symmetrical across the cluster, or if you're using chaining.<br>
Otherwise, I think it's recommended to use more than one brick.<br>
<br>
There are a number of reasons why you might want more than one brick per host.<br>
<br>
* Splitting of large RAID sets (might be better to have 2x12 drives<br>
per RAID6, instead of one giant RAID6 set)<br>
<br>
* More parallel IO workload (you might want to see how much<br>
performance gains you get from more bricks and your workload. keep<br>
adding until you plateau. Puppet-Gluster is a useful tool for<br>
deploying a cluster (vm's or iron) to test a certain config, and then<br>
using it again to re-deploy and test a new brick count).<br>
<br>
* More than one brick per server is required if you want to do volume<br>
chaining. (Advanced, unsupported feature, but has cool implictions.)<br>
<br>
And so on...<br>
<br>
The famous "semiosis" has given at least one talk, explaining how he<br>
chose his brick count, and detailing his method. I believe he uses 6<br>
or 8 bricks per host. If there's a reference, maybe he can chime in<br>
and add some context.<br>
<br>
HTH,<br>
James<br>
<br>
[1] <a href="https://github.com/purpleidea/puppet-gluster" target="_blank">https://github.com/purpleidea/puppet-gluster</a><br>
<div class="HOEnZb"><div class="h5">_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</div></div></blockquote></div><br></div>