<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 6 July 2014 19:17, Vijay Bellur <span dir="ltr"><<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div>On 07/01/2014 05:13 PM, Jonathan Barber wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Hello all,<br>
<br>
I'm investigating GlusterFS+Swift for use in a "large" (starting at<br>
~150TB) scale out file system for storing and serving photographic images.<br>
<br>
Currently I'm thinking of using servers with JBODs and it's clear how to<br>
use Gluster's replication sets to give resiliency at the server level.<br>
However, I'd like to have multiple bricks per server (with a brick per<br>
drive controller) and managing the replication sets starts to look more<br>
complicated from a management point of view. Also,when it comes to<br>
expanding the solution in the future, I reckon that I will be adding<br>
bricks of different sizes with different numbers of bricks per server -<br>
further complicating management.<br>
<br>
So, I was wondering if there is support for (or plans for) failure<br>
domains (like Oracle's ASM failure groups) which would allow you to<br>
describe groups of bricks within which replicas can't be co-located?<br>
(e.g. bricks from the same server are placed in the same failure domain,<br>
meaning that no-two replicas are allowed on these groups of bricks).<br>
<br>
</blockquote>
<br></div></div>
A warning does already get displayed in CLI when bricks from the same server are attempted to be part of the same replica set:<br>
<br>
[root@deepthought lo]# gluster volume create myvol replica 2 deepthought:/d/brick1 deepthought:/d/brick2<br>
Multiple bricks of a replicate volume are present on the same server. This setup is not optimal.<br>
Do you still want to continue creating the volume? (y/n) n<br>
Volume create failed<br></blockquote><div><br></div><div>Yes, I'd seen that warning. It isn't reported when adding bricks to an existing volume though, e.g.:</div><div><br></div><div># gluster volume add-brick myvol $HOSTNAME:/d/brick{1,2}</div>
<div>volume add-brick: success<br></div><div>#</div><div><br></div><div>(with Gluster 3.5.1 from the gluster-epel repo)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
What other policies would you be interested in associating with failure domains?<br></blockquote><div><br></div><div>I was also thinking about failure domains that span hosts (perhaps because some of the machines in a volume share a single point of failure such as a top of rack switch, the same UPS, or are in the same room). It would also then be possible to have a brick per drive and so not need RAID in the server (if we have cross-server replication, I don't think additional replication from RAID is necessary).</div>
<div><br></div><div>It'd also be nice to able to just say "I want 2 replicas, but I don't care which bricks they are on as long as they aren't in the same failure domain". This could let you have odd numbers of servers without having to manually slice the storage and place the replicas.</div>
<div><br></div><div>Obviously, I have no idea about the internals of Gluster so I don't know how complicated this is to achieve.</div><div><br></div><div>Cheers</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Regards,<br>
Vijay<br>
<br>
<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Jonathan Barber <<a href="mailto:jonathan.barber@gmail.com" target="_blank">jonathan.barber@gmail.com</a>>
</div></div>