<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">On 9/9/2014 7:12 PM, Michael Bushey
wrote:<br>
</div>
<blockquote
cite="mid:CAPJTyAVcZHxX=xhKOb9QACUsnnxXZghCukKQCdLU3gqcVwm2hA@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div dir="ltr">
<div>I'm currently using GlusterFS 3.5.2 on a pair of production
servers to share uploaded files and it's been reliable and
working well with just the two servers. I've done some local
tests of trying to add and remove servers and the results have
not been good. I'm starting to think I have the wrong idea of
what GlusterFS does and I'm trying to stick the square peg in
the round hole. Is there some config to say "number of
replicas = number of servers"?</div>
<div><br>
</div>
<div>My need is to have a copy of the data local on each server.
If I have 5 servers, I want five copies of the data. Basically
each server should have it's own copy like it's a local
ext4/zfs file-system, except it will immediately sync files
added or removed on the other servers. I'm not sure the what
the gluster definition of brick is, but I believe I want the
number of bricks to equal the number of servers (at least for
this particular file-system).</div>
<div><br>
</div>
<div>I've played around a bit and lost my data every time I've
tried to add or remove a node. There is plenty of
documentation, but it all says the same thing and doesn't
answer the basics.</div>
<div><br>
</div>
<div>From `gluster volume info`: Number of Bricks: 1 x 2 = 2</div>
<div><br>
</div>
<div>I'm assuming the middle 2 is the number of bricks. I have
no clue why we're multiplying by 1 to get itself.</div>
<div><a moz-do-not-send="true"
href="http://gluster.org/community/documentation/index.php/Gluster_3.2:_Displaying_Volume_Information">http://gluster.org/community/documentation/index.php/Gluster_3.2:_Displaying_Volume_Information</a>
- does not show this magic multiplication. The page for 3.3
does not exist.</div>
<div><br>
</div>
<div>We're cloud based and we want to be able to add and remove
servers on the fly. If for example I want to scale from 5 to 7
servers, it looks like I need to create a new storage pool
with 7 replicas. Would it be the most logical to name my brick
with the number of replicas? For example server1 to server5
already have /var/gluster/files5. I could then create
/var/gluster/files7 on server1 to server7, create the pool
with 7 replicas, then pick a machine like server1 to copy the
data from /var/gluster/files5 to /var/gluster/files7, then
destroy /var/gluster/files5. This seems extremely convoluted
but it does not appear possible to expand a storage pool and
expand the number of replicants with it.</div>
<div><br>
</div>
<div>Thanks in advance for help and your time to read this.</div>
<div>Michael</div>
</div>
</blockquote>
I have done this a few times, but it is not real recent, so I will
try to explain in a way that is both correct and understandable.<br>
<br>
1. As to the question about the "`gluster volume info`: Number of
Bricks: 1 x 2 = 2"<br>
The first number is the number of bricks that the data is
distributed across. If that number is 3, each of three bricks has
1/3 of the data. This is the "Distribute" mode of operating
gluster. From what you describe, you are not interested in that.
You are interested in the "Replicate" mode. The numbering was
adopted because gluster makes it possible to use various
combinations of Distribute and Replicate at the same time, and this
numbering describes the arrangement very succinctly.<br>
<br>
2. As to how to you add a brick to your volume, so you can add a
server to your system. You use the 'gluster volume add-brick'
command. You are telling gluster: You are currently a two-brick
system and I want to make you a 3-brick system (each brick being on
a different server) by adding 'server3'. Typing 'gluster volume
add-brick help' returns<br>
<pre>gluster volume add-brick <VOLNAME> [<stripe|replica> <COUNT>] <NEW-BRICK> ... [force]</pre>
Assuming that the brick is ready and freshly formatted with xfs, the
command ends up looking something like:<br>
<pre>gluster volume add-brick upload replica 3 server3:/bricks/upload/upload</pre>
This will create the gluster structure on the brick, and will start
the process of replicating the gluster data onto the brick. The
gluster mount should be usable at that point, pulling the data from
other bricks if it has not been replicated onto the local brick yet.<br>
<br>
If this server has been in service before, it is probably best to
reformat the brick before putting it into service. Otherwise
gluster will see remnants of the past use on the brick, and not want
to use it, assuming there may be valuable information already stored
there.<br>
<br>
CAUTION: Unless they have fixed this issue, the replication process
can bring a server to its knees until it is fully replicated.<br>
<br>
3. To remove a server from your system, the process is basically
reversed. If you want to go from 5 servers to 4, you issue the
command:<br>
<pre>gluster volume remove-brick upload replica 4 server5:/bricks/upload/upload</pre>
This tells gluster to drop the brick on server5 from the volume, and
reassures gluster that you know that you will now have a replicated
volume with 4 replicas on it.<br>
<br>
Hope this helps.<br>
<br>
Ted Miller<br>
Elkhart, IN<br>
<br>
P.S. I am not reading the list regularly, so if you need more info,
best to copy me when sending to the list. Just saw your request and
said "I've done that, I can share some information."<br>
</body>
</html>