<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:839464368;
mso-list-type:hybrid;
mso-list-template-ids:1400025842 67698705 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">As a follow up:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I created another replicated striped volume – two brick replica, striped against two sets of servers (four servers in all) – same config as mentioned below. I started pouring data into it, and here’s my output
from `du`:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> peer1 - 47G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> peer2 - 47G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> peer3 - 47G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> peer4 - 24G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Peer1 and Peer2 should be mirrored, and peer3 and peer4 should be mirrored.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">This time, data seems more balanced – EXCEPT that peer4 continues to lag FAR behind – like in the example below.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Any suggestions would be appreciated.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Jason<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> gluster-users-bounces@gluster.org [mailto:gluster-users-bounces@gluster.org]
<b>On Behalf Of </b>Kiebzak, Jason M.<br>
<b>Sent:</b> Thursday, December 04, 2014 10:08 AM<br>
<b>To:</b> gluster-users@gluster.org<br>
<b>Subject:</b> [Gluster-users] replication and blanacing issues<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">It seems that I have two issues:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="font-family:"Courier New","serif""><span style="mso-list:Ignore">1)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-family:"Courier New","serif"">Data is not balanced between all bricks<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="font-family:"Courier New","serif""><span style="mso-list:Ignore">2)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-family:"Courier New","serif"">one replication “pair” is not staying in sync<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">I have four servers/peers, each with one brick, all running 3.6.1. There are two volumes, each running as distributed replicated volume. Below, I’ve included some info. All Daemon are running.
The four peers were all added at the same time.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">Problem 1) for volume1, the peer1/peer2 set have 236G, while peer3 has 3.9T. Shouldn’t it be split more evenly – close to 2T on each set of servers? A similar issue is seen with volume2, but
the total data set (thus the diff) is not as large.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">Problem 2) peer3 and peer4 should be replicated to each other. Peer1 and peer2 have identical disk usage, where as peer3 and peer4 are egregiously out of sync. Data on both peer3 and peer4
continues to grow (I am actively migrating 50T to volume 1).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">`gluster volume info` give this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume Name: volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Type: Distributed-Replicate<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume ID: bf461760-c412-42df-9e1d-7db7f793d344<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Status: Started<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Number of Bricks: 2 x 2 = 4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Transport-type: tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Bricks:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick1: ip1:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick2: ip2:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick3: ip3:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick4: ip4:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Options Reconfigured:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> features.quota: on<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> auth.allow: serverip<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume Name: volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Type: Distributed-Replicate<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume ID: 54a8dbee-387f-4a61-9f67-3e2accb83072<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Status: Started<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Number of Bricks: 2 x 2 = 4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Transport-type: tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Bricks:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick1: ip1:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick2: ip2:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick3: ip3:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick4: ip4:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Options Reconfigured:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> auth.allow: serverip<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">If I do a `# du –h --max-dpeth=1` on each peer, I get this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 236G /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 177G /data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 236G /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 177G /data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer3<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 3.9T /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 179G /data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 524G /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 102G /data/volume2<o:p></o:p></span></p>
</div>
</body>
</html>