<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:839464368;
mso-list-type:hybrid;
mso-list-template-ids:1400025842 67698705 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">It seems that I have two issues:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:"Courier New","serif""><span style="mso-list:Ignore">1)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-family:"Courier New","serif"">Data is not balanced between all bricks<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:"Courier New","serif""><span style="mso-list:Ignore">2)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-family:"Courier New","serif"">one replication “pair” is not staying in sync<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">I have four servers/peers, each with one brick, all running 3.6.1. There are two volumes, each running as distributed replicated volume. Below, I’ve included some info. All Daemon are running.
The four peers were all added at the same time.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">Problem 1) for volume1, the peer1/peer2 set have 236G, while peer3 has 3.9T. Shouldn’t it be split more evenly – close to 2T on each set of servers? A similar issue is seen with volume2, but
the total data set (thus the diff) is not as large.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">Problem 2) peer3 and peer4 should be replicated to each other. Peer1 and peer2 have identical disk usage, where as peer3 and peer4 are egregiously out of sync. Data on both peer3 and peer4
continues to grow (I am actively migrating 50T to volume 1).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">`gluster volume info` give this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume Name: volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Type: Distributed-Replicate<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume ID: bf461760-c412-42df-9e1d-7db7f793d344<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Status: Started<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Number of Bricks: 2 x 2 = 4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Transport-type: tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Bricks:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick1: ip1:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick2: ip2:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick3: ip3:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick4: ip4:/data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Options Reconfigured:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> features.quota: on<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> auth.allow: serverip<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume Name: volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Type: Distributed-Replicate<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Volume ID: 54a8dbee-387f-4a61-9f67-3e2accb83072<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Status: Started<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Number of Bricks: 2 x 2 = 4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Transport-type: tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Bricks:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick1: ip1:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick2: ip2:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick3: ip3:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Brick4: ip4:/data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Options Reconfigured:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> auth.allow: serverip<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif"">If I do a `# du –h --max-dpeth=1` on each peer, I get this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 236G /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 177G /data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 236G /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 177G /data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer3<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 3.9T /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 179G /data/volume2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> Peer4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 524G /data/volume1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New","serif""> 102G /data/volume2<o:p></o:p></span></p>
</div>
</body>
</html>