<div dir="ltr"><span id="docs-internal-guid-4cff586d-0b5f-d324-f772-22cce4c0e9df"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">I have read architecture documents about GlusterFS, Ceph and Swift. Mostly I can’t understand and worry about GlusterFS is rebalacing. From my understanding, It looks like that GlusterFS move too much data when it do ‘rebalance layout and migrate’, especially compare with solution of Ceph and Swift. I’m not sure that i understand it correctly. </span></p>
<br><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">It look like Ceph and Swift map file to disk with similar way. They map file to disk like this. </span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">filename->hashfile name-> map to PG(placement group at ceph) or partition(at swif)-> PG or partition to disk.</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><br></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">And if i did not misunderstand, GlusterFs do this way. </span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">filename->hashfile name->map to disk</span></p>
<br><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Existence of PG(or parition) makes only minimul data migration from old disks to new disks. That way they flatten storage cluster with minimul data migration. Theoretically, worst case scenario, ceph and swift moves amount of data which is equal to newly added disk size(at this point, I assumed that there only exist homogeneous files)</span></p>
<br><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">But GlusterFs, from my understand, just add new disk to tail of hash ring. And I found a article which is wrote by jeff darcy. He also said that there really exist this issue. </span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><a href="http://hekafs.org/index.php/author/Jeff%20Darcy/" style="text-decoration:none"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">http://he</span></a><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><a href="http://kafs.org/index.php/2012/03/glusterfs-algorithms-distribution/">kafs.org/index.php/2012/03/glusterfs-algorithms-distribution/</a></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">If storage cluster get larger and larger 50% data replacement will be more and more horrible issue. But could not find worries about this issue. How do they works in real field? I’m worrying about that there will be too much data traffic while ‘rebalancing and migration’. Maybe just rebalance the layout and forget about migration can be solution. In this way, exchange of little latency maybe relieve migration traffic and time. But this looks a patchwork solution. Moreover,without migration and flattening, Write requests will converge upon newly added disk.</span></p>
<br><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Am i understand correctly about glusterFs? Above articles about glusterFs is quite old and I couldn’t find any docuemnts about architecture except it. If my understanding is correct, How glusterFs user solve those kind of issue(huge migration data traffic, converge of write request)? Are there some design decision or target usecase?</span></p>
<br><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Thanks in advances for any helps. </span></p>
<div><span style="font-size:15px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><br></span></div></span></div>