<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FR-BE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hello all,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I’m running gluster 3.4.2-1 on rhel 6.5 (don’t worry, we’ll be moving to RHS soon ;)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I just added a brick to an existing volume and suddenly my files can no longer be accessed by the client.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">This is what I did:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 in]# gluster vol info prd_yearlyextract<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Volume Name: prd_yearlyextract<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Type: Distribute<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Volume ID: d3849bfa-7d1e-43bb-9dcf-af6ff0746c19<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Status: Started<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Number of Bricks: 2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Transport-type: tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Bricks:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Brick1: lpr-nas01:/brick-xiv2/prd_yearlyextract<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 in]# gluster vol brick-add prd_yearlyextract lpr-nas01:/brick-xiv1/prd_yearlyextract<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 in]# gluster vol rebalance prd_yearlyextract<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Now in the two bricks I have the following folders:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 in]# ll /brick-xiv1/prd_yearlyextract/sigedis/in/<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">total 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">---------T 2 444 ftpsiged 129337731 8. Sep 15:40 YCR_20130121_44.xml.DONE<o:p></o:p></span></p>
<p class="MsoNormal">---------T 2 444 ftpsiged 128549601 8. Sep 15:40 YCR_20130121_48.xml<o:p></o:p></p>
<p class="MsoNormal">---------T 2 444 ftpsiged 128166675 8. Sep 15:40 YCR_20130121_50.xml<o:p></o:p></p>
<p class="MsoNormal">---------T 2 444 ftpsiged 128227308 8. Sep 15:40 YCR_20130121_52.xml<o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"><TRUNCATED><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">---------T 2 998 ftpsiged 10544189 8. Sep 15:40 YCR_20140128_98.xml.GZ<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 in]# ll /brick-xiv2/prd_yearlyextract/sigedis/in/<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">total 5648632<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 444 ftpsiged 129337731 8. Mai 2013 YCR_20130121_44.xml.DONE<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 444 ftpsiged 125619849 8. Mai 2013 YCR_20130121_45.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 444 ftpsiged 129315886 8. Mai 2013 YCR_20130121_46.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 444 ftpsiged 125900420 8. Mai 2013 YCR_20130121_47.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 444 ftpsiged 128549601 8. Mai 2013 YCR_20130121_48.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 444 ftpsiged 126278398 8. Mai 2013 YCR_20130121_49.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 444 ftpsiged 128166675 8. Mai 2013 YCR_20130121_50.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><TRUNCATED><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rw-rw-r-- 2 998 ftpsiged 10425421 12. Feb 2014 YCR_20140128_99.xml.GZ<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">On the client I can only see the files that were in the original brick (brick2). The files that were put there afterwards are gone.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">In the rebalance log I lines like this:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">+------------------------------------------------------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.426033] I [rpc-clnt.c:1676:rpc_clnt_reconfig] 0-prd_yearlyextract-client-0: changing port to 49282 (from 0)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.426069] W [socket.c:514:__socket_rwv] 0-prd_yearlyextract-client-0: readv failed (No data available)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.428805] I [rpc-clnt.c:1676:rpc_clnt_reconfig] 0-prd_yearlyextract-client-1: changing port to 49321 (from 0)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.428854] W [socket.c:514:__socket_rwv] 0-prd_yearlyextract-client-1: readv failed (No data available)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.431634] I [client-handshake.c:1659:select_server_supported_programs] 0-prd_yearlyextract-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.431698] I [client-handshake.c:1659:select_server_supported_programs] 0-prd_yearlyextract-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.431895] I [client-handshake.c:1456:client_setvolume_cbk] 0-prd_yearlyextract-client-0: Connected to 192.168.151.21:49282, attached to remote volume '/brick-xiv2/prd_yearlyextract'.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.431908] I [client-handshake.c:1468:client_setvolume_cbk] 0-prd_yearlyextract-client-0: Server and Client lk-version numbers are not same, reopening the fds<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.431991] I [client-handshake.c:1456:client_setvolume_cbk] 0-prd_yearlyextract-client-1: Connected to 192.168.151.21:49321, attached to remote volume '/brick-xiv1/prd_yearlyextract'.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.432001] I [client-handshake.c:1468:client_setvolume_cbk] 0-prd_yearlyextract-client-1: Server and Client lk-version numbers are not same, reopening the fds<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.432104] I [client-handshake.c:450:client_set_lk_version_cbk] 0-prd_yearlyextract-client-0: Server lk version = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.432130] I [client-handshake.c:450:client_set_lk_version_cbk] 0-prd_yearlyextract-client-1: Server lk version = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.437758] I [dht-common.c:2618:dht_setxattr] 0-prd_yearlyextract-dht: fixing the layout of /<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.438260] I [dht-rebalance.c:1085:gf_defrag_migrate_data] 0-prd_yearlyextract-dht: migrate data called on /<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.445939] I [dht-rebalance.c:1296:gf_defrag_migrate_data] 0-prd_yearlyextract-dht: Migration operation on dir / took 0.01 secs<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.448304] I [dht-common.c:2618:dht_setxattr] 0-prd_yearlyextract-dht: fixing the layout of /sigedis<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.448800] I [dht-rebalance.c:1085:gf_defrag_migrate_data] 0-prd_yearlyextract-dht: migrate data called on /sigedis<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.475828] I [dht-rebalance.c:1296:gf_defrag_migrate_data] 0-prd_yearlyextract-dht: Migration operation on dir /sigedis took 0.03 secs<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.478187] I [dht-common.c:2618:dht_setxattr] 0-prd_yearlyextract-dht: fixing the layout of /sigedis/in<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.492970] I [dht-rebalance.c:1085:gf_defrag_migrate_data] 0-prd_yearlyextract-dht: migrate data called on /sigedis/in<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.924592] I [dht-rebalance.c:666:dht_migrate_file] 0-prd_yearlyextract-dht: /sigedis/in/YCR_20130121_44.xml.DONE: attempting to move from prd_yearlyextract-client-0 to prd_yearlyextract-client-1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.925840] W [dht-rebalance.c:382:__dht_check_free_space] 0-prd_yearlyextract-dht: data movement attempted from node (prd_yearlyextract-client-0) with higher disk space to a node (prd_yearlyextract-client-1)
with lesser disk space (/sigedis/in/YCR_20130121_44.xml.DONE)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.928050] I [dht-rebalance.c:666:dht_migrate_file] 0-prd_yearlyextract-dht: /sigedis/in/YCR_20140128_112.xml.GZ: attempting to move from prd_yearlyextract-client-0 to prd_yearlyextract-client-1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-09-08 12:14:41.929562] W [dht-rebalance.c:382:__dht_check_free_space] 0-prd_yearlyextract-dht: data movement attempted from node (prd_yearlyextract-client-0) with higher disk space to a node (prd_yearlyextract-client-1)
with lesser disk space (/sigedis/in/YCR_20140128_112.xml.GZ)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The lines at the bottom concern the files that are invisible. What did I do wrong?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Wannes<o:p></o:p></span></p>
</div>
</body>
</html>