<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>After the upgrade, one of my four storage servers was successfully running glusterd, but no glusterfsd processes were present for bricks on that server.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I believe this had to do with the fact that this was the first server I created under 3.1.1, and that I had somehow misconfigured peers for it, since it showed no peers.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I also an entry for this server and one other storage server in /etc/hosts, which I removed.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Once these issues were resolved, restarting the glusterd process worked fine and the bricks were visible.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The client problem was that, after doing the above to the storage servers, the client(s) obviously had cache issues. Doing an “ls” on a native GlusterFS mount showed only several subdirectories out of many, and a “du –sh” of the mount point gave this message on the client:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> du: fts_read failed: No such file or directory<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Clearing the client cache as suggested did not resolve this problem, but restarting the client (I have my own /etc/init.d/gluster-client script, which basically loads and unloads the gluster-fuse module and does the mounts and dismounts) cleared up the problem.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Arial","sans-serif";color:#003D79'>James Burnash<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Arial","sans-serif";color:#003D79'>Unix Engineer<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Arial","sans-serif";color:#003D79'>Knight Capital Group<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Harshavardhana [mailto:harsha@gluster.com] <br><b>Sent:</b> Wednesday, June 22, 2011 3:11 PM<br><b>To:</b> Burnash, James<br><b>Cc:</b> John Mark Walker; gluster-users@gluster.org<br><b>Subject:</b> Re: [Gluster-users] GlusterFS 3.1.5 now available<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>That was to clear page-cache on the client side, usually some of the issues related to cache-coherency with stat-prefetch, quick-read and io-cache get resolved temporarily. <o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>In general this is not a good practice since the application has to invalidate it caches proactively and keep track of them as necessary. <o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>James, i need to know the reason behind doing it. Was it just a hunch or is there any specific errors you are seeing after the upgrade? <o:p></o:p></p><div><p class=MsoNormal>On Wed, Jun 22, 2011 at 12:05 PM, Burnash, James <<a href="mailto:jburnash@knight.com">jburnash@knight.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;color:#1F497D'>I’ll add it </span><span style='font-size:11.0pt;font-family:Wingdings;color:#1F497D'>J</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.0pt;color:#003D79'>James Burnash</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.0pt;color:#003D79'>Unix Engineer</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:8.0pt;color:#003D79'>Knight Capital Group</span><o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;color:#1F497D'> </span><o:p></o:p></p></div><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:10.0pt'>From:</span></b><span style='font-size:10.0pt'> John Mark Walker [mailto:<a href="mailto:jwalker@gluster.com" target="_blank">jwalker@gluster.com</a>] <br><b>Sent:</b> Wednesday, June 22, 2011 3:04 PM<br><b>To:</b> Harshavardhana; Burnash, James</span><o:p></o:p></p><div><p class=MsoNormal><br><b>Cc:</b> <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><o:p></o:p></p></div><p class=MsoNormal><b>Subject:</b> RE: [Gluster-users] GlusterFS 3.1.5 now available<o:p></o:p></p></div></div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><span style='font-size:10.0pt;color:black'>This looks like the type of Q &A which should go on <a href="http://community.gluster.org/" target="_blank">http://community.gluster.org/</a><br><br>:)<br><br>I'll add it if nobody else does.<br><br>-JM</span><o:p></o:p></p><div><div class=MsoNormal align=center style='text-align:center'><span style='color:black'><hr size=2 width="100%" align=center></span></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><b><span style='font-size:10.0pt;color:black'>From:</span></b><span style='font-size:10.0pt;color:black'> <a href="mailto:gluster-users-bounces@gluster.org" target="_blank">gluster-users-bounces@gluster.org</a> [<a href="mailto:gluster-users-bounces@gluster.org" target="_blank">gluster-users-bounces@gluster.org</a>] on behalf of Harshavardhana [<a href="mailto:harsha@gluster.com" target="_blank">harsha@gluster.com</a>]<br><b>Sent:</b> Wednesday, June 22, 2011 11:56 AM<br><b>To:</b> Burnash, James<br><b>Cc:</b> <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br><b>Subject:</b> Re: [Gluster-users] GlusterFS 3.1.5 now available</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'> </span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'>On Wed, Jun 22, 2011 at 11:53 AM, Burnash, James <<a href="mailto:jburnash@knight.com" target="_blank">jburnash@knight.com</a>> wrote:</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><span style='color:black'>Does anyone know how we can get the native GlusterFS clients to clear their cached data short of restarting them (or dismounting and remounting their gluster storage?<br><br>Clients are running 3.1.3, servers are now running 3.1.5.</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'> </span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'>Client side 'caches' are in page cache to invalidate those all you have to do is </span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'> </span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'>echo '3' > /proc/sys/vm/drop_caches </span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'> </span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'>Regards </span><o:p></o:p></p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'>-- </span><o:p></o:p></p><pre><i><span style='color:black'>Harsha</span></i><o:p></o:p></pre><pre><i><span style='color:black'>Gluster Inc</span></i><o:p></o:p></pre><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:black'> </span><o:p></o:p></p></div></div></div></div></div></div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal><span style='font-size:7.5pt;color:navy'>DISCLAIMER:<br>This e-mail, and any attachments thereto, is intended only for use by the addressee(s)named herein and<br>may contain legally privileged and/or confidential information. If you are not the intended recipient of this<br>e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachments</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:7.5pt;color:navy'>thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently<o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:7.5pt;color:navy'>delete the original and any printout thereof.</span><span style='font-size:7.5pt'> <span style='color:navy'>E-mail transmission cannot be guaranteed to be secure or error-free.<o:p></o:p></span></span></p><div><p class=MsoNormal><span style='font-size:7.5pt;color:navy'><br>The sender therefore does not accept liability for any errors or omissions in the contents of this message which<br>arise as a result of e-mail transmission.<br>NOTICE REGARDING PRIVACY AND CONFIDENTIALITY<br>Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications.<o:p></o:p></span></p></div><p><a href="http://www.knight.com/" target="_blank"><span style='font-size:7.5pt;color:navy'>http://www.knight.com</span></a><o:p></o:p></p><p class=MsoNormal> <o:p></o:p></p></div></div><p class=MsoNormal><br><br clear=all><br>-- <o:p></o:p></p><pre><i>Harsha<o:p></o:p></i></pre><pre><i>Gluster Inc<o:p></o:p></i></pre><p class=MsoNormal><o:p> </o:p></p></div></div></body></html>