<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=us-ascii" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 9.00.8112.16457"><LINK rel=stylesheet
href="BLOCKQUOTE{margin-Top: 0px; margin-Bottom: 0px; margin-Left: 2em}"></HEAD>
<BODY style="MARGIN: 10px; FONT-FAMILY: verdana; FONT-SIZE: 10pt">
<DIV><FONT size=2 face=Verdana>
<DIV>Hi,</DIV>
<DIV> </DIV>
<DIV>We have a gluster cluster with 2 nodes as a replicate pair whith GlusterFS verison 3.2.5. </DIV>
<DIV>For the used space of some nodes is up to 95%, we add some nodes into the cluster and issued rebalance command to balance the storage of cluster.</DIV>
<DIV>We issued the command from node A, when rebalancing gone for about 4 days, node A is down and restarted 15 minutes later. From then on, we encounterd some problems:</DIV>
<DIV> </DIV>
<DIV>1. Conflicting entries comes with warning: "gfid differs on subvolume 1 "</DIV>
<DIV>log from mount client: [2012-12-24 11:43:58.455218] E [afr-self-heal-common.c:1333:afr_sh_common_lookup_cbk] 24-jss-r2-replicate-17: Conflicting entries for /6002/music/6/fe46c0eebbf249858e437c37fe412798.mp3</DIV>
<DIV> </DIV>
<DIV>2. background entry self-heal fail </DIV>
<DIV>log: [2012-12-24 11:43:58.460876] E [afr-self-heal-common.c:2074:afr_self_heal_completion_cbk] 24-jss-r2-replicate-17: background entry self-heal failed on /6002/music/6</DIV>
<DIV> </DIV>
<DIV>3. Non Blocking data inodelks fail</DIV>
<DIV>log: [2012-12-24 10:57:07.699049] E [afr-self-heal-data.c:1075:afr_sh_data_post_nonblocking_inodelk_cbk] 24-jss-r2-replicate-34: Non Blocking data inodelks failed for /6002/music/84/89aed70d53e24ef0b80910bd2d71c67b.mp3.</DIV>
<DIV> </DIV>
<DIV>4. Non Blocking entrylks failed </DIV>
<DIV>log: [2012-12-24 11:59:15.119191] E [afr-self-heal-entry.c:2201:afr_sh_post_nonblocking_entry_cbk] 24-jss-r2-replicate-17: Non Blocking entrylks failed for /6002/music/17.</DIV>
<DIV> </DIV>
<DIV>5. No such file or directory</DIV>
<DIV>log: [2012-12-22 05:22:55.514031] E [afr-self-heal-common.c:1054:afr_sh_common_lookup_resp_handler] 24-jss-r2-replicate-16: path /2002/CLUB_COMM_COMMENT_REPLY/35/60e1ff2052ee43a4b447aba77a263406 on subvolume jss-r2-client-33 => -1 (No such file or directory)</DIV>
<DIV></DIV>
<DIV> </DIV>
<DIV>We didn't rebalance for the second time before we make the situation clear. here's some more info:</DIV>
<DIV>Some of files having "gfid differs" can accessed by client and most can not, and we can't make clear which files are bad before we try to access them, and some files cannot self heal.</DIV>
<DIV>who can tell me how to fix it,please!</DIV></FONT></DIV>
<DIV> </DIV>
<DIV><FONT size=2 face=Verdana></FONT> </DIV>
<DIV align=left><FONT color=#c0c0c0 size=2 face=Verdana>2012-12-28
</FONT></DIV><FONT size=2 face=Verdana>
<HR style="WIDTH: 122px; HEIGHT: 2px" align=left SIZE=2>
<DIV><FONT color=#c0c0c0 size=2 face=Verdana><SPAN>cdliuhong</SPAN>
</FONT></DIV></FONT></BODY></HTML>