<div style="line-height:1.7;color:#000000;font-size:14px;font-family:arial">I test glusterfs 3.3.x, &nbsp;after rebalance the file distribution likes 3.2.X.<div>The lucky is that &nbsp;all files can be access.<br><pre>&gt;lierihanmei&nbsp;&lt;lierihanmei@163.com&gt;&nbsp;wrote:
&gt;
&gt;HI&nbsp;all.
&gt;I&nbsp;have&nbsp;got&nbsp;a&nbsp;problem.&nbsp;after&nbsp;remove-bricks&nbsp;and&nbsp;rebalanced,&nbsp;when&nbsp;i&nbsp;read&nbsp;some&nbsp;file&nbsp;,it&nbsp;gets&nbsp;io&nbsp;errors.
&gt;
&gt;
&gt;version:&nbsp;&nbsp;&nbsp;&nbsp;glusterfs&nbsp;3.2.7
&gt;OS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;&nbsp;&nbsp;&nbsp;Centos6
&gt;
&gt;
&gt;steps:
&gt;1,&nbsp;gluster&nbsp;volume&nbsp;create&nbsp;str&nbsp;stripe&nbsp;2&nbsp;ip:/data1&nbsp;ip:/data2&nbsp;ip:/data3&nbsp;ip:/data4&nbsp;ip:/data5&nbsp;ip:/data6
&gt;2.&nbsp;gluster&nbsp;volume&nbsp;start&nbsp;str
&gt;3.&nbsp;mount&nbsp;-t&nbsp;glusterfs&nbsp;ip:/str&nbsp;/mnt/str
&gt;4.&nbsp;write&nbsp;some&nbsp;file&nbsp;into&nbsp;/mnt/str&nbsp;(&nbsp;then&nbsp;from&nbsp;/mnt/str,&nbsp;&nbsp;all&nbsp;files&nbsp;are&nbsp;OK,&nbsp;can&nbsp;be&nbsp;read&nbsp;and&nbsp;write)
&gt;5.&nbsp;gluster&nbsp;volume&nbsp;remove-brick&nbsp;str&nbsp;ip:/data3&nbsp;ip:/data4(all&nbsp;files&nbsp;are&nbsp;OK&nbsp;too)
&gt;6,&nbsp;gluster&nbsp;rebalance&nbsp;str&nbsp;start
&gt;7,&nbsp;after&nbsp;rebalance&nbsp;complete(some&nbsp;files&nbsp;are&nbsp;OK,&nbsp;some&nbsp;can&nbsp;not&nbsp;access,&nbsp;get&nbsp;IO&nbsp;ERROR)
&gt;and&nbsp;i&nbsp;test&nbsp;glusterfs&nbsp;3.2.5,&nbsp;3.2.6,&nbsp;it&nbsp;get&nbsp;the&nbsp;same&nbsp;error.
&gt;
&gt;
&gt;after&nbsp;rebalance&nbsp;,&nbsp;the&nbsp;data&nbsp;are&nbsp;all&nbsp;in&nbsp;ip:/data5&nbsp;and&nbsp;ip:/data6,&nbsp;&nbsp;&nbsp;but&nbsp;in&nbsp;ip:/data1&nbsp;and&nbsp;ip:/data2only&nbsp;link&nbsp;file&nbsp;like&nbsp;"---------T&nbsp;1&nbsp;root&nbsp;root&nbsp;0&nbsp;Mar&nbsp;26&nbsp;14:07&nbsp;/data/str1/file11"&nbsp;&nbsp;retains.
&gt;
&gt;
&gt;files&nbsp;which&nbsp;can&nbsp;not&nbsp;get&nbsp;all&nbsp;have&nbsp;link&nbsp;files.
&gt;the&nbsp;link&nbsp;file's&nbsp;attr:
&gt;[root@centos6-template&nbsp;dht]#&nbsp;getfattr&nbsp;-m&nbsp;.&nbsp;-e&nbsp;hex&nbsp;-d&nbsp;/data/str1/file11
&gt;getfattr:&nbsp;Removing&nbsp;leading&nbsp;'/'&nbsp;from&nbsp;absolute&nbsp;path&nbsp;names
&gt;#&nbsp;file:&nbsp;data/str1/file11
&gt;trusted.gfid=0xef8c64fa4516424a80bbcc65ad988780
&gt;trusted.glusterfs.dht.linkto=0x7374722d7374726970652d3100
&gt;trusted.str-stripe-0.stripe-count=0x3200
&gt;trusted.str-stripe-0.stripe-index=0x3000
&gt;trusted.str-stripe-0.stripe-size=0x31333130373200
&gt;the&nbsp;data&nbsp;file's&nbsp;attr:
&gt;[root@centos6-template&nbsp;dht]#&nbsp;getfattr&nbsp;-m&nbsp;.&nbsp;-e&nbsp;hex&nbsp;-d&nbsp;/data/str6/file11
&gt;getfattr:&nbsp;Removing&nbsp;leading&nbsp;'/'&nbsp;from&nbsp;absolute&nbsp;path&nbsp;names
&gt;#&nbsp;file:&nbsp;data/str6/file11
&gt;trusted.gfid=0xef8c64fa4516424a80bbcc65ad988780
&gt;trusted.str-stripe-2.stripe-count=0x3200
&gt;trusted.str-stripe-2.stripe-index=0x3100
&gt;trusted.str-stripe-2.stripe-size=0x31333130373200
&gt;
&gt;
&gt;files&nbsp;that&nbsp;can&nbsp;access:
&gt;[root@centos6-template&nbsp;dht]#&nbsp;getfattr&nbsp;-m&nbsp;.&nbsp;-e&nbsp;hex&nbsp;-d&nbsp;/data/str6/file6
&gt;getfattr:&nbsp;Removing&nbsp;leading&nbsp;'/'&nbsp;from&nbsp;absolute&nbsp;path&nbsp;names
&gt;#&nbsp;file:&nbsp;data/str6/file6
&gt;trusted.gfid=0xf1d6a5ec4f054926a65a1114ed3ed619
&gt;trusted.glusterfs.dht.linkto=0x7374722d7374726970652d3000
&gt;trusted.str-stripe-1.stripe-count=0x3200
&gt;trusted.str-stripe-1.stripe-index=0x3100
&gt;trusted.str-stripe-1.stripe-size=0x31333130373200
&gt;but&nbsp;from&nbsp;mountpoint&nbsp;,&nbsp;the&nbsp;attr&nbsp;is:
&gt;[root@centos6-template&nbsp;dht]#&nbsp;getfattr&nbsp;-m&nbsp;.&nbsp;-e&nbsp;hex&nbsp;-d&nbsp;file6
&gt;#&nbsp;file:&nbsp;file6
&gt;trusted.str-stripe-0.stripe-count=0x3200
&gt;trusted.str-stripe-0.stripe-index=0x3000
&gt;trusted.str-stripe-0.stripe-size=0x31333130373200
&gt;
&gt;
&gt;log&nbsp;of&nbsp;glusterfs:
&gt;nsport&nbsp;(str-dht2-client-2)
&gt;[2013-03-26&nbsp;11:56:30.957186]&nbsp;T&nbsp;[rpc-clnt.c:1224:rpc_clnt_record]&nbsp;0-str-dht2-client-3:&nbsp;Auth&nbsp;Info:&nbsp;pid:&nbsp;0,&nbsp;uid:&nbsp;0,&nbsp;gid:&nbsp;0,&nbsp;owner:&nbsp;0
&gt;[2013-03-26&nbsp;11:56:30.957202]&nbsp;T&nbsp;[rpc-clnt.c:1125:rpc_clnt_record_build_header]&nbsp;0-rpc-clnt:&nbsp;Request&nbsp;fraglen&nbsp;344,&nbsp;payload:&nbsp;216,&nbsp;rpc&nbsp;hdr:&nbsp;128
&gt;[2013-03-26&nbsp;11:56:30.957233]&nbsp;T&nbsp;[rpc-clnt.c:1429:rpc_clnt_submit]&nbsp;0-rpc-clnt:&nbsp;submitted&nbsp;request&nbsp;(XID:&nbsp;0x11x&nbsp;Program:&nbsp;GlusterFS&nbsp;3.1,&nbsp;ProgVers:&nbsp;310,&nbsp;Proc:&nbsp;27)&nbsp;to&nbsp;rpc-transport&nbsp;(str-dht2-client-3)
&gt;[2013-03-26&nbsp;11:56:30.957479]&nbsp;T&nbsp;[rpc-clnt.c:638:rpc_clnt_reply_init]&nbsp;0-str-dht2-client-2:&nbsp;received&nbsp;rpc&nbsp;message&nbsp;(RPC&nbsp;XID:&nbsp;0x11x&nbsp;Program:&nbsp;GlusterFS&nbsp;3.1,&nbsp;ProgVers:&nbsp;310,&nbsp;Proc:&nbsp;27)&nbsp;from&nbsp;rpc-transport&nbsp;(str-dht2-client-2)
&gt;[2013-03-26&nbsp;11:56:30.957536]&nbsp;T&nbsp;[rpc-clnt.c:638:rpc_clnt_reply_init]&nbsp;0-str-dht2-client-3:&nbsp;received&nbsp;rpc&nbsp;message&nbsp;(RPC&nbsp;XID:&nbsp;0x11x&nbsp;Program:&nbsp;GlusterFS&nbsp;3.1,&nbsp;ProgVers:&nbsp;310,&nbsp;Proc:&nbsp;27)&nbsp;from&nbsp;rpc-transport&nbsp;(str-dht2-client-3)
&gt;[2013-03-26&nbsp;11:56:30.957563]&nbsp;D&nbsp;[stripe.c:2673:stripe_open_lookup_cbk]&nbsp;0-str-dht2-stripe-1:&nbsp;/file11:&nbsp;stripe&nbsp;info&nbsp;need&nbsp;to&nbsp;be&nbsp;healed
&gt;[2013-03-26&nbsp;11:56:30.957576]&nbsp;E&nbsp;[stripe.c:2691:stripe_open_lookup_cbk]&nbsp;0-str-dht2-stripe-1:&nbsp;stripe&nbsp;size&nbsp;not&nbsp;set
&gt;[2013-03-26&nbsp;11:56:30.957591]&nbsp;D&nbsp;[dht-common.c:2527:dht_fd_cbk]&nbsp;0-str-dht2-dht:&nbsp;subvolume&nbsp;str-dht2-stripe-1&nbsp;returned&nbsp;-1&nbsp;(Input/output&nbsp;error)
&gt;[2013-03-26&nbsp;11:56:30.957618]&nbsp;W&nbsp;[quick-read.c:1640:qr_fstat_helper]&nbsp;0-str-dht2-quick-read:&nbsp;open&nbsp;failed&nbsp;on&nbsp;path&nbsp;(/file11)&nbsp;(Input/output&nbsp;error),&nbsp;unwinding&nbsp;fstat&nbsp;call
&gt;[2013-03-26&nbsp;11:56:30.957643]&nbsp;W&nbsp;[fuse-bridge.c:516:fuse_attr_cbk]&nbsp;0-glusterfs-fuse:&nbsp;7:&nbsp;FSTAT()&nbsp;/file11&nbsp;=&gt;&nbsp;-1&nbsp;(Input/output&nbsp;error)
&gt;[2013-03-26&nbsp;11:56:30.957869]&nbsp;T&nbsp;[fuse-bridge.c:2086:fuse_flush]&nbsp;0-glusterfs-fuse:&nbsp;8:&nbsp;FLUSH&nbsp;0x7f1f9eb69024
&gt;[2013-03-26&nbsp;11:56:30.957956]&nbsp;T&nbsp;[fuse-bridge.c:994:fuse_err_cbk]&nbsp;0-glusterfs-fuse:&nbsp;8:&nbsp;FLUSH()&nbsp;ERR&nbsp;=&gt;&nbsp;0
&gt;[2013-03-26&nbsp;11:56:30.957998]&nbsp;T&nbsp;[fuse-bridge.c:2110:fuse_release]&nbsp;0-glusterfs-fuse:&nbsp;9:&nbsp;RELEASE&nbsp;0x7f1f9eb69024
&gt;[2013-03-26&nbsp;11:56:33.809728]&nbsp;T&nbsp;[rpc-clnt.c:1224:rpc_clnt_record]&nbsp;0-str-dht2-client-0:&nbsp;Auth&nbsp;Info:&nbsp;pid:&nbsp;9105,&nbsp;uid:&nbsp;0,&nbsp;gid:&nbsp;0,&nbsp;owner:&nbsp;9105
&gt;[2013-03-26&nbsp;11:56:33.809759]&nbsp;T&nbsp;[rpc-clnt.c:1125:rpc_clnt_record_build_header]&nbsp;0-rpc-clnt:&nbsp;Request&nbsp;fraglen&nbsp;284,&nbsp;payload:&nbsp;156,&nbsp;rpc&nbsp;hdr:&nbsp;128
&gt;[2013-03-26&nbsp;11:56:33.809794]&nbsp;T&nbsp;[rpc-clnt.c:1429:rpc_clnt_submit]&nbsp;0-rpc-clnt:&nbsp;submitted&nbsp;request&nbsp;(XID:&nbsp;0x12x&nbsp;Program:&nbsp;GlusterFS&nbsp;3.1,&nbsp;ProgVers:&nbsp;310,&nbsp;Proc:&nbsp;27)&nbsp;to&nbsp;rpc-transport&nbsp;(str-dht2-client-0)
&gt;[2013-03-26&nbsp;11:56:33.809824]&nbsp;T&nbsp;[rpc-clnt.c:1224:rpc_clnt_record]&nbsp;0-str-dht2-client-1:&nbsp;Auth&nbsp;Info:&nbsp;pid:&nbsp;9105,&nbsp;uid:&nbsp;0,&nbsp;gid:&nbsp;0,&nbsp;owner:&nbsp;9105
&gt;[2013-03-26&nbsp;11:56:33.809840]&nbsp;T&nbsp;[rpc-clnt.c:1125:rpc_clnt_record_build_header]&nbsp;0-rpc-clnt:&nbsp;Request&nbsp;fraglen&nbsp;284,&nbsp;payload:&nbsp;156,&nbsp;rpc&nbsp;hdr:&nbsp;128
&gt;[2013-03-26&nbsp;11:56:33.809871]&nbsp;T&nbsp;[rpc-clnt.c:1429:rpc_clnt_submit]&nbsp;0-rpc-clnt:&nbsp;submitted&nbsp;request&nbsp;(XID:&nbsp;0x12x&nbsp;Program:&nbsp;GlusterFS&nbsp;3.1,&nbsp;ProgVers:&nbsp;310,&nbsp;Proc:&nbsp;27)&nbsp;to&nbsp;rpc-transport&nbsp;(str-dht2-client-1)
&gt;[2013-03-26&nbsp;11:56:33.809901]&nbsp;T&nbsp;[rpc-clnt.c:1224:rpc_clnt_record]&nbsp;0-str-dht2-client-2:&nbsp;Auth&nbsp;Info:&nbsp;pid:&nbsp;9105,&nbsp;uid:&nbsp;0,&nbsp;gid:&nbsp;0,&nbsp;owner:&nbsp;9105
&gt;
&gt;
&gt;&nbsp;
&gt;
&gt;
&gt;
&gt;------------------------------
&gt;
&gt;_______________________________________________
&gt;Gluster-users&nbsp;mailing&nbsp;list
&gt;Gluster-users@gluster.org
&gt;http://supercolony.gluster.org/mailman/listinfo/gluster-users
&gt;
&gt;End&nbsp;of&nbsp;Gluster-users&nbsp;Digest,&nbsp;Vol&nbsp;59,&nbsp;Issue&nbsp;73
&gt;*********************************************
</pre></div></div><br><br><span title="neteasefooter"><span id="netease_mail_footer"></span></span>