<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Dan,<br>
<br>
I created two bricks both have ext4 file system.<br>
<br>
The issue seems to be in fs features that you have disabled.<br>
<br>
Formatted the <b>brick1</b> with ext4:<br>
<br>
root@SERVER1 mnt]# dumpe2fs /dev/sda| grep 'Filesystem features'<br>
dumpe2fs 1.41.12 (17-May-2010)<br>
Filesystem features: has_journal ext_attr resize_inode
dir_index filetype needs_recovery extent flex_bg sparse_super
large_file huge_file uninit_bg dir_nlink extra_isize<br>
<br>
Formatted <b>brick 2</b> with ext4:<br>
[root@SERVER2 ~]# dumpe2fs /dev/sda| grep 'Filesystem features'<br>
dumpe2fs 1.41.12 (17-May-2010)<br>
Filesystem features: has_journal ext_attr resize_inode
dir_index filetype extent flex_bg sparse_super large_file<br>
<br>
As you said i have disabled some of the features from <b>brick2</b>.<br>
<br>
I created a distribute volume with these two bricks. created some
files on the mount point and tried setting xattr for these files.<br>
<br>
I got error messages<br>
=======================================================================================<br>
[2011-12-30 01:57:22.551634] I
[client3_1-fops.c:818:client3_1_setxattr_cbk] 1-test-client-1:
remote operation failed: Operation not supported<br>
[2011-12-30 01:57:22.551658] W [fuse-bridge.c:850:fuse_err_cbk]
0-glusterfs-fuse: 201305: SETXATTR() /92 => -1 (Operation not
supported)<br>
[2011-12-30 01:57:22.556490] I
[client3_1-fops.c:818:client3_1_setxattr_cbk] 1-test-client-1:
remote operation failed: Operation not supported<br>
[2011-12-30 01:57:22.556520] W [fuse-bridge.c:850:fuse_err_cbk]
0-glusterfs-fuse: 201311: SETXATTR() /95 => -1 (Operation not
supported)<br>
[2011-12-30 01:57:22.564089] I
[client3_1-fops.c:818:client3_1_setxattr_cbk] 1-test-client-1:
remote operation failed: Operation not supported<br>
[2011-12-30 01:57:22.564114] W [fuse-bridge.c:850:fuse_err_cbk]
0-glusterfs-fuse: 201321: SETXATTR() /100 => -1 (Operation not
supported)<br>
========================================================================================<br>
<br>
where as i created another volume with only <b>brick1</b> and
everything went smoothly. <br>
so i suspect problem is not with rebalance but with ext4 features
that are disabled on <b>brick2</b>.<br>
<br>
Please let me know if i am missing anything that can be tried.<br>
<br>
<br>
<br>
<br>
Thanks,<br>
Shylesh<br>
<br>
<blockquote
cite="mid:756EA019FD2A1648B9F14B0C16B440E8FC601D@mbx024-e1-nj-8.exch024.domain.local"
type="cite">
<div style="direction: ltr; font-family: Tahoma; color: rgb(0, 0,
0); font-size: 10pt;">
<div style="font-family: Times New Roman; color: rgb(0, 0, 0);
font-size: 16px;">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF787186"><font
color="#000000" face="Tahoma" size="2"><b>From:</b>
<a class="moz-txt-link-abbreviated" href="mailto:gluster-users-bounces@gluster.org">gluster-users-bounces@gluster.org</a>
[<a class="moz-txt-link-abbreviated" href="mailto:gluster-users-bounces@gluster.org">gluster-users-bounces@gluster.org</a>] on behalf of Dan
Bretherton [<a class="moz-txt-link-abbreviated" href="mailto:d.a.bretherton@reading.ac.uk">d.a.bretherton@reading.ac.uk</a>]<br>
<b>Sent:</b> Thursday, December 29, 2011 6:05 AM<br>
<b>To:</b> gluster-users<br>
<b>Subject:</b> [Gluster-users] fix-layout stalls with
xattr errors<br>
</font><br>
</div>
<div>Hello All-<br>
I am having problems with rebalance ... fix-layout in
version 3.2.5. I extended a volume with add-brick but the
fix-layout stalls after a small number of layout fixes and
does not make any more progress. I have tried the operation
twice on different servers with the same result. The
following errors are found in the fuse mount log file on the
server carrying out the operation.<br>
<br>
<blockquote>[2011-12-28 21:38:14.840013] I
[afr-common.c:1038:afr_launch_self_heal]
0-nemo2-replicate-4: background data self-heal triggered.
path:
/users/hzu/DATA/ERAINT/ORCA025/2010/snow_ERAINT_2010.nc<br>
[2011-12-28 21:38:15.93079] E
[client3_1-fops.c:1498:client3_1_fxattrop_cbk]
0-nemo2-client-8: remote operation failed: Operation not
supported<br>
[2011-12-28 21:38:15.93141] E
[client3_1-fops.c:1498:client3_1_fxattrop_cbk]
0-nemo2-client-9: remote operation failed: Operation not
supported<br>
[2011-12-28 21:38:15.93385] I
[client3_1-fops.c:1187:client3_1_fstat_cbk]
0-nemo2-client-8: remote operation failed: Operation not
supported<br>
[2011-12-28 21:38:15.93521] I
[client3_1-fops.c:1187:client3_1_fstat_cbk]
0-nemo2-client-9: remote operation failed: Operation not
supported<br>
</blockquote>
<br>
The file in the error message is a link, and it is not
broken as seen from the volume mount point or the bricks.<br>
<br>
There are some worrying error messages in the brick log
files for nemo2-client-8 and nemo2-client-9. Here are some
exerpts from the nemo2-client-8 log, which is similar to the
0-nemo2-client-9 log.<br>
<br>
<blockquote>[2011-12-28 21:23:05.827877] W
[posix.c:3928:do_xattrop] 0-nemo2-posix: Extended
attributes not supported by filesystem<br>
[2011-12-28 21:23:05.827932] I
[server3_1-fops.c:1705:server_fxattrop_cbk]
0-nemo2-server: 8438: FXATTROP 0 (-2111276040) ==> -1
(Operation not support<br>
ed)<br>
[2011-12-28 21:23:05.828848] E [posix.c:4200:posix_fstat]
0-nemo2-posix: fstat failed on fd=0x2aaaac703804:
Operation not supported<br>
[2011-12-28 21:23:05.828879] I
[server3_1-fops.c:1113:server_fstat_cbk] 0-nemo2-server:
8439: FSTAT 0 (-2111276040) ==> -1 (Operation not
supported)<br>
[2011-12-28 21:29:29.871213] W
[socket.c:1494:__socket_proto_state_machine]
0-tcp.nemo2-server: reading from socket failed. Error
(Transport endpoint i<br>
s not connected), peer (192.171.166.81:1003)<br>
[2011-12-28 21:29:29.871305] I
[server-helpers.c:360:do_lock_table_cleanup]
0-nemo2-server: inodelk released on
/users/hzu/DATA/ERAINT/ORCA025/2010/sno<br>
w_ERAINT_2010.nc<br>
[2011-12-28 21:29:29.871345] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on
/users/hzu/DATA/ERAINT/ORCA025/2010/snow_ERAINT_2010.<br>
nc<br>
<br>
[2011-12-28 21:34:36.190023] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on /<br>
[2011-12-28 21:34:36.190055] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on /users<br>
[2011-12-28 21:34:36.190086] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on /users/hzu<br>
[2011-12-28 21:34:36.190102] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on /users/hzu/DATA<br>
[2011-12-28 21:34:36.190135] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on /users/hzu/DATA/ERAINT<br>
[2011-12-28 21:34:36.190154] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on /users/hzu/DATA/ERAINT/ORCA025<br>
[2011-12-28 21:34:36.190171] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd
cleanup on /users/hzu/DATA/ERAINT/ORCA025/2009<br>
<br>
[2011-12-28 21:38:15.92433] I
[server3_1-fops.c:1705:server_fxattrop_cbk]
0-nemo2-server: 12228: FXATTROP 7 (-2111276040) ==> -1
(Operation not supported)<br>
[2011-12-28 21:38:15.92743] E [posix.c:4200:posix_fstat]
0-nemo2-posix: fstat failed on fd=0x2aaaac703804:
Operation not supported<br>
[2011-12-28 21:38:15.92775] I
[server3_1-fops.c:1113:server_fstat_cbk] 0-nemo2-server:
12229: FSTAT 7 (-2111276040) ==> -1 (Operation not
supported)<br>
</blockquote>
<br>
The backend filesystems are ext4 and the are mounted with
options "acl,user_xattr". I tested extended attribute
support (as suggested here:
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://gluster.org/pipermail/gluster-users/2010-December/006257.html"
target="_blank">
http://gluster.org/pipermail/gluster-users/2010-December/006257.html</a>)
and could not find any problems, so I don't understand the
"Extended attributes not supported by filesystem" error.
The only unusual thing about the filesystems is the reduced
number of filesystem features enabled compared to other
bricks. These are the ext4 features enabled.<br>
<br>
has_journal ext_attr resize_inode dir_index filetype
needs_recovery sparse_super large_file<br>
<br>
All the other bricks in the volume have these features plus
extent, flex_bg, huge_file, uninit_bg, dir_nlink and
extra_isize. I don't know if any of these missing ext4
features are part of the problem. Does anybody know what's
going on here?<br>
<br>
Regards<br>
Dan.<br>
<br>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>