<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body text="#000000" bgcolor="#ffffff">
Hello All-<br>
I am having problems with rebalance ... fix-layout in version 3.2.5.
I extended a volume with add-brick but the fix-layout stalls after
a small number of layout fixes and does not make any more progress.
I have tried the operation twice on different servers with the same
result. The following errors are found in the fuse mount log file
on the server carrying out the operation.<br>
<br>
<blockquote>[2011-12-28 21:38:14.840013] I
[afr-common.c:1038:afr_launch_self_heal] 0-nemo2-replicate-4:
background data self-heal triggered. path:
/users/hzu/DATA/ERAINT/ORCA025/2010/snow_ERAINT_2010.nc<br>
[2011-12-28 21:38:15.93079] E
[client3_1-fops.c:1498:client3_1_fxattrop_cbk] 0-nemo2-client-8:
remote operation failed: Operation not supported<br>
[2011-12-28 21:38:15.93141] E
[client3_1-fops.c:1498:client3_1_fxattrop_cbk] 0-nemo2-client-9:
remote operation failed: Operation not supported<br>
[2011-12-28 21:38:15.93385] I
[client3_1-fops.c:1187:client3_1_fstat_cbk] 0-nemo2-client-8:
remote operation failed: Operation not supported<br>
[2011-12-28 21:38:15.93521] I
[client3_1-fops.c:1187:client3_1_fstat_cbk] 0-nemo2-client-9:
remote operation failed: Operation not supported<br>
</blockquote>
<br>
The file in the error message is a link, and it is not broken as
seen from the volume mount point or the bricks.<br>
<br>
There are some worrying error messages in the brick log files for
nemo2-client-8 and nemo2-client-9. Here are some exerpts from the
nemo2-client-8 log, which is similar to the 0-nemo2-client-9 log.<br>
<br>
<blockquote>[2011-12-28 21:23:05.827877] W [posix.c:3928:do_xattrop]
0-nemo2-posix: Extended attributes not supported by filesystem<br>
[2011-12-28 21:23:05.827932] I
[server3_1-fops.c:1705:server_fxattrop_cbk] 0-nemo2-server: 8438:
FXATTROP 0 (-2111276040) ==> -1 (Operation not support<br>
ed)<br>
[2011-12-28 21:23:05.828848] E [posix.c:4200:posix_fstat]
0-nemo2-posix: fstat failed on fd=0x2aaaac703804: Operation not
supported<br>
[2011-12-28 21:23:05.828879] I
[server3_1-fops.c:1113:server_fstat_cbk] 0-nemo2-server: 8439:
FSTAT 0 (-2111276040) ==> -1 (Operation not supported)<br>
[2011-12-28 21:29:29.871213] W
[socket.c:1494:__socket_proto_state_machine] 0-tcp.nemo2-server:
reading from socket failed. Error (Transport endpoint i<br>
s not connected), peer (192.171.166.81:1003)<br>
[2011-12-28 21:29:29.871305] I
[server-helpers.c:360:do_lock_table_cleanup] 0-nemo2-server:
inodelk released on /users/hzu/DATA/ERAINT/ORCA025/2010/sno<br>
w_ERAINT_2010.nc<br>
[2011-12-28 21:29:29.871345] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/users/hzu/DATA/ERAINT/ORCA025/2010/snow_ERAINT_2010.<br>
nc<br>
<br>
[2011-12-28 21:34:36.190023] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/<br>
[2011-12-28 21:34:36.190055] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/users<br>
[2011-12-28 21:34:36.190086] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/users/hzu<br>
[2011-12-28 21:34:36.190102] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/users/hzu/DATA<br>
[2011-12-28 21:34:36.190135] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/users/hzu/DATA/ERAINT<br>
[2011-12-28 21:34:36.190154] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/users/hzu/DATA/ERAINT/ORCA025<br>
[2011-12-28 21:34:36.190171] I
[server-helpers.c:485:do_fd_cleanup] 0-nemo2-server: fd cleanup on
/users/hzu/DATA/ERAINT/ORCA025/2009<br>
<br>
[2011-12-28 21:38:15.92433] I
[server3_1-fops.c:1705:server_fxattrop_cbk] 0-nemo2-server: 12228:
FXATTROP 7 (-2111276040) ==> -1 (Operation not supported)<br>
[2011-12-28 21:38:15.92743] E [posix.c:4200:posix_fstat]
0-nemo2-posix: fstat failed on fd=0x2aaaac703804: Operation not
supported<br>
[2011-12-28 21:38:15.92775] I
[server3_1-fops.c:1113:server_fstat_cbk] 0-nemo2-server: 12229:
FSTAT 7 (-2111276040) ==> -1 (Operation not supported)<br>
</blockquote>
<br>
The backend filesystems are ext4 and the are mounted with options
"acl,user_xattr". I tested extended attribute support (as suggested
here:
<a class="moz-txt-link-freetext" href="http://gluster.org/pipermail/gluster-users/2010-December/006257.html">http://gluster.org/pipermail/gluster-users/2010-December/006257.html</a>)
and could not find any problems, so I don't understand the "Extended
attributes not supported by filesystem" error. The only unusual
thing about the filesystems is the reduced number of filesystem
features enabled compared to other bricks. These are the ext4
features enabled.<br>
<br>
has_journal ext_attr resize_inode dir_index filetype needs_recovery
sparse_super large_file<br>
<br>
All the other bricks in the volume have these features plus extent,
flex_bg, huge_file, uninit_bg, dir_nlink and extra_isize. I don't
know if any of these missing ext4 features are part of the problem.
Does anybody know what's going on here?<br>
<br>
Regards<br>
Dan.<br>
<br>
<br>
<pre class="moz-signature" cols="72"></pre>
</body>
</html>