<div>Hello, </div><div><br></div><div>Here are the logs we have just before replication failed :</div><div><br></div><div>[2011-08-02 08:32:33.549303] I [client3_1-fops.c:453:client3_1_readlink_cbk] 0-REP_SVG-client-1: remote operation failed: No such file or directory</div>
<div>[2011-08-02 08:32:33.549334] W [fuse-bridge.c:982:fuse_readlink_cbk] 0-glusterfs-fuse: 2169522: /Sauvegarde/<a href="http://ecrevisse.coe.int/etc/20110720/rc3.d">ecrevisse.coe.int/etc/20110720/rc3.d</a> => -1 (Transport endpoint is not connected)</div>
<div>[2011-08-02 08:32:33.566909] I [client3_1-fops.c:453:client3_1_readlink_cbk] 0-REP_SVG-client-1: remote operation failed: No such file or directory</div><div>[2011-08-02 08:32:33.566937] W [fuse-bridge.c:982:fuse_readlink_cbk] 0-glusterfs-fuse: 2169654: /Sauvegarde/<a href="http://ecrevisse.coe.int/etc/20110720/rmt">ecrevisse.coe.int/etc/20110720/rmt</a> => -1 (Transport endpoint is not connected)</div>
<div>[2011-08-02 08:32:33.598883] I [client3_1-fops.c:453:client3_1_readlink_cbk] 0-REP_SVG-client-1: remote operation failed: No such file or directory</div><div>[2011-08-02 08:32:33.598910] W [fuse-bridge.c:982:fuse_readlink_cbk] 0-glusterfs-fuse: 2169919: /Sauvegarde/<a href="http://ecrevisse.coe.int/etc/20110720/init.d">ecrevisse.coe.int/etc/20110720/init.d</a> => -1 (Transport endpoint is not connected)</div>
<div>[2011-08-02 08:33:40.925756] I [client3_1-fops.c:2132:client3_1_opendir_cbk] 0-REP_SVG-client-1: remote operation failed: No such file or directory</div><div>[2011-08-02 08:33:40.925809] W [fuse-bridge.c:582:fuse_fd_cbk] 0-glusterfs-fuse: 2419699: OPENDIR() /Sauvegarde/<a href="http://taal.coe.int/usrshare/20110720/vim/vim72/autoload">taal.coe.int/usrshare/20110720/vim/vim72/autoload</a> => -1 (No such file or directory)</div>
<div>[2011-08-02 08:33:41.458567] I [fuse-bridge.c:3218:fuse_thread_proc] 0-fuse: unmounting /etc/glusterd/mount/REP_SVG</div><div>[2011-08-02 08:33:41.473995] W [glusterfsd.c:712:cleanup_and_exit] (-->/lib/libc.so.6(clone+0x6d) [0x7fed6a91d02d] (-->/lib/libpthread.so.0(+0x68ba) [0x7fed6abb58ba] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x40536d]))) 0-: received signum (15), shutting down</div>
<div><br></div><div><div># gluster volume rebalance REP_SVG status</div><div>rebalance failed</div></div><div><br></div><div>Do you have some solutions ?</div><div><br></div><div>Thank you in advance !</div><div><br></div>
<b style="color:rgb(0, 0, 153)">Julien Groselle</b><font class="Apple-style-span" face="Arial"><br></font>
<br><br><div class="gmail_quote">2011/8/1 Julien Groselle <span dir="ltr"><<a href="mailto:julien.groselle@gmail.com">julien.groselle@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
First of all, we have milions of this lines :<div><div>[2011-08-01 10:30:30.155065] W [inode.c:1035:inode_path] 0-/storage/backup/inode: no dentry for non-root inode 392601459: ef33731f-7c5c-435e-b300-2ffe2ab1c155</div><div>
[2011-08-01 10:30:30.155767] W [inode.c:1035:inode_path] 0-/storage/backup/inode: no dentry for non-root inode 392601459: ef33731f-7c5c-435e-b300-2ffe2ab1c155</div><div>[2011-08-01 10:30:30.156144] W [inode.c:1035:inode_path] 0-/storage/backup/inode: no dentry for non-root inode 392601459: ef33731f-7c5c-435e-b300-2ffe2ab1c155</div>
<div>[2011-08-01 10:30:30.156641] W [inode.c:1035:inode_path] 0-/storage/backup/inode: no dentry for non-root inode 392601459: ef33731f-7c5c-435e-b300-2ffe2ab1c155</div><div>[2011-08-01 10:30:30.159850] W [inode.c:1035:inode_path] 0-/storage/backup/inode: no dentry for non-root inode 392601459: ef33731f-7c5c-435e-b300-2ffe2ab1c155</div>
<div>[2011-08-01 10:30:30.160271] W [inode.c:1035:inode_path] 0-/storage/backup/inode: no dentry for non-root inode 392601459: ef33731f-7c5c-435e-b300-2ffe2ab1c155</div><div><br></div><div>What does it means ?</div><div>
<br>
</div><div>And we have many errors "No such file or directory" like this one :</div><div>[2011-08-01 10:37:23.342608] E [posix.c:1085:posix_readlink] 0-REP_SVG-posix: readlink on /Sauvegarde/<a href="http://vulcano.coe.int/usrshare/20110729/terminfo/v/vt102" target="_blank">vulcano.coe.int/usrshare/20110729/terminfo/v/vt102</a> failed: No such file or directory</div>
<div><br></div><div>I hope i've helped.</div><div><br></div><font color="#888888"><b style="color:rgb(0, 0, 153)">Julien Groselle</b></font><div><div></div><div class="h5"><br><font face="Arial"></font>
<br><br><div class="gmail_quote">2011/8/1 Julien Groselle <span dir="ltr"><<a href="mailto:julien.groselle@gmail.com" target="_blank">julien.groselle@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>I'm sorry, someone of my team has flushed the logs, and restart rebalance... </div><div>I will give you logs as soon as possible, when the problem will come again.</div><div><br></div><div>You want me to create a <a href="http://www.gluster.com/community/documentation/index.php/Gluster_3.2:_Creating_Distributed_Replicated_Volumes" target="_blank">distributed / replicated</a> volume, right ?</div>
<div>I had used this command to create my volume : </div><div># gluster volume create REP_SVG replica 2 transport tcp server1:/storage/backup server2:/storage/backup</div><div><br></div><div>What is the command to create distributed / replicated volume ?</div>
<div><br></div><div>Thank you very much for your quick answer !</div><br clear="all"><font color="#888888"><b style="color:rgb(0, 0, 153)">Julien Groselle</b></font><div><div></div><div><br><font face="Arial"></font>
<br><div class="gmail_quote">2011/8/1 Anand Avati <span dir="ltr"><<a href="mailto:anand.avati@gmail.com" target="_blank">anand.avati@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Can you please provide logs for the errors you are facing? Also, rebalance was not the right operation to be done in your situation. You don't seem to have a distributed setup (but a pure replicate instead) in which case rebalance is really not achieving anything for you.<div>
<br></div><div>Avati<br><br><div class="gmail_quote"><div><div></div><div>On Mon, Aug 1, 2011 at 12:30 PM, Julien Groselle <span dir="ltr"><<a href="mailto:julien.groselle@gmail.com" target="_blank">julien.groselle@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div></div><div>
Hello,<div><br></div><div>I have installed GlusterFS one month ago, and replication have many issues :</div><div>First of all, our infrastructure, 2 storage array of 8Tb in replication mode... We have our backups file on this arrays, so 6Tb of datas.</div>
<div><br></div><div>I want replicate datas on the second storrage array, so, i use this command :</div><div># gluster volume rebalance REP_SVG migrate-data start</div><div>And gluster start to replicate, in 2 weeks we had 2.6Yb of datas replicated.</div>
<div>But now, replication fail after about one day of replication.... with many errors.</div><div><br></div><div>So i have two questions, are there any option or command to boost replication ?</div><div>We have to continue to backup our servers... so during the réplication, many files are rotated/moved/added. </div>
<div>Is it a problem for gluster to replicatde datas during a backup session ?</div><div><br></div><div>For now, we can't replicate any datas more ! We need help.</div><div><br></div><div>FYI :</div><div><div># gluster --version</div>
<div>glusterfs 3.2.1 built on Jun 12 2011 12:29:36</div><div>Repository revision: v3.2.1</div><div>Copyright (c) 2006-2010 Gluster Inc. <<a href="http://www.gluster.com" target="_blank">http://www.gluster.com</a>></div>
<div>GlusterFS comes with ABSOLUTELY NO WARRANTY.</div>
<div>You may redistribute copies of GlusterFS under the terms of the GNU Affero General Public License.</div></div><div><br></div><div><div># uname -a</div><div>Linux toomba 2.6.32-5-amd64 #1 SMP Wed Jan 12 03:40:32 UTC 2011 x86_64 GNU/Linux</div>
</div><div><br></div><div><div># cat /etc/debian_version </div><div>6.0.2</div></div><div><br></div><div><div># gluster peer status</div><div>Number of Peers: 1</div><div><br></div><div>Hostname: <a href="http://kaiserstuhl-svg.coe.int" target="_blank">kaiserstuhl-svg.coe.int</a></div>
<div>Uuid: 5b79b4bc-c8d2-48d4-bd43-37991197ab47</div><div>State: Peer in Cluster (Connected)</div></div><div><br></div><div><div># gluster volume info all</div><div><br></div><div>Volume Name: REP_SVG</div><div>Type: Replicate</div>
<div>Status: Started</div><div>Number of Bricks: 2</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: toomba-svg.coe.int:/storage/backup</div><div>Brick2: kaiserstuhl-svg.coe.int:/storage/backup</div><div>
Options Reconfigured:</div>
<div>performance.write-behind-window-size: 16MB</div><div>performance.cache-size: 256MB</div><div>diagnostics.brick-log-level: WARNING</div></div><div><br clear="all"><b style="color:rgb(0, 0, 153)">Julien Groselle</b><font face="Arial"><br>
</font>
</div>
<br></div></div>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br>
</div></div></blockquote></div><br></div></div></div>
</blockquote></div><br>