<div dir="ltr">Khoi,<div>Looking at your logs, my guess is that the client was mounted with a very high --entry-timeout=N value (or client-2 accesses file, runs strace etc. all within the default 1sec after client-1 recreated the file through vi). If not, I don't see how your client could get this log entry:</div>
<div><br></div><div><span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;line-height:14.390625px;background-color:rgb(240,240,240)">[2013-12-19 20:08:51.729392] W [fuse-bridge.c:705:fuse_attr_cbk] 0-glusterfs-fuse: 42: STAT() /world => -1 (Stale file handle)</span><br>
</div><div class="gmail_extra"><br>If a file is deleted/recreated within the entry-timeout period from another client, this can happen, and that is a gluster independent FUSE behavior.</div><div class="gmail_extra"><br></div>
<div class="gmail_extra">Avati</div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 19, 2013 at 12:48 PM, Khoi Mai <span dir="ltr"><<a href="mailto:KHOIMAI@up.com" target="_blank">KHOIMAI@up.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><tt><font>Gluster community,</font></tt><br><br><tt><font> </font></tt><a href="https://bugzilla.redhat.com/show_bug.cgi?id=1041109" target="_blank"><tt><font>https://bugzilla.redhat.com/show_bug.cgi?id=1041109</font></tt></a><br>
<br><tt><font>I've updated the above bugzilla while testing the
latest gluster package glusterfs-3.4.2qa4 with reproducible results. I
am unsure if there is any feature options that can remedy this behavior.</font></tt><br><br><font face="sans-serif">Khoi<br><br><br></font><br><br><br><br><font size="1" color="#5f5f5f" face="sans-serif">From:
</font><font size="1" face="sans-serif"><a href="mailto:gluster-users-request@gluster.org" target="_blank">gluster-users-request@gluster.org</a></font><br><font size="1" color="#5f5f5f" face="sans-serif">To:
</font><font size="1" face="sans-serif"><a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a></font><br><font size="1" color="#5f5f5f" face="sans-serif">Date:
</font><font size="1" face="sans-serif">12/19/2013 06:00 AM</font><br><font size="1" color="#5f5f5f" face="sans-serif">Subject:
</font><font size="1" face="sans-serif">Gluster-users
Digest, Vol 68, Issue 20</font><br><font size="1" color="#5f5f5f" face="sans-serif">Sent by:
</font><font size="1" face="sans-serif"><a href="mailto:gluster-users-bounces@gluster.org" target="_blank">gluster-users-bounces@gluster.org</a></font><br><hr noshade><br><br><br><tt><font>Send Gluster-users mailing list submissions to<br>
<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br><br>To subscribe or unsubscribe via the World Wide Web, visit<br>
</font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank"><tt><font>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font><br>or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:gluster-users-request@gluster.org" target="_blank">gluster-users-request@gluster.org</a><br><br>You can reach the person managing the list at<br>
<a href="mailto:gluster-users-owner@gluster.org" target="_blank">gluster-users-owner@gluster.org</a><br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Gluster-users digest..."<br>
<br><br>Today's Topics:<br><br> 1. Re: help with replace-brick migrate (Mariusz Sobisiak)<br> 2. Re: help with replace-brick migrate (Raphael Rabelo)<br> 3. Re: help with replace-brick migrate (Mariusz Sobisiak)<br>
4. Problem adding brick (replica) (Sa?a Friedrich)<br> 5. Shared storage for critical infrastructure (Pieter Baele)<br> 6. Debugging gfapi (Kelly Burkhart)<br> 7. Re: [Gluster-devel] glusterfs-3.4.2qa4 released (Kaleb Keithley)<br>
8. Cancelled: Gluster Community Weekly Meeting (Vijay Bellur)<br> 9. Cancelled: Gluster Community Weekly Meeting (Vijay Bellur)<br> 10. Gluster Community Weekly Meeting Minutes --
2013-12-18<br> (Vijay Bellur)<br> 11. Re: glusterfs-3.4.2qa4 released (Vijay Bellur)<br> 12. Re: Debugging gfapi (Jeff Darcy)<br> 13. Trying to start glusterd (Knut Moe)<br> 14. gfapi from non-root (Kelly Burkhart)<br>
15. Passing noforget option to glusterfs native client
mounts<br> (Chalcogen)<br> 16. Re: Passing noforget option to glusterfs native
client mounts<br> (Chalcogen)<br> 17. Re: Problem adding brick (replica) (Anirban Ghoshal)<br> 18. Re: Problem adding brick (replica) (Sa?a Friedrich)<br> 19. Re: gfapi from non-root (Kelly Burkhart)<br>
20. failed to create volume ends with a prefix of it is
already<br> part of a volume (William Kwan)<br> 21. Re: Trying to start glusterd (Kaushal M)<br> 22. Re: qemu remote insecure connections (Vijay Bellur)<br> 23. Re: failed to create volume ends with a prefix of
it is<br> already part of a volume (Bernhard Glomm)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Wed, 18 Dec 2013 13:21:26 +0100<br>From: "Mariusz Sobisiak" <<a href="mailto:MSobisiak@ydp.pl" target="_blank">MSobisiak@ydp.pl</a>><br>
To: <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br>Subject: Re: [Gluster-users] help with replace-brick migrate<br>Message-ID:<br>
<507D8C234E515F4F969362F9666D7EBBED1CB7@nagato1.intranet.ydp><br>Content-Type: text/plain;
charset="us-ascii"<br><br>> I don't knew that can be a lot of trash (orphan) files in .glusterfs,<br>so here what i do:<br><br>I think you can easily check by this command (on old gluster server):<br>
find .glusterfs/ -type f -links 1<br>If something returns that means file have only one link and doesn't have<br>a "real" file on the brick so it unintended (and it's orphan
file).<br><br>> # du -hs *<br>> 3.5G documents<br>> 341G home<br>> 58G archives<br>> 808G secure_folder<br>> 93G secure_folder2<br><br>So you have files on new gluster. I understood that you have just<br>
.glusterfs directory...<br><br>> 1.3T .glusterfs/<br><br>It looks OK. It's not taking any space because it's hardlinks.<br><br>> So, i have 1.3Tb in gluster!! So, i think that replace-brick worked<br>correctly ... right ?<br>
<br>Probably yes.<br><br>> So, how can i restart the replace-brick command again ?<br><br>I am not sure what for you want to restart the replace-brick command?<br>You wrote that status show: migration complete... So it's ok and do just<br>
commit (first ensure is everything is OK).<br><br>If you're not sure if all files had copied maybe you can compare files<br>on both nodes (old one and migrated) like this:<br>find /where/the/brick/is/ -path "*/.glusterfs/*" -prune -o -name
'*'<br>-print|wc -l<br>If the command return the same values that mean you have all files :D<br><br>But In my opinion everything looks okay (except that why so many files<br>are orphaned on old glusterfs).<br><br>
--<br>Mariusz<br><br><br><br>------------------------------<br><br>Message: 2<br>Date: Wed, 18 Dec 2013 10:59:12 -0200<br>From: Raphael Rabelo <<a href="mailto:rabeloo@gmail.com" target="_blank">rabeloo@gmail.com</a>><br>
To: Mariusz Sobisiak <<a href="mailto:MSobisiak@ydp.pl" target="_blank">MSobisiak@ydp.pl</a>><br>Cc: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>Subject: Re: [Gluster-users] help with replace-brick migrate<br>
Message-ID:<br>
<<a href="mailto:CAMOH6nDr8ZCRjdybDfE51V_%2B4UDCkrTx9xaSfJM5JhXM8fvoqw@mail.gmail.com" target="_blank">CAMOH6nDr8ZCRjdybDfE51V_+4UDCkrTx9xaSfJM5JhXM8fvoqw@mail.gmail.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br>
<br>> I think you can easily check by this command (on old gluster server):<br>find .glusterfs/ -type f -links 1<br>If something returns that means file have only one link and doesn't have<br>a "real" file on the brick so it unintended (and it's orphan
file).<br><br>The result of # find .glusterfs/ -type f -links 1 is empty ...<br><br>> I am not sure what for you want to restart the replace-brick command?<br>You wrote that status show: migration complete... So it's ok and do just<br>
commit (first ensure is everything is OK).<br><br>Before commit the replace, i erased all files on the new node, thinking<br>that's not ok... :(<br>I thinked that add these 2 new bricks in the same volume with replica 4,<br>
and use self-heal to replicate all data... what you think ?<br><br>Tks!<br><br>2013/12/18 Mariusz Sobisiak <<a href="mailto:MSobisiak@ydp.pl" target="_blank">MSobisiak@ydp.pl</a>><br><br>> > I don't knew that can be a lot of trash (orphan) files in .glusterfs,<br>
> so here what i do:<br>><br>> I think you can easily check by this command (on old gluster server):<br>> find .glusterfs/ -type f -links 1<br>> If something returns that means file have only one link and doesn't
have<br>> a "real" file on the brick so it unintended (and it's orphan
file).<br>><br>> > # du -hs *<br>> > 3.5G documents<br>> > 341G home<br>> > 58G archives<br>> > 808G secure_folder<br>> > 93G secure_folder2<br>><br>> So you have files on new gluster. I understood that you have just<br>
> .glusterfs directory...<br>><br>> > 1.3T .glusterfs/<br>><br>> It looks OK. It's not taking any space because it's hardlinks.<br>><br>> > So, i have 1.3Tb in gluster!! So, i think that replace-brick
worked<br>> correctly ... right ?<br>><br>> Probably yes.<br>><br>> > So, how can i restart the replace-brick command again ?<br>><br>> I am not sure what for you want to restart the replace-brick command?<br>
> You wrote that status show: migration complete... So it's ok and do
just<br>> commit (first ensure is everything is OK).<br>><br>> If you're not sure if all files had copied maybe you can compare files<br>> on both nodes (old one and migrated) like this:<br>> find /where/the/brick/is/ -path "*/.glusterfs/*" -prune
-o -name '*'<br>> -print|wc -l<br>> If the command return the same values that mean you have all files
:D<br>><br>> But In my opinion everything looks okay (except that why so many files<br>> are orphaned on old glusterfs).<br>><br>> --<br>> Mariusz<br>><br>> _______________________________________________<br>
> Gluster-users mailing list<br>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank"><tt><font>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font><br>
><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/22c98a96/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/22c98a96/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 3<br>Date: Wed, 18 Dec 2013 14:28:41 +0100<br>From: "Mariusz Sobisiak" <<a href="mailto:MSobisiak@ydp.pl" target="_blank">MSobisiak@ydp.pl</a>><br>To: <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br>
Subject: Re: [Gluster-users] help with replace-brick migrate<br>Message-ID:<br>
<507D8C234E515F4F969362F9666D7EBBED1D5A@nagato1.intranet.ydp><br>Content-Type: text/plain;
charset="iso-8859-2"<br><br>> The result of ?# find .glusterfs/ -type f -links 1 is empty ...?<br><br>You run it on old gluster (where is 2TB)? It may take a long time. So in
fact it very strange. <br>You can use that other find command to compare amount of data.<br><br>> I thinked that add these 2 new bricks in the same volume with replica
4, and use self-heal to replicate all data... what you think ?<br><br>You can abort replace-brick. And do it again. I thought you want migrate
data to another server, now you want expand volume? Of course if you want
just only expand you can use add-brick command.<br><br>--<br>Mariusz<br><br><br>------------------------------<br><br>Message: 4<br>Date: Wed, 18 Dec 2013 15:38:39 +0100<br>From: Sa?a Friedrich <<a href="mailto:sasa.friedrich@bitlab.si" target="_blank">sasa.friedrich@bitlab.si</a>><br>
To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>Subject: [Gluster-users] Problem adding brick (replica)<br>Message-ID: <<a href="mailto:52B1B36F.1020704@bitlab.si" target="_blank">52B1B36F.1020704@bitlab.si</a>><br>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"<br><br>Hi!<br><br>I have some trouble adding a brick to existing gluster volume.<br><br>When I try to (in CLI):<br><br> gluster> volume add-brick data_domain replica 3<br>
gluster2.data:/glusterfs/data_domain<br><br>I get:<br><br> volume add-brick: failed:<br><br>I probed the peer successfully, peer status returns:<br><br> Hostname: gluster3.data<br> Uuid: e694f552-636a-4cf3-a04f-997ec87a880c<br>
State: Peer in Cluster (Connected)<br><br> Hostname: gluster2.data<br> Port: 24007<br> Uuid: 36922d4c-55f2-4cc6-85b9-a9541e5619a2<br> State: Peer in Cluster (Connected)<br><br>Existing volume info:<br><br>
Volume Name: data_domain<br> Type: Replicate<br> Volume ID: ae096e7d-cf0c-46ed-863a-9ecc3e8ce288<br> Status: Started<br> Number of Bricks: 1 x 2 = 2<br> Transport-type: tcp<br> Bricks:<br> Brick1: gluster1.data:/glusterfs/data_domain<br>
Brick2: gluster3.data:/glusterfs/data_domain<br> Options Reconfigured:<br> storage.owner-gid: 36<br> storage.owner-uid: 36<br> server.allow-insecure: on<br> network.remote-dio: enable<br> cluster.eager-lock: enable<br>
performance.stat-prefetch: off<br> performance.io-cache: off<br> performance.read-ahead: off<br> performance.quick-read: off<br><br><br>Only thing I found in log is:<br><br> (/var/log/glusterfs/cli.log)<br>
[2013-12-18 12:09:17.281310] W [cli-rl.c:106:cli_rl_process_line]<br> 0-glusterfs: failed to process line<br> [2013-12-18 12:10:07.650267] I<br> [cli-rpc-ops.c:332:gf_cli_list_friends_cbk] 0-cli: Received
resp to<br> list: 0<br><br> (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log)<br> [2013-12-18 12:12:38.887911] I<br> [glusterd-brick-ops.c:370:__glusterd_handle_add_brick] 0-management:<br> Received add brick req<br>
[2013-12-18 12:12:38.888064] I<br> [glusterd-brick-ops.c:417:__glusterd_handle_add_brick] 0-management:<br> replica-count is 3<br> [2013-12-18 12:12:38.888124] I<br> [glusterd-brick-ops.c:256:gd_addbr_validate_replica_count]<br>
0-management: Changing the replica count of volume data_domain
from<br> 2 to 3<br><br><br>I'm running some VM-s on this volume so I'd really like to avoid <br>restarting glusterd service.<br>OS is FC19, kernel 3.11.10-200.fc19.x86_64, glusterfs.x86_64 3.4.1-1.fc19<br><br>
<br>tnx for help!<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/4b36895e/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/4b36895e/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 5<br>Date: Wed, 18 Dec 2013 16:11:38 +0100<br>From: Pieter Baele <<a href="mailto:pieter.baele@gmail.com" target="_blank">pieter.baele@gmail.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
Subject: [Gluster-users] Shared storage for critical infrastructure<br>Message-ID:<br>
<<a href="mailto:CADDXySqPg2jkrS4LX2sCC7uPQZ_eKF2JX60OepLFOOnRVJCiDA@mail.gmail.com" target="_blank">CADDXySqPg2jkrS4LX2sCC7uPQZ_eKF2JX60OepLFOOnRVJCiDA@mail.gmail.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br>
<br>Hello,<br><br>Some critical infrastructure needs a file share for messaging.<br>Can gluster (RH Storage) be used for HA purposes?<br><br>I was also considering GlusterFS, but I need to be sure that it is<br>compatible - and handles well -<br>
the requirements from the software vendor:<br><br>- Write Order<br>- Synchronous Write persistence<br>- Distributed File Locking<br>- Unique Write Ownership<br><br>Sincerely,<br>PieterB<br>-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/4cd25064/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/4cd25064/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 6<br>Date: Wed, 18 Dec 2013 09:23:43 -0600<br>From: Kelly Burkhart <<a href="mailto:kelly.burkhart@gmail.com" target="_blank">kelly.burkhart@gmail.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
Subject: [Gluster-users] Debugging gfapi<br>Message-ID:<br>
<<a href="mailto:CAND8VyCSM%2BE3ecRv1m8CUJc8vcTp_cdCvDfn3ETmsy83p23xEw@mail.gmail.com" target="_blank">CAND8VyCSM+E3ecRv1m8CUJc8vcTp_cdCvDfn3ETmsy83p23xEw@mail.gmail.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br>
<br>Is there some trick I need to do to use gdb on libgfapi? I configured<br>gluster like this:<br><br>./configure --disable-ibverbs --enable-debug<br><br>And also tried this:<br><br>CFLAGS=-g CPPFLAGS=-g LDFLAGS=-g ./configure --disable-ibverbs<br>
--enable-debug<br><br>I can't step into calls like glfs_new, the debugger skips over the call.<br>Is there some magic that makes gdb think that gfapi is not debuggable?<br><br>-K<br>-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/5ea2b239/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/5ea2b239/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 7<br>Date: Wed, 18 Dec 2013 11:11:40 -0500 (EST)<br>From: Kaleb Keithley <<a href="mailto:kkeithle@redhat.com" target="_blank">kkeithle@redhat.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>, <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a><br>
Subject: Re: [Gluster-users] [Gluster-devel] glusterfs-3.4.2qa4<br>
released<br>Message-ID:<br>
<<a href="mailto:2119012122.44202998.1387383100132.JavaMail.root@redhat.com" target="_blank">2119012122.44202998.1387383100132.JavaMail.root@redhat.com</a>><br>Content-Type: text/plain; charset=utf-8<br><br><br>YUM repos for EPEL (5 & 6) and Fedora (18, 19, 20) are at </font></tt><a href="http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.2qa4/" target="_blank"><tt><font>http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.4.2qa4/</font></tt></a><tt><font><br>
<br><br>----- Original Message -----<br>From: "Gluster Build System" <<a href="mailto:jenkins@build.gluster.org" target="_blank">jenkins@build.gluster.org</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>, <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a><br>
Sent: Monday, December 16, 2013 11:53:40 PM<br>Subject: [Gluster-devel] glusterfs-3.4.2qa4 released<br><br><br>RPM: </font></tt><a href="http://bits.gluster.org/pub/gluster/glusterfs/3.4.2qa4/" target="_blank"><tt><font>http://bits.gluster.org/pub/gluster/glusterfs/3.4.2qa4/</font></tt></a><tt><font><br>
<br>SRC: </font></tt><a href="http://bits.gluster.org/pub/gluster/glusterfs/src/glusterfs-3.4.2qa4.tar.gz" target="_blank"><tt><font>http://bits.gluster.org/pub/gluster/glusterfs/src/glusterfs-3.4.2qa4.tar.gz</font></tt></a><tt><font><br>
<br>This release is made off jenkins-release-53<br><br>-- Gluster Build System<br><br>_______________________________________________<br>Gluster-devel mailing list<br><a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
</font></tt><a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank"><tt><font>https://lists.nongnu.org/mailman/listinfo/gluster-devel</font></tt></a><tt><font><br><br><br>------------------------------<br>
<br>Message: 8<br>Date: Wed, 18 Dec 2013 11:36:10 -0500 (EST)<br>From: Vijay Bellur <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>, <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a><br>
Subject: [Gluster-users] Cancelled: Gluster Community Weekly Meeting<br>Message-ID:<br>
<<a href="mailto:1076006422.19710808.1387384570071.JavaMail.root@redhat.com" target="_blank">1076006422.19710808.1387384570071.JavaMail.root@redhat.com</a>><br>Content-Type: text/plain; charset="utf-8"<br>
<br>A single instance of the following meeting has been cancelled:<br><br>Subject: Gluster Community Weekly Meeting <br>Organizer: "Vijay Bellur" <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>> <br>
<br>Location: #gluster-meeting on <a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a> <br>Time: Wednesday, December 25, 2013, 8:30:00 PM - 9:30:00 PM GMT +05:30
Chennai, Kolkata, Mumbai, New Delhi<br> <br>Invitees: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>; <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a>; <a href="mailto:Christian.Heggland@nov.com" target="_blank">Christian.Heggland@nov.com</a>;
<a href="mailto:vmallika@redhat.com" target="_blank">vmallika@redhat.com</a>; <a href="mailto:bobby.jacob@alshaya.com" target="_blank">bobby.jacob@alshaya.com</a>; <a href="mailto:kevin.stevenard@alcatel-lucent.com" target="_blank">kevin.stevenard@alcatel-lucent.com</a>;
<a href="mailto:radek.dymacz@databarracks.com" target="_blank">radek.dymacz@databarracks.com</a>; <a href="mailto:pportant@redhat.com" target="_blank">pportant@redhat.com</a>; <a href="mailto:ryade@mcs.anl.gov" target="_blank">ryade@mcs.anl.gov</a>;
<a href="mailto:roger@dutchmillerauto.com" target="_blank">roger@dutchmillerauto.com</a>; <a href="mailto:Thomas.Seitner@gugler.at" target="_blank">Thomas.Seitner@gugler.at</a> ... <br><br><br>*~*~*~*~*~*~*~*~*~*<br><br>
Cancelling this instance due to the holiday season.<br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: meeting.ics<br>Type: text/calendar<br>Size: 3392 bytes<br>Desc: not available<br>
URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/dbf372c9/attachment-0001.ics" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/dbf372c9/attachment-0001.ics</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 9<br>Date: Wed, 18 Dec 2013 11:37:05 -0500 (EST)<br>From: Vijay Bellur <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>, <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a><br>
Subject: [Gluster-users] Cancelled: Gluster Community Weekly Meeting<br>Message-ID: <<a href="mailto:83718219.19711692.1387384625186.JavaMail.root@redhat.com" target="_blank">83718219.19711692.1387384625186.JavaMail.root@redhat.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br><br>A single instance of the following meeting has been cancelled:<br><br>Subject: Gluster Community Weekly Meeting <br>Organizer: "Vijay Bellur" <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>> <br>
<br>Location: #gluster-meeting on <a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a> <br>Time: Wednesday, January 1, 2014, 8:30:00 PM - 9:30:00 PM GMT +05:30 Chennai,
Kolkata, Mumbai, New Delhi<br> <br>Invitees: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>; <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a>; <a href="mailto:Christian.Heggland@nov.com" target="_blank">Christian.Heggland@nov.com</a>;
<a href="mailto:vmallika@redhat.com" target="_blank">vmallika@redhat.com</a>; <a href="mailto:bobby.jacob@alshaya.com" target="_blank">bobby.jacob@alshaya.com</a>; <a href="mailto:kevin.stevenard@alcatel-lucent.com" target="_blank">kevin.stevenard@alcatel-lucent.com</a>;
<a href="mailto:radek.dymacz@databarracks.com" target="_blank">radek.dymacz@databarracks.com</a>; <a href="mailto:pportant@redhat.com" target="_blank">pportant@redhat.com</a>; <a href="mailto:ryade@mcs.anl.gov" target="_blank">ryade@mcs.anl.gov</a>;
<a href="mailto:roger@dutchmillerauto.com" target="_blank">roger@dutchmillerauto.com</a>; <a href="mailto:Thomas.Seitner@gugler.at" target="_blank">Thomas.Seitner@gugler.at</a> ... <br><br><br>*~*~*~*~*~*~*~*~*~*<br><br>
Cancelling this instance due to the holiday season.<br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: meeting.ics<br>Type: text/calendar<br>Size: 3390 bytes<br>Desc: not available<br>
URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/b3557b59/attachment-0001.ics" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/b3557b59/attachment-0001.ics</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 10<br>Date: Wed, 18 Dec 2013 23:15:44 +0530<br>From: Vijay Bellur <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>><br>To: "'<a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a>'" <<a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a>>,<br>
gluster-users Discussion List <<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>><br>Subject: [Gluster-users] Gluster Community Weekly Meeting Minutes --<br>
2013-12-18<br>Message-ID: <<a href="mailto:52B1DF48.60104@redhat.com" target="_blank">52B1DF48.60104@redhat.com</a>><br>Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br><br>Meeting minutes available at:<br>
<br></font></tt><a href="http://meetbot.fedoraproject.org/gluster-meeting/2013-12-18/gluster-meeting.2013-12-18-15.00.html" target="_blank"><tt><font>http://meetbot.fedoraproject.org/gluster-meeting/2013-12-18/gluster-meeting.2013-12-18-15.00.html</font></tt></a><tt><font><br>
<br>-Vijay<br><br><br>------------------------------<br><br>Message: 11<br>Date: Wed, 18 Dec 2013 23:17:02 +0530<br>From: Vijay Bellur <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>><br>
To: Luk?? Bezdi?ka <<a href="mailto:lukas.bezdicka@gooddata.com" target="_blank">lukas.bezdicka@gooddata.com</a>><br>Cc: "<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>>,
Gluster<br>
Devel <<a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a>><br>Subject: Re: [Gluster-users] glusterfs-3.4.2qa4 released<br>Message-ID: <<a href="mailto:52B1DF96.7060809@redhat.com" target="_blank">52B1DF96.7060809@redhat.com</a>><br>
Content-Type: text/plain; charset=UTF-8; format=flowed<br><br>On 12/17/2013 09:27 PM, Luk?? Bezdi?ka wrote:<br>> Quite high memory usage for nfs daemon which we don't use at all.<br>> USER PID %CPU %MEM VSZ RSS
TTY STAT START TIME COMMAND<br>> root 23246 0.0 1.9 485524 313116 ?
Ssl 15:43 0:00<br>> /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p<br>> /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S<br>> /var/run/a90b5253b325435599e00f1a6534b95c.socket<br>
><br><br>Can you please check if setting volume option nfs.drc to off brings down
<br>the memory usage?<br><br>Thanks,<br>Vijay<br><br><br><br>------------------------------<br><br>Message: 12<br>Date: Wed, 18 Dec 2013 12:54:55 -0500<br>From: Jeff Darcy <<a href="mailto:jdarcy@redhat.com" target="_blank">jdarcy@redhat.com</a>><br>
To: Kelly Burkhart <<a href="mailto:kelly.burkhart@gmail.com" target="_blank">kelly.burkhart@gmail.com</a>>,<br>
<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>Subject: Re: [Gluster-users] Debugging gfapi<br>Message-ID: <<a href="mailto:52B1E16F.2030803@redhat.com" target="_blank">52B1E16F.2030803@redhat.com</a>><br>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br><br>On 12/18/2013 10:23 AM, Kelly Burkhart wrote:<br>> Is there some trick I need to do to use gdb on libgfapi? I<br>> configured gluster like this:<br>
><br>> ./configure --disable-ibverbs --enable-debug<br>><br>> And also tried this:<br>><br>> CFLAGS=-g CPPFLAGS=-g LDFLAGS=-g ./configure --disable-ibverbs<br>> --enable-debug<br>><br>> I can't step into calls like glfs_new, the debugger skips over the<br>
> call. Is there some magic that makes gdb think that gfapi is
not<br>> debuggable?<br><br>The formula for getting GlusterFS to build with the proper flags seems<br>to change frequently. If you look at <a href="http://configure.ac" target="_blank">configure.ac</a> the current magic<br>
seems to be:<br><br>
export enable_debug=yes<br>
configure/rpmbuild/whatever<br><br>That's what I do for my own build, and I'm generally able to step<br>through anything in a translator. I haven't tried with gfapi, but
it<br>should be the same because it's also a library. Executables are a<br>different matter because they're explicitly stripped during the build<br>process.<br><br><br><br>------------------------------<br><br>Message: 13<br>
Date: Wed, 18 Dec 2013 11:19:24 -0700<br>From: Knut Moe <<a href="mailto:kmoe66@gmail.com" target="_blank">kmoe66@gmail.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
Subject: [Gluster-users] Trying to start glusterd<br>Message-ID:<br>
<CADXLLPhgNiA1P=<a href="mailto:LOULw06X0b-7gbqi7r2NXaGt20WPevhDmh4A@mail.gmail.com" target="_blank">LOULw06X0b-7gbqi7r2NXaGt20WPevhDmh4A@mail.gmail.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br>
<br>I have two Ubuntu servers set up, downloaded GlusterFS 3.4 and by all<br>accounts it seems to have installed properly using the apt-get install<br>command.<br><br>However, when I issue a glusterd start or glusterd status command I am<br>
getting the following error:<br><br>ERROR: failed to create log file (/var/log/glusterfs/start.log) (Permission<br>denied).<br><br>Is there a way to determine if gluster is installed properly and also<br>troubleshoot the above?<br>
<br>If I issue sudo glusterd start or sudo glusterd status I am returned to
the<br>prompt with no additional info.<br><br>Thx.<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/933d17a0/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/933d17a0/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 14<br>Date: Wed, 18 Dec 2013 13:43:34 -0600<br>From: Kelly Burkhart <<a href="mailto:kelly.burkhart@gmail.com" target="_blank">kelly.burkhart@gmail.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
Subject: [Gluster-users] gfapi from non-root<br>Message-ID:<br>
<CAND8VyCuidxwU7whEYfkG1th-ejtM2oLsM8usDWgK0w=<a href="mailto:fpvohA@mail.gmail.com" target="_blank">fpvohA@mail.gmail.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br><br>How does one run a gfapi app without being root?<br>
</font></tt><br><tt><font>I've set server.allow-insecure on on the server side
(and bounced all<br>gluster processes). Is there something else required?<br><br>My test program just stats a file on the cluster volume. It works
as root<br>and fails as a normal user. Local log file shows a message about
failing<br>to bind a privileged port.<br><br>-K<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/16f6e37c/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/16f6e37c/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 15<br>Date: Thu, 19 Dec 2013 01:16:15 +0530<br>From: Chalcogen <<a href="mailto:chalcogen_eg_oxygen@yahoo.com" target="_blank">chalcogen_eg_oxygen@yahoo.com</a>><br>
To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>Subject: [Gluster-users] Passing noforget option to glusterfs native<br>
client
mounts<br>Message-ID: <<a href="mailto:52B1FB87.8070002@yahoo.com" target="_blank">52B1FB87.8070002@yahoo.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"<br><br>Hi everybody,<br>
<br>A few months back I joined a project where people want to replace their
<br>legacy fuse-based (twin-server) replicated file-system with GlusterFS.
<br>They also have a high-availability NFS server code tagged with the <br>kernel NFSD that they would wish to retain (the nfs-kernel-server, I <br>mean). The reason they wish to retain the kernel NFS and not use the NFS
<br>server that comes with GlusterFS is mainly because there's this bit of
<br>code that allows NFS IP's to be migrated from one host server to the <br>other in the case that one happens to go down, and tweaks on the export
<br>server configuration allow the file-handles to remain identical on the
<br>new host server.<br><br>The solution was to mount gluster volumes using the mount.glusterfs <br>native client program and then export the directories over the kernel <br>NFS server. This seems to work most of the time, but on rare occasions,
<br>'stale file handle' is reported off certain clients, which really puts
a <br>damper over the 'high-availability' thing. After suitably instrumenting
<br>the nfsd/fuse code in the kernel, it seems that decoding of the <br>file-handle fails on the server because the inode record corresponding
<br>to the nodeid in the handle cannot be looked up. Combining this with the
<br>fact that a second attempt by the client to execute lookup on the same
<br>file passes, one might suspect that the problem is identical to what <br>many people attempting to export fuse mounts over the kernel's NFS <br>server are facing; viz, fuse 'forgets' the inode records thereby causing
<br>ilookup5() to fail. Miklos and other fuse developers/hackers would point
<br>towards '-o noforget' while mounting their fuse file-systems.<br><br>I tried passing '-o noforget' to mount.glusterfs, but it does not
seem <br>to recognize it. Could somebody help me out with the correct syntax to
<br>pass noforget to gluster volumes? Or, something we could pass to <br>glusterfs that would instruct fuse to allocate a bigger cache for our <br>inodes?<br><br>Additionally, should you think that something else might be behind our
<br>problems, please do let me know.<br><br>Here's my configuration:<br><br>Linux kernel version: 2.6.34.12<br>GlusterFS versionn: 3.4.0<br>nfs.disable option for volumes: OFF on all volumes<br><br>Thanks a lot for your time!<br>
Anirban<br><br>P.s. I found quite a few pages on the web that admonish users that <br>GlusterFS is not compatible with the kernel NFS server, but do not <br>really give much detail. Is this one of the reasons for saying so?<br>
-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/35365020/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/35365020/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 16<br>Date: Thu, 19 Dec 2013 01:40:29 +0530<br>From: Chalcogen <<a href="mailto:chalcogen_eg_oxygen@yahoo.com" target="_blank">chalcogen_eg_oxygen@yahoo.com</a>><br>
To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>Subject: Re: [Gluster-users] Passing noforget option to glusterfs<br>
native
client mounts<br>Message-ID: <<a href="mailto:52B20135.6030902@yahoo.com" target="_blank">52B20135.6030902@yahoo.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"<br><br>
P.s. I think I need to clarify this:<br><br>I am only reading from the mounts, and not modifying anything on the <br>server. and so the commonest causes on stale file handles do not appy.<br><br>Anirban<br><br>On Thursday 19 December 2013 01:16 AM, Chalcogen wrote:<br>
> Hi everybody,<br>><br>> A few months back I joined a project where people want to replace
<br>> their legacy fuse-based (twin-server) replicated file-system with
<br>> GlusterFS. They also have a high-availability NFS server code tagged
<br>> with the kernel NFSD that they would wish to retain (the <br>> nfs-kernel-server, I mean). The reason they wish to retain the kernel
<br>> NFS and not use the NFS server that comes with GlusterFS is mainly
<br>> because there's this bit of code that allows NFS IP's to be migrated
<br>> from one host server to the other in the case that one happens to
go <br>> down, and tweaks on the export server configuration allow the <br>> file-handles to remain identical on the new host server.<br>><br>> The solution was to mount gluster volumes using the mount.glusterfs
<br>> native client program and then export the directories over the kernel
<br>> NFS server. This seems to work most of the time, but on rare <br>> occasions, 'stale file handle' is reported off certain clients, which
<br>> really puts a damper over the 'high-availability' thing. After <br>> suitably instrumenting the nfsd/fuse code in the kernel, it seems
that <br>> decoding of the file-handle fails on the server because the inode
<br>> record corresponding to the nodeid in the handle cannot be looked
up. <br>> Combining this with the fact that a second attempt by the client to
<br>> execute lookup on the same file passes, one might suspect that the
<br>> problem is identical to what many people attempting to export fuse
<br>> mounts over the kernel's NFS server are facing; viz, fuse 'forgets'
<br>> the inode records thereby causing ilookup5() to fail. Miklos and other
<br>> fuse developers/hackers would point towards '-o noforget' while <br>> mounting their fuse file-systems.<br>><br>> I tried passing '-o noforget' to mount.glusterfs, but it does
not <br>> seem to recognize it. Could somebody help me out with the correct
<br>> syntax to pass noforget to gluster volumes? Or, something we could
<br>> pass to glusterfs that would instruct fuse to allocate a bigger cache
<br>> for our inodes?<br>><br>> Additionally, should you think that something else might be behind
our <br>> problems, please do let me know.<br>><br>> Here's my configuration:<br>><br>> Linux kernel version: 2.6.34.12<br>> GlusterFS versionn: 3.4.0<br>> nfs.disable option for volumes: OFF on all volumes<br>
><br>> Thanks a lot for your time!<br>> Anirban<br>><br>> P.s. I found quite a few pages on the web that admonish users that
<br>> GlusterFS is not compatible with the kernel NFS server, but do not
<br>> really give much detail. Is this one of the reasons for saying so?<br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/f6b6b5bd/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/f6b6b5bd/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 17<br>Date: Thu, 19 Dec 2013 04:27:19 +0800 (SGT)<br>From: Anirban Ghoshal <<a href="mailto:chalcogen_eg_oxygen@yahoo.com" target="_blank">chalcogen_eg_oxygen@yahoo.com</a>><br>
To: Sa?a Friedrich <<a href="mailto:sasa.friedrich@bitlab.si" target="_blank">sasa.friedrich@bitlab.si</a>>,<br>
"<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br>Subject: Re: [Gluster-users] Problem adding brick (replica)<br>
Message-ID:<br>
<<a href="mailto:1387398439.78214.YahooMailNeo@web193901.mail.sg3.yahoo.com" target="_blank">1387398439.78214.YahooMailNeo@web193901.mail.sg3.yahoo.com</a>><br>Content-Type: text/plain; charset="utf-8"<br>
<br>Ok, I am not associated with, or part of the glusterFS development team
in any way; I fact I only started using glusterfs since the past 3-4 months
or so, but I have often observed that useful info might be found at <log
file dir>/.cmd_history.log, which is, in your case,<br><br>/var/log/glusterfs/.cmd_history.log<br><br><br><br><br>On Wednesday, 18 December 2013 8:08 PM, Sa?a Friedrich <<a href="mailto:sasa.friedrich@bitlab.si" target="_blank">sasa.friedrich@bitlab.si</a>>
wrote:<br> <br>Hi!<br><br>I have some trouble adding a brick to existing gluster volume.<br><br>When I try to (in CLI):<br><br><br>gluster> volume add-brick data_domain replica 3 gluster2.data:/glusterfs/data_domain<br>
><br>><br>I get:<br><br><br>volume add-brick: failed:<br>><br>><br>I probed the peer successfully, peer status returns:<br><br><br>Hostname: gluster3.data<br>>Uuid: e694f552-636a-4cf3-a04f-997ec87a880c<br>>State: Peer in Cluster (Connected)<br>
><br>>Hostname: gluster2.data<br>>Port: 24007<br>>Uuid: 36922d4c-55f2-4cc6-85b9-a9541e5619a2<br>>State: Peer in Cluster (Connected)<br>><br>><br>Existing volume info:<br><br><br>Volume Name: data_domain<br>
>Type: Replicate<br>>Volume ID: ae096e7d-cf0c-46ed-863a-9ecc3e8ce288<br>>Status: Started<br>>Number of Bricks: 1 x 2 = 2<br>>Transport-type: tcp<br>>Bricks:<br>>Brick1: gluster1.data:/glusterfs/data_domain<br>
>Brick2: gluster3.data:/glusterfs/data_domain<br>>Options Reconfigured:<br>>storage.owner-gid: 36<br>>storage.owner-uid: 36<br>>server.allow-insecure: on<br>>network.remote-dio: enable<br>>cluster.eager-lock: enable<br>
>performance.stat-prefetch: off<br>>performance.io-cache: off<br>>performance.read-ahead: off<br>>performance.quick-read: off<br>><br>Only thing I found in log is:<br><br><br>(/var/log/glusterfs/cli.log)<br>
>[2013-12-18 12:09:17.281310] W [cli-rl.c:106:cli_rl_process_line]<br> 0-glusterfs: failed to process line<br>>[2013-12-18 12:10:07.650267] I<br> [cli-rpc-ops.c:332:gf_cli_list_friends_cbk] 0-cli:
Received resp<br> to list: 0<br>><br>>(/var/log/glusterfs/etc-glusterfs-glusterd.vol.log)<br>>[2013-12-18 12:12:38.887911] I<br> [glusterd-brick-ops.c:370:__glusterd_handle_add_brick]<br> 0-management: Received add brick req<br>
>[2013-12-18 12:12:38.888064] I<br> [glusterd-brick-ops.c:417:__glusterd_handle_add_brick]<br> 0-management: replica-count is 3<br>>[2013-12-18 12:12:38.888124] I<br> [glusterd-brick-ops.c:256:gd_addbr_validate_replica_count]<br>
0-management: Changing the replica count of volume
data_domain<br> from 2 to 3<br>><br>I'm running some VM-s on this volume so I'd really like to avoid<br> restarting glusterd service.<br>OS is FC19, kernel 3.11.10-200.fc19.x86_64, glusterfs.x86_64<br>
3.4.1-1.fc19<br><br><br>tnx for help!<br><br>_______________________________________________<br>Gluster-users mailing list<br><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
</font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank"><tt><font>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font><br>-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/d9bec9fd/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/d9bec9fd/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 18<br>Date: Wed, 18 Dec 2013 21:32:43 +0100<br>From: Sa?a Friedrich <<a href="mailto:sasa.friedrich@bitlab.si" target="_blank">sasa.friedrich@bitlab.si</a>><br>To: Anirban Ghoshal <<a href="mailto:chalcogen_eg_oxygen@yahoo.com" target="_blank">chalcogen_eg_oxygen@yahoo.com</a>>,<br>
"<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br>Subject: Re: [Gluster-users] Problem adding brick (replica)<br>
Message-ID: <<a href="mailto:52B2066B.1070606@bitlab.si" target="_blank">52B2066B.1070606@bitlab.si</a>><br>Content-Type: text/plain; charset="utf-8"; Format="flowed"<br><br>Here is the line that gets thrown in that log file (which I wasn't aware
<br>of - thanks Anirban)<br><br>[2013-12-18 20:31:21.005913] : volume add-brick iso_domain replica
2 <br>gluster2.data:/glusterfs/iso_domain : FAILED :<br><br><br><br>Dne 18. 12. 2013 21:27, pi?e Anirban Ghoshal:<br>> Ok, I am not associated with, or part of the glusterFS development
<br>> team in any way; I fact I only started using glusterfs since the past
<br>> 3-4 months or so, but I have often observed that useful info might
be <br>> found at <log file dir>/.cmd_history.log, which is, in your
case,<br>><br>> /var/log/glusterfs/.cmd_history.log<br>><br>><br>><br>> On Wednesday, 18 December 2013 8:08 PM, Sa?a Friedrich <br>> <<a href="mailto:sasa.friedrich@bitlab.si" target="_blank">sasa.friedrich@bitlab.si</a>> wrote:<br>
> Hi!<br>><br>> I have some trouble adding a brick to existing gluster volume.<br>><br>> When I try to (in CLI):<br>><br>> gluster> volume add-brick data_domain replica 3<br>> gluster2.data:/glusterfs/data_domain<br>
><br>> I get:<br>><br>> volume add-brick: failed:<br>><br>> I probed the peer successfully, peer status returns:<br>><br>> Hostname: gluster3.data<br>> Uuid: e694f552-636a-4cf3-a04f-997ec87a880c<br>
> State: Peer in Cluster (Connected)<br>><br>> Hostname: gluster2.data<br>> Port: 24007<br>> Uuid: 36922d4c-55f2-4cc6-85b9-a9541e5619a2<br>> State: Peer in Cluster (Connected)<br>><br>
> Existing volume info:<br>><br>> Volume Name: data_domain<br>> Type: Replicate<br>> Volume ID: ae096e7d-cf0c-46ed-863a-9ecc3e8ce288<br>> Status: Started<br>> Number of Bricks: 1 x 2 = 2<br>
> Transport-type: tcp<br>> Bricks:<br>> Brick1: gluster1.data:/glusterfs/data_domain<br>> Brick2: gluster3.data:/glusterfs/data_domain<br>> Options Reconfigured:<br>> storage.owner-gid: 36<br>
> storage.owner-uid: 36<br>> server.allow-insecure: on<br>> network.remote-dio: enable<br>> cluster.eager-lock: enable<br>> performance.stat-prefetch: off<br>> performance.io-cache: off<br>
> performance.read-ahead: off<br>> performance.quick-read: off<br>><br>><br>> Only thing I found in log is:<br>><br>> (/var/log/glusterfs/cli.log)<br>> [2013-12-18 12:09:17.281310] W [cli-rl.c:106:cli_rl_process_line]<br>
> 0-glusterfs: failed to process line<br>> [2013-12-18 12:10:07.650267] I<br>> [cli-rpc-ops.c:332:gf_cli_list_friends_cbk] 0-cli: Received
resp<br>> to list: 0<br>><br>> (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log)<br>> [2013-12-18 12:12:38.887911] I<br>> [glusterd-brick-ops.c:370:__glusterd_handle_add_brick]<br>> 0-management: Received add brick req<br>
> [2013-12-18 12:12:38.888064] I<br>> [glusterd-brick-ops.c:417:__glusterd_handle_add_brick]<br>> 0-management: replica-count is 3<br>> [2013-12-18 12:12:38.888124] I<br>> [glusterd-brick-ops.c:256:gd_addbr_validate_replica_count]<br>
> 0-management: Changing the replica count of volume data_domain<br>> from 2 to 3<br>><br>><br>> I'm running some VM-s on this volume so I'd really like to avoid <br>> restarting glusterd service.<br>
> OS is FC19, kernel 3.11.10-200.fc19.x86_64, glusterfs.x86_64 3.4.1-1.fc19<br>><br>><br>> tnx for help!<br>><br>> _______________________________________________<br>> Gluster-users mailing list<br>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a> <</font></tt><a href="mailto:Gluster-users@gluster.org" target="_blank"><tt><font>mailto:Gluster-users@gluster.org</font></tt></a><tt><font>><br>
> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank"><tt><font>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font><br>><br><br>-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/f357938f/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/f357938f/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 19<br>Date: Wed, 18 Dec 2013 14:49:28 -0600<br>From: Kelly Burkhart <<a href="mailto:kelly.burkhart@gmail.com" target="_blank">kelly.burkhart@gmail.com</a>><br>To: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>
Subject: Re: [Gluster-users] gfapi from non-root<br>Message-ID:<br>
<CAND8VyDW=<a href="mailto:V3qZCQoOwYg2f1GVdhEn4LO6ch8u_uN5R2w9kBZaA@mail.gmail.com" target="_blank">V3qZCQoOwYg2f1GVdhEn4LO6ch8u_uN5R2w9kBZaA@mail.gmail.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br>
<br>OK, I believe I solved it by doing this by 1. setting the volume property<br>allow-insecure with the following command:<br><br>gluster volume set gv0 server.allow-insecure on<br><br>and 2. editing the /usr/local/etc/glusterfs/glusterd.vol file and adding<br>
the following line between 'volume-management' and 'end-volume':<br><br> option rpc-auth-allow-insecure on<br><br><br>Is there some mechanism for setting glusterd.vol options without manually<br>editing a file on each host in the cluster?<br>
If I add a new host to the cluster at a later point, will it slurp the<br>glusterd.vol file from one of the already established hosts? Or do
I have<br>to manage keeping this file identical on every host?<br><br>-K<br><br><br>On Wed, Dec 18, 2013 at 1:43 PM, Kelly Burkhart <<a href="mailto:kelly.burkhart@gmail.com" target="_blank">kelly.burkhart@gmail.com</a>>wrote:<br>
<br>> How does one run a gfapi app without being root?<br>><br>> I've set server.allow-insecure on on the server side (and bounced
all<br>> gluster processes). Is there something else required?<br>><br>> My test program just stats a file on the cluster volume. It
works as root<br>> and fails as a normal user. Local log file shows a message about
failing<br>> to bind a privileged port.<br>><br>> -K<br>><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/5c79ada1/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/5c79ada1/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 20<br>Date: Wed, 18 Dec 2013 14:00:59 -0800 (PST)<br>From: William Kwan <<a href="mailto:potatok@yahoo.com" target="_blank">potatok@yahoo.com</a>><br>To: "<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>" <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br>
Subject: [Gluster-users] failed to create volume ends with a prefix of<br>
it is
already part of a volume<br>Message-ID:<br>
<<a href="mailto:1387404059.18772.YahooMailNeo@web140403.mail.bf1.yahoo.com" target="_blank">1387404059.18772.YahooMailNeo@web140403.mail.bf1.yahoo.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br>
<br>Hi all,<br><br>Env: CentOS 6.5 with?glusterfs 3.4.1?<br><br>I just start working on Gluster. ?I have two test hosts. ?Both of them
have a xfs on top of LVM. ? I searched, but there are lots of result like
this. I'm not sure if this is a bug in my version?<br><br># gluster volume create gvol1 replica 2 transport tcp ghost1:/data ghost2:/data<br>volume create: gvol1: failed<br># gluster volume list all<br>No volumes present in cluster<br>
# gluster volume create gvol1 replica 2 transport tcp ghost1:/data ghost2:/data<br>volume create: gvol1: failed: /data or a prefix of it is already part of
a volume<br><br>Thanks<br>Will<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/c1967735/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131218/c1967735/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 21<br>Date: Thu, 19 Dec 2013 07:38:35 +0530<br>From: Kaushal M <<a href="mailto:kshlmster@gmail.com" target="_blank">kshlmster@gmail.com</a>><br>To: Knut Moe <<a href="mailto:kmoe66@gmail.com" target="_blank">kmoe66@gmail.com</a>><br>
Cc: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>Subject: Re: [Gluster-users] Trying to start glusterd<br>Message-ID:<br>
<CAOujamU6qu5x6vdXSrOVsdV36xnVJgp+po+t=<a href="mailto:moJh00Fk9VM4g@mail.gmail.com" target="_blank">moJh00Fk9VM4g@mail.gmail.com</a>><br>Content-Type: text/plain; charset="iso-8859-1"<br><br>All gluster processes need to be run as root. So you will need to use<br>
'sudo' to run any command.<br><br>On Ubuntu, the correct way to start Glusterd is by using 'service glusterd<br>start'. You can check if glusterd is running using 'service glusterd<br>status'.<br>On 18-Dec-2013 11:49 PM, "Knut Moe" <<a href="mailto:kmoe66@gmail.com" target="_blank">kmoe66@gmail.com</a>>
wrote:<br><br>> I have two Ubuntu servers set up, downloaded GlusterFS 3.4 and by
all<br>> accounts it seems to have installed properly using the apt-get install<br>> command.<br>><br>> However, when I issue a glusterd start or glusterd status command
I am<br>> getting the following error:<br>><br>> ERROR: failed to create log file (/var/log/glusterfs/start.log)<br>> (Permission denied).<br>><br>> Is there a way to determine if gluster is installed properly and also<br>
> troubleshoot the above?<br>><br>> If I issue sudo glusterd start or sudo glusterd status I am returned
to<br>> the prompt with no additional info.<br>><br>> Thx.<br>><br>> _______________________________________________<br>> Gluster-users mailing list<br>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank"><tt><font>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font><br>><br>-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/f2f750a4/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/f2f750a4/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>Message: 22<br>Date: Thu, 19 Dec 2013 12:34:59 +0530<br>From: Vijay Bellur <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>><br>To: Joe Topjian <<a href="mailto:joe@topjian.net" target="_blank">joe@topjian.net</a>><br>
Cc: <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>, Bharata B Rao<br>
<<a href="mailto:bharata@linux.vnet.ibm.com" target="_blank">bharata@linux.vnet.ibm.com</a>><br>Subject: Re: [Gluster-users] qemu remote insecure connections<br>Message-ID: <<a href="mailto:52B29A9B.3040503@redhat.com" target="_blank">52B29A9B.3040503@redhat.com</a>><br>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br><br>On 12/16/2013 08:42 AM, Joe Topjian wrote:<br>> Hello,<br>><br>> I apologize for the delayed reply.<br>><br>> I've collected some logs and posted them here:<br>
> </font></tt><a href="https://gist.github.com/jtopjian/7981763" target="_blank"><tt><font>https://gist.github.com/jtopjian/7981763</font></tt></a><tt><font><br>><br>> I stopped the Gluster service on 192.168.1.11, moved /var/log/glusterfs<br>
> to a backup, then started Gluster so that the log files were more
succinct.<br>><br>> I then used the qemu-img command as mentioned before as root, which
was<br>> successful. Then I ran the command as libvirt-qemu and let the command<br>> hang for 2 minutes before I killed it.<br>><br><br>I did not notice anything in the logs which refer to failures from a <br>gluster perspective. This is observed in the log file:<br>
<br>"[2013-12-16 02:58:16.078774] I <br>[client-handshake.c:1456:client_setvolume_cbk] 0-instances-client-1: <br>Connected to <a href="http://192.168.1.12:49152" target="_blank">192.168.1.12:49152</a>, attached to remote volume <br>
'/gluster/instances'."<br><br>It does look like a connection has been established but qemu-img is <br>blocked on something. Can you please start qemu-img with strace -f and
<br>capture the output?<br><br>Bharata: Any additional things that we could try here?<br><br>Thanks,<br>Vijay<br><br><br>------------------------------<br><br>Message: 23<br>Date: Thu, 19 Dec 2013 08:55:39 +0000<br>From: "Bernhard Glomm" <<a href="mailto:bernhard.glomm@ecologic.eu" target="_blank">bernhard.glomm@ecologic.eu</a>><br>
To: <a href="mailto:potatok@yahoo.com" target="_blank">potatok@yahoo.com</a>, <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><br>Subject: Re: [Gluster-users] failed to create volume ends with a<br>
prefix of
it is already part of a volume<br>Message-ID: <<a href="mailto:0c18df4ee6aa911fd20cfe3ed5ab2ad2d4e3384c@ecologic.eu" target="_blank">0c18df4ee6aa911fd20cfe3ed5ab2ad2d4e3384c@ecologic.eu</a>><br>Content-Type: text/plain; charset="utf-8"<br>
<br><br>Hi Will,<br><br>Had similar issues.<br>Did you see?<a href="http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/" target="_blank">http://joejulian.name/blog/glusterfs-path-or-a-prefix-of-it-is-already-part-of-a-volume/</a><br>
<br><br>run:<br>setfattr -x trusted.glusterfs.volume-id $brick_path<br>setfattr -x trusted.gfid $brick_path<br>rm -rf $brick_path/.glusterfs<br><br><br>on BOTH/ALL sides of your mirror<br>than run<br><br>gluster peer probe <partnerhost><br>
<br>on BOTH/ALL sides of your mirror<br>only than run?<br><br>gluster volume create ....<br><br>hth<br><br>Bernhard<br>Am 18.12.2013 23:00:59, schrieb William Kwan:<br>> Hi all,<br>> <br>> Env: CentOS 6.5 with?glusterfs 3.4.1?<br>
> <br>> I just start working on Gluster. ?I have two test hosts. ?Both of
them have a xfs on top of LVM. ? I searched,<br> but there are lots of result like this. I'm not sure if this is a bug
in my version?<br>> <br>> # gluster volume create gvol1 replica 2 transport tcp ghost1:/data
ghost2:/data<br>> > volume create: gvol1: failed<br>> > # gluster volume list all<br>> > No volumes present in cluster<br>> > # gluster volume create gvol1 replica 2 transport tcp ghost1:/data
ghost2:/data</font></tt><br><tt><font>> <br>> > volume create: gvol1: failed: /data or a prefix of it is already
part of a volume<br>> <br><br>> <br>> > Thanks<br>> > Will<br>> <br>> <br>> > _______________________________________________<br>> Gluster-users mailing list<br>> <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank"><tt><font>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font><br><br><br><br>-- <br>
<br><br> <br> <br> <br> <br> <br> <br> <br> Bernhard Glomm<br><br> IT Administration<br><br><br> <br> <br> Phone:<br> <br>
<br> +49 (30)
86880 134<br> <br> <br> Fax:<br> <br> <br> +49 (30)
86880 100<br> <br> <br> Skype:<br> <br> <br> bernhard.glomm.ecologic<br> <br> <br> <br> <br>
<br> <br> <br> <br> <br> <br> <br> <br> <br> <br> <br> <br> Ecologic Institut gemeinn?tzige GmbH
| Pfalzburger Str. 43/44 | 10717 Berlin | Germany<br><br> GF: R. Andreas Kraemer | AG: Charlottenburg
HRB 57947 | USt/VAT-IdNr.: DE811963464<br><br> Ecologic? is a Trade Mark (TM) of Ecologic
Institut gemeinn?tzige GmbH<br> <br> <br> <br> <br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/382a606b/attachment-0001.html" target="_blank"><tt><font>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131219/382a606b/attachment-0001.html</font></tt></a><tt><font>><br>
<br>------------------------------<br><br>_______________________________________________<br>Gluster-users mailing list<br><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br></font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank"><tt><font>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font><br>
<br>End of Gluster-users Digest, Vol 68, Issue 20<br>*********************************************<br></font></tt><br><br>
**<br>
<br>
This email and any attachments may contain information that is confidential and/or privileged for the sole use of the intended recipient. Any use, review, disclosure, copying, distribution or reliance by others, and any forwarding of this email or its contents, without the express permission of the sender is strictly prohibited by law. If you are not the intended recipient, please contact the sender immediately, delete the e-mail and destroy all copies.<br>
**<br><br>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br></div></div>