<html><body><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; color: #000000"><div>Hi Joe,</div><div><br></div><div>since I only use GlusterFS in production, I cannot reboot my servers whenever I want (mainly due to this "VM freeze" issue). I'll try it next time I have to reboot them, probably in the next two weeks.</div><div><br></div><div><span style="font-size: 10pt;">Thank you</span></div><div><span name="x"></span><div><meta http-equiv="content-type" content="text/html; charset=utf-8"></div><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0cm; margin-left: 0px; font-size: 10pt; font-family: arial, helvetica, sans-serif; " data-mce-style="margin-top: 0px; margin-right: 0px; margin-bottom: 0cm; margin-left: 0px; font-size: 10pt; font-family: arial, helvetica, sans-serif;"><b>Fabio Rosati</b></p><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0cm; margin-left: 0px; font-size: 10pt; font-family: arial, helvetica, sans-serif; " data-mce-style="margin-top: 0px; margin-right: 0px; margin-bottom: 0cm; margin-left: 0px; font-size: 10pt; font-family: arial, helvetica, sans-serif;"><br></p><span name="x"></span><br></div><hr id="zwchr"><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;" data-mce-style="color: #000; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>Da: </b>"Joe Julian" <joe@julianfamily.org><br><b>A: </b>"Fabio Rosati" <fabio.rosati@geminformatica.it>, "Paul Penev" <ppquant@gmail.com><br><b>Cc: </b>Gluster-users@gluster.org<br><b>Inviato: </b>Mercoledì, 9 aprile 2014 16:24:53<br><b>Oggetto: </b>Re: [Gluster-users] libgfapi failover problem on replica bricks<br><div><br></div>I've asked before and not had any luck getting someone to try this:<br> <br> On the server you're about to reboot, "killall glusterfsd" and let me know if you still see the same problem at the client. <br><div><br></div><div class="gmail_quote">On April 9, 2014 1:19:43 AM PDT, Fabio Rosati <fabio.rosati@geminformatica.it> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;" data-mce-style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;"><pre class="k9mail">Hi Paul,<br></pre><div><br></div><pre class="k9mail">you're not alone. I get the same issue after rebooting a brick belonging to a 2 x 2 volume and the same is true for João P. and Nick M. (added in cc).<br></pre><div><br></div><pre class="k9mail">[root@networker ~]# gluster volume info gv_pri<br> <br>Volume Name: gv_pri<br>Type: Distributed-Replicate<br>Volume ID: 3d91b91e-4d72-484f-8655-e5ed8d38bb28<br>Status: Started<br>Number of Bricks: 2 x 2 = 4<br>Transport-type: tcp<br>Bricks:<br>Brick1: nw1glus.gem.local:/glustexp/pri1/brick<br>Brick2: nw2glus.gem.local:/glustexp/pri1/brick<br>Brick3: nw3glus.gem.local:/glustexp/pri2/brick<br>Brick4: nw4glus.gem.local:/glustexp/pri2/brick<br>Options Reconfigured:<br>storage.owner-gid: 107<br>storage.owner-uid: 107<br>server.allow-insecure: on<br>network.remote-dio: on<br>performance.write-behind-window-size: 16MB<br>performance.cache-size: 128MB<br></pre><div><br></div><pre class="k9mail"><br>I hope someone will address this problem in the near future since not being able to
shutdown a server hosting a brick is a big limitation.<br>It seems someone solved the problem using cgroups: <a href="http://www.gluster.org/author/andrew-lau" target="_blank" data-mce-href="http://www.gluster.org/author/andrew-lau">http://www.gluster.org/author/andrew-lau</a>/<br>Anyway, I think it's not easy to implement because cgroups is already configured and in use for libvirt, if I had a test environment and some spare time I would have tried.<br></pre><div><br></div><pre class="k9mail"><br>Regards,<br>Fabio Rosati <br></pre><div><br></div><pre class="k9mail"><br>----- Messaggio originale -----<br>Da: "Paul Penev" <ppquant@gmail.com><br>A: Gluster-users@gluster.org<br>Inviato: Domenica, 6 aprile 2014 17:52:53<br>Oggetto: [Gluster-users] libgfapi failover problem on replica bricks<br></pre><div><br></div><pre class="k9mail">Hello,<br></pre><div><br></div><pre class="k9mail">I'm having an issue with rebooting bricks holding images for live KVM<br>machines (using libgfapi).<br></pre><div><br></div><pre class="k9mail">I have a replicated+distributed setup of 4 bricks (2x2). The cluster<br>contains images for a couple of kvm virtual machines.<br></pre><div><br></div><pre class="k9mail">My problem is that when I reboot
a brick containing a an image of a<br>VM, the VM will start throwing disk errors and eventually die.<br></pre><div><br></div><pre class="k9mail">The gluster volume is made like this:<br></pre><div><br></div><pre class="k9mail"># gluster vol info pool<br></pre><div><br></div><pre class="k9mail">Volume Name: pool<br>Type: Distributed-Replicate<br>Volume ID: xxxxxxxxxxxxxxxxxxxx<br>Status: Started<br>Number of Bricks: 2 x 2 = 4<br>Transport-type: tcp<br>Bricks:<br>Brick1: srv10g:/data/gluster/brick<br>Brick2: srv11g:/data/gluster/brick<br>Brick3: srv12g:/data/gluster/brick<br>Brick4: srv13g:/data/gluster/brick<br>Options Reconfigured:<br>network.ping-timeout: 10<br>cluster.server-quorum-type: server<br>diagnostics.client-log-level: WARNING<br>auth.allow: 192.168.0.*,127.*<br>nfs.disable: on<br></pre><div><br></div><pre class="k9mail">The KVM instances run on the same gluster bricks, with disks mounted<br>as : file=gluster://localhost/pool/images/vm-xxx-disk-1.raw,.......,cache=writethrough,aio=native<br></pre><div><br></div><pre class="k9mail">My self-heal backlog is not always 0. It looks like some writes
are<br>not going to all bricks at the same time (?).<br></pre><div><br></div><pre class="k9mail">gluster vol heal pool info<br></pre><div><br></div><pre class="k9mail">sometime shows the images needing sync on one brick, the other or both.<br></pre><div><br></div><pre class="k9mail">There are no network problems or errors on the wire.<br></pre><div><br></div><pre class="k9mail">Any ideas what could be causing this ?<br></pre><div><br></div><pre class="k9mail">Thanks.<br></pre><hr><pre class="k9mail"><br>Gluster-users mailing list<br>Gluster-users@gluster.org<br><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank" data-mce-href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br></pre><hr><pre class="k9mail"><br>Gluster-users mailing list<br>Gluster-users@gluster.org<br><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank" data-mce-href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre></blockquote></div><br> -- <br> Sent from my Android device with K-9 Mail. Please excuse my brevity.</div><div><br></div></div></body></html>