<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><title></title><style type="text/css">.felamimail-body-blockquote {margin: 5px 10px 0 3px;padding-left: 10px;border-left: 2px solid #000088;} </style></head><body>On 17.01.2014 16:45:20, Samuli Heinonen wrote:<br><blockquote class="felamimail-body-blockquote">Hello Bernhard,<div><br></div><div>Can you test if setting option network.remote-dio to enable allows you to use cache=none?</div><div></div></blockquote><p><br></p><p>Hi Samuli,</p><p>nope network.remote-dio enable didn't changed it.</p><p>got a too unclean system meanwhile,</p><p>will do a full reinstall of the hosts</p><p>and than come back to this topic</p><p>thnx </p><p>Bernhard</p><p><br></p><blockquote class="felamimail-body-blockquote"><div>-samuli</div><div><br><div><div>Bernhard Glomm <<a href="mailto:bernhard.glomm@ecologic.eu" target="_blank">bernhard.glomm@ecologic.eu</a>> kirjoitti 17.1.2014 kello 16.41:</div><br class="Apple-interchange-newline"><blockquote class="felamimail-body-blockquote"><div style="font-family:Helvetica;font-size:medium;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;"><div>Pranith,</div><div>I stopped the volume</div><div>started it again,</div><div>mounted it on both hosts</div><div>started the VM</div><div>did the livemigration</div><div>and collected the logs:</div><div>- etc-glusterfs-glusterd.vol.log</div><div>- glustershd.log</div><div>- srv-vm_infrastructure-vm-atom01.log</div><div>- cli.log</div><div>from the beginning of the gluster volume start.</div><div>You can found them here (part 1 to 3):</div><div><a href="http://pastebin.com/mnATm2BE" target="_blank">http://pastebin.com/mnATm2BE</a></div><div><a href="http://pastebin.com/RYZFP3E9" target="_blank">http://pastebin.com/RYZFP3E9</a></div><div><a href="http://pastebin.com/HAXEGd54" target="_blank">http://pastebin.com/HAXEGd54</a></div><div><br></div><div>further more:</div><div>gluster --version: glusterfs 3.4.2 built on Jan 11 2014 03:21:47</div><div>ubuntu: raring</div><div>filesystem on the gluster bricks: zfs-0.6.2</div><div><br></div><div><div>gluster volume info fs_vm_atom01 </div><div>Volume Name: fs_vm_atom01</div><div>Type: Replicate</div><div>Volume ID: fea9bdcf-783e-442a-831d-f564f8dbe551</div><div>Status: Started</div><div>Number of Bricks: 1 x 2 = 2</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: 172.24.1.11:/zp_ping_1/fs_vm_atom01</div><div>Brick2: 172.24.1.13:/zp_pong_1/fs_vm_atom01</div><div>Options Reconfigured:</div><div>diagnostics.client-log-level: DEBUG</div><div>server.allow-insecure: on</div></div><div><br></div><div>disk part of VM configuration</div><div><br></div><div><div><emulator>/usr/bin/kvm-spice</emulator></div><div> <disk type='file' device='disk'></div><div> <driver name='qemu' type='raw' cache='writethrough'/></div><div> <source file='/srv/vm_infrastructure/vm/atom01/atom01.img'/></div><div> <target dev='vda' bus='virtio'/></div><div> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/></div><div> </disk></div></div><div><br></div><div>can't use <source protocol='gluster' ...</div><div>as josh suggested because couldn't get</div><div>my qemu recompiled with gluster enabled yet.</div><div><br></div><div>Are there other special tuning parameter for kvm/qemu/ to set on gluster?</div><div>as mentioned: all works except the livemigration (disk image file becomes read only)</div><div>and I have to use something different than cache=none...</div><div><br></div><div>TIA</div><div><br></div><div>Bernhard</div><div><br></div><div><br></div>On 17.01.2014 05:04:52, Pranith Kumar Karampuri wrote:<br><blockquote class="felamimail-body-blockquote" style="margin:5px 10px 0px 3px;padding-left:10px;border-left-width:2px;border-left-style:solid;border-left-color:rgb(0,0,136);">Bernhard,<br>Configuration seems ok. Could you please give the log files of the bricks and mount please. If you think it is not a big procedure to do this live migration, could you set client-log-level to DEBUG and provide the log files of that run.<br><br>Pranith<br><br>----- Original Message -----<br><blockquote class="felamimail-body-blockquote" style="margin:5px 10px 0px 3px;padding-left:10px;border-left-width:2px;border-left-style:solid;border-left-color:rgb(0,0,136);">From: "Bernhard Glomm" <<a href="#" class="tinebase-email-link" target="_blank">bernhard.glomm@ecologic.eu</a>><br>To:<span class="Apple-converted-space"> </span><a href="#" class="tinebase-email-link" target="_blank">pkarampu@redhat.com</a><br>Cc:<span class="Apple-converted-space"> </span><a href="#" class="tinebase-email-link" target="_blank">gluster-users@gluster.org</a><br>Sent: Thursday, January 16, 2014 5:58:17 PM<br>Subject: Re: [Gluster-users] gluster and kvm livemigration<br><br><br>hi Pranith<br><br># gluster volume info fs_vm_atom01<br> <br>Volume Name: fs_vm_atom01<br>Type: Replicate<br>Volume ID: fea9bdcf-783e-442a-831d-f564f8dbe551<br>Status: Started<br>Number of Bricks: 1 x 2 = 2<br>Transport-type: tcp<br>Bricks:<br>Brick1: 172.24.1.11:/zp_ping_1/fs_vm_atom01<br>Brick2: 172.24.1.13:/zp_pong_1/fs_vm_atom01<br>Options Reconfigured:<br>diagnostics.client-log-level: ERROR<br><br><br>TIA<br>Bernhard<br><br><br>On 16.01.2014 13:05:12, Pranith Kumar Karampuri wrote:<br><blockquote class="felamimail-body-blockquote" style="margin:5px 10px 0px 3px;padding-left:10px;border-left-width:2px;border-left-style:solid;border-left-color:rgb(0,0,136);">hi Bernhard,<br>Could you give gluster volume info output?<br><br>Pranith<br><br>----- Original Message -----<br><blockquote class="felamimail-body-blockquote" style="margin:5px 10px 0px 3px;padding-left:10px;border-left-width:2px;border-left-style:solid;border-left-color:rgb(0,0,136);">From: "Bernhard Glomm" <> ><span class="Apple-converted-space"> </span><a href="#" class="tinebase-email-link" target="_blank">bernhard.glomm@ecologic.eu</a>> > ><br>To: > ><span class="Apple-converted-space"> </span><a href="#" class="tinebase-email-link" target="_blank">gluster-users@gluster.org</a><br>Sent: Thursday, January 16, 2014 4:22:36 PM<br>Subject: [Gluster-users] gluster and kvm livemigration<br><br>I experienced a strange behavior of glusterfs during livemigration<br>of a qemu-kvm guest<br>using a 10GB file on a mirrored gluster 3.4.2 volume<br>(both on ubuntu 13.04)<br>I run<br>virsh migrate --verbose --live --unsafe --p2p --domain atom01 --desturi<br>qemu+ssh://<target_ip>/system<br>and the migration works,<br>the running machine is pingable and keeps sending pings.<br>nevertheless, when I let the machine touch a file during migration<br>it stops, complaining that it's filesystem is read only (from that moment<br>that<br>migration finished)<br>A reboot from inside the machine failes,<br>machine goes down and comes up with an error<br>unable to write to sector xxxxxx on hd0<br>(than falling into the initrd).<br>a<br>virsh destroy VM && virsh start VM<br>leads to a perfect running VM again,<br>no matter on which of the two hosts I start the machine<br>anybody better experience with livemigration?<br>any hint on a procedure how to debug that?<br>TIA<br>Bernhard<br><br>--<br><br><br>Bernhard Glomm<br>IT Administration<br><br>Phone: +49 (30) 86880 134<br>Fax: +49 (30) 86880 100<br>Skype: bernhard.glomm.ecologic<br><br>Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717<br>Berlin<br>| Germany<br>GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.:<br>DE811963464<br>Ecologic is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH<br><br><br>_______________________________________________<br>Gluster-users mailing list<br><a href="#" class="tinebase-email-link" target="_blank">Gluster-users@gluster.org</a><br><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></blockquote></blockquote></blockquote></blockquote></div></blockquote></div></div></blockquote></body></html>