I figured out how to work around this but I'm not sure of the exact reason why it happened.<div><br></div><div>The Gluster bricks I was using were LVM LV partitions that sat on top of a software RAID1. I broke the software RAID and dedicated one hard drive to LVM in order for OpenStack to use it for nova-volumes. I then used the other drive strictly for the gluster brick.</div>
<div><br></div><div>This removed mdadm and LVM out of the equation and the problem went away. I then tried with just LVM and still did not see this problem. </div><div><br></div><div>Unfortunately I don't have enough hardware at the moment to create another RAID1 mirror, so I can't single that out. I will try when I get a chance -- unless anyone else knows if it would cause a problem? Or maybe it is the mdamd+LVM combination?</div>
<div><br></div><div>Thanks,</div><div>Joe</div><div><br><div class="gmail_quote">On Fri, Aug 24, 2012 at 3:17 PM, Joe Topjian <span dir="ltr"><<a href="mailto:joe@topjian.net" target="_blank">joe@topjian.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<div><br></div><div>I'm seeing a weird issue with OpenStack and Gluster. </div><div><br></div><div>I have /var/lib/nova/instances mounted as a glusterfs volume. The owner of /var/lib/nova/instances is nova:nova.</div>
<div><br></div><div>When I launch a vm and watch it launching, I see the following:</div><div><br></div><div><div>root@c01:/var/lib/nova/instances/instance-00000012# ls -l</div><div>total 8</div><div>-rw-rw---- 1 nova nova 0 Aug 24 14:22 console.log</div>
<div>-rw-rw-r-- 1 nova nova 1459 Aug 24 14:22 libvirt.xml</div></div><div><br></div><div>This is correct.</div><div><br></div><div>Then it changes ownership to libvirt-qemu:</div><div><br></div><div><div>root@c01:/var/lib/nova/instances/instance-00000012# ls -l</div>
<div>total 22556</div><div>-rw-rw---- 1 libvirt-qemu kvm 0 Aug 24 14:22 console.log</div><div>-rw-r--r-- 1 libvirt-qemu kvm 27262976 Aug 24 14:22 disk</div><div>-rw-rw-r-- 1 nova nova 1459 Aug 24 14:22 libvirt.xm</div>
</div><div><br></div><div>Again, this is correct.</div><div><br></div><div>But then it changes to root:</div><div><br></div><div><div>root@c01:/var/lib/nova/instances/instance-00000012# ls -l</div><div>total 22556</div><div>
-rw-rw---- 1 root root 0 Aug 24 14:22 console.log</div><div>-rw-r--r-- 1 root root 27262976 Aug 24 14:22 disk</div><div>-rw-rw-r-- 1 nova nova 1459 Aug 24 14:22 libvirt.xml</div></div><div><br></div><div>OpenStack then errors out due to not being able to correctly access the files.</div>
<div><br></div><div>If I remove the /var/lib/nova/instances mount and just use the normal filesystem, the root ownership part does not happen.</div><div><br></div><div>I have successfully had Gluster working with OpenStack in this way on a different installation, so I'm not sure why I'm seeing this issue now.</div>
<div><br></div><div>Any ideas?</div><div><br></div><div>Thanks,</div><div>Joe</div>
</blockquote></div><br></div>