<div dir="ltr">Hello,<div><br></div><div>I apologize for the delayed reply.</div><div><br></div><div>I've collected some logs and posted them here:</div><div><a href="https://gist.github.com/jtopjian/7981763">https://gist.github.com/jtopjian/7981763</a><br>
</div><div><br></div><div>I stopped the Gluster service on 192.168.1.11, moved /var/log/glusterfs to a backup, then started Gluster so that the log files were more succinct. </div><div><br></div><div>I then used the qemu-img command as mentioned before as root, which was successful. Then I ran the command as libvirt-qemu and let the command hang for 2 minutes before I killed it.</div>
<div><br></div><div>It might also be helpful to note that this is on an Ubuntu 12.04 server using the packages found here:</div><div><a href="https://launchpad.net/~semiosis/+archive/ubuntu-glusterfs-3.4">https://launchpad.net/~semiosis/+archive/ubuntu-glusterfs-3.4</a><br>
</div><div><br></div><div>Thanks,</div><div>Joe</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Dec 14, 2013 at 12:41 AM, Vijay Bellur <span dir="ltr"><<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 12/13/2013 10:58 AM, Joe Topjian wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
Hello,<br>
<br>
I'm having a problem getting remote servers to connect to Gluster with qemu.<br>
<br>
I have 5 servers, 4 of which run Gluster and host a volume. The qemu<br>
user on all 5 servers has the same uid.<br>
<br>
storage.owner-uid and storage.owner-gid is set to that user.<br>
<br>
In addition, server.allow-insecure is on and is also set in the<br>
glusterd.vol file. glusterd has also been restarted (numerous times).<br>
<br>
When attempting to create a qemu file by connecting to the same server,<br>
everything works:<br>
<br>
</div><a href="mailto:qemu@192.168.1.11" target="_blank">qemu@192.168.1.11</a> <mailto:<a href="mailto:qemu@192.168.1.11" target="_blank">qemu@192.168.1.11</a>>> qemu-img create<br>
gluster://<a href="http://192.168.1.11/volumes/v.img" target="_blank">192.168.1.11/<u></u>volumes/v.img</a> <<a href="http://192.168.1.11/volumes/v.img" target="_blank">http://192.168.1.11/volumes/<u></u>v.img</a>> 1M<br>
Formatting 'gluster://<a href="http://192.168.1.11/volumes/v.img" target="_blank">192.168.1.11/<u></u>volumes/v.img</a><br>
<<a href="http://192.168.1.11/volumes/v.img" target="_blank">http://192.168.1.11/volumes/<u></u>v.img</a>>', fmt=raw size=1048576<br>
<a href="mailto:qemu@192.168.1.11" target="_blank">qemu@192.168.1.11</a> <mailto:<a href="mailto:qemu@192.168.1.11" target="_blank">qemu@192.168.1.11</a>>><div class="im"><br>
<br>
But when trying to do it remotely, the command hangs indefinitely:<br>
<br>
</div><a href="mailto:qemu@192.168.1.12" target="_blank">qemu@192.168.1.12</a> <mailto:<a href="mailto:qemu@192.168.1.12" target="_blank">qemu@192.168.1.12</a>>> qemu-img create<br>
gluster://<a href="http://192.168.1.11/volumes/v.img" target="_blank">192.168.1.11/<u></u>volumes/v.img</a> <<a href="http://192.168.1.11/volumes/v.img" target="_blank">http://192.168.1.11/volumes/<u></u>v.img</a>> 1M<br>
Formatting 'gluster://<a href="http://192.168.1.11/volumes/v.img" target="_blank">192.168.1.11/<u></u>volumes/v.img</a><br>
<<a href="http://192.168.1.11/volumes/v.img" target="_blank">http://192.168.1.11/volumes/<u></u>v.img</a>>', fmt=raw size=1048576<div class="im"><br>
^C<br>
<br>
Yet when 192.168.1.12 connects to gluster://<a href="http://192.168.1.12" target="_blank">192.168.1.12</a><br></div>
<<a href="http://192.168.1.12" target="_blank">http://192.168.1.12</a>>, the command works and the file shows up in the<div class="im"><br>
distributed volume.<br>
<br>
Further, when turning server.allow-insecure off, I get an immediate<br>
error no matter what the source and destination connection is:<br>
<br>
</div><a href="mailto:qemu@192.168.1.12" target="_blank">qemu@192.168.1.12</a> <mailto:<a href="mailto:qemu@192.168.1.12" target="_blank">qemu@192.168.1.12</a>>> qemu-img create<br>
gluster://<a href="http://192.168.1.11/volumes/v.img" target="_blank">192.168.1.11/<u></u>volumes/v.img</a> <<a href="http://192.168.1.11/volumes/v.img" target="_blank">http://192.168.1.11/volumes/<u></u>v.img</a>> 1M<br>
Formatting 'gluster://<a href="http://192.168.1.11/volumes/v.img" target="_blank">192.168.1.11/<u></u>volumes/v.img</a><br>
<<a href="http://192.168.1.11/volumes/v.img" target="_blank">http://192.168.1.11/volumes/<u></u>v.img</a>>', fmt=raw size=1048576<div class="im"><br>
qemu-img: Gluster connection failed for server=192.168.1.11 port=0<br>
volume=volumes image=v.img transport=tcp<br>
qemu-img: gluster://<a href="http://192.168.1.11/volumes/v.img" target="_blank">192.168.1.11/<u></u>volumes/v.img</a><br></div>
<<a href="http://192.168.1.11/volumes/v.img" target="_blank">http://192.168.1.11/volumes/<u></u>v.img</a>>: error while creating raw: No data<div class="im"><br>
available<br>
<br>
Does anyone have any ideas how I can have an unprivileged user connect<br>
to remote gluster servers?<br>
<br>
</div></blockquote>
<br>
Can you please provide glusterd and glusterfsd logs from 192.168.1.11?<span class="HOEnZb"><font color="#888888"><br>
<br>
-Vijay<br>
<br>
</font></span></blockquote></div><br></div>