Hi, Harsh:<div> I make some break in glusterd , and can gdb the qemu-kvm forked from libvirtd.</div><div><br></div><div>break in glusterd:</div><div><br><div><div>(gdb) i b</div><div>Num Type Disp Enb Address What</div>
<div>1 breakpoint keep y 0x00007f903ef1a0a0 in server_getspec at glusterd-handshake.c:122</div><div>2 breakpoint keep y 0x00000034f4607070 in rpcsvc_program_actor at rpcsvc.c:137</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>breakpoint already hit 2 times</div>
<div>3 breakpoint keep y 0x00007f903ef199f0 in glusterd_set_clnt_mgmt_program at glusterd-handshake.c:359</div><div>4 breakpoint keep y 0x00007f903ef1a0a0 in server_getspec at glusterd-handshake.c:122</div>
<div><br></div><div>in <span style="font-family:courier;font-size:10pt">rpcsvc_handle_rpc_call fun, it call </span><span style="font-family:courier;font-size:10pt">rpcsvc_program_actor and return right.</span></div><div><span style="font-family:courier;font-size:10pt"><div>
(gdb) p *actor</div><div>$13 = {procname = "GETSPEC", '\000' <repeats 24 times>, procnum = 2, actor = 0x7f903ef1a0a0 <server_getspec>, vector_sizer = 0, unprivileged = _gf_false}</div><div><br>
</div><div>but in </div><div><p style="font-size:10pt"><span style="color:#808080"> </span><span style="color:#000080;font-weight:bold">if</span><span style="color:#808080"> </span><span style="font-weight:bold">(</span><span style="color:#008080">0</span><span style="color:#808080"> </span><span style="font-weight:bold">==</span><span style="color:#808080"> </span>svc<span style="font-weight:bold">-></span>allow_insecure<span style="color:#808080"> </span><span style="font-weight:bold">&&</span><span style="color:#808080"> </span>unprivileged<span style="color:#808080"> </span><span style="font-weight:bold">&&</span><span style="color:#808080"> </span><span style="font-weight:bold">!</span>actor<span style="font-weight:bold">-></span>unprivileged<span style="font-weight:bold">)</span><span style="color:#808080"> </span><span style="font-weight:bold">{</span><span style="color:#808080"><br>
</span><span style="color:#0000ff;font-style:italic">/* Non-privileged user, fail request */</span><span style="color:#808080"><br> </span>gf_log<span style="color:#808080"> </span><span style="font-weight:bold">(</span><span style="color:#cc0000">"glusterd"</span><span style="font-weight:bold">,</span><span style="color:#808080"> </span>GF_LOG_ERROR<span style="font-weight:bold">,</span><span style="color:#808080"><br>
</span><span style="color:#cc0000">"Request received from non-"</span><span style="color:#808080"><br> </span><span style="color:#cc0000">"privileged port. Failing request"</span><span style="font-weight:bold">);</span><span style="color:#808080"><br>
</span>rpcsvc_request_destroy<span style="color:#808080"> </span><span style="font-weight:bold">(</span>req<span style="font-weight:bold">);</span><span style="color:#808080"><br> </span><span style="color:#000080;font-weight:bold">return</span><span style="color:#808080"> </span><span style="font-weight:bold">-</span><span style="color:#008080">1</span><span style="font-weight:bold">;</span><span style="color:#808080"><br>
</span><span style="font-weight:bold">}</span></p></div></span></div><div>so the <span style="font-family:courier;font-size:13.63636302947998px">server_getspec on server not be called, which cause qemu-kvm progress failed.</span></div>
<div><span style="font-family:courier;font-size:13.63636302947998px"><br></span></div><div><span style="font-family:courier;font-size:13.63636302947998px">my question:</span></div><div><span style="font-family:courier;font-size:13.63636302947998px">1.</span><span style="font-family:courier;font-size:10pt;font-weight:bold">(</span><span style="font-family:courier;font-size:10pt;color:rgb(0,128,128)">0</span><span style="font-family:courier;font-size:10pt;color:rgb(128,128,128)"> </span><span style="font-family:courier;font-size:10pt;font-weight:bold">==</span><span style="font-family:courier;font-size:10pt;color:rgb(128,128,128)"> </span><span style="font-family:courier;font-size:10pt">svc</span><span style="font-family:courier;font-size:10pt;font-weight:bold">-></span><span style="font-family:courier;font-size:10pt">allow_insecure</span><span style="font-family:courier;font-size:10pt;color:rgb(128,128,128)"> </span><span style="font-family:courier;font-size:10pt;font-weight:bold">&&</span><span style="font-family:courier;font-size:10pt;color:rgb(128,128,128)"> </span><span style="font-family:courier;font-size:10pt">unprivileged</span><span style="font-family:courier;font-size:10pt;color:rgb(128,128,128)"> </span><span style="font-family:courier;font-size:10pt;font-weight:bold">&&</span><span style="font-family:courier;font-size:10pt;color:rgb(128,128,128)"> </span><span style="font-family:courier;font-size:10pt;font-weight:bold">!</span><span style="font-family:courier;font-size:10pt">actor</span><span style="font-family:courier;font-size:10pt;font-weight:bold">-></span><span style="font-family:courier;font-size:10pt">unprivileged</span><span style="font-family:courier;font-size:10pt;font-weight:bold">) which one wrong here ?</span><span style="font-family:courier;font-size:10pt"> </span></div>
<div><span style="font-family:courier;font-size:10pt"><br></span></div><div>Best Regards,</div><div>Yin Yin</div><div><br></div><div class="gmail_quote">On Thu, Aug 30, 2012 at 9:14 AM, Yin Yin <span dir="ltr"><<a href="mailto:maillistofyinyin@gmail.com" target="_blank">maillistofyinyin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi, Harsh:<div> I've try your patch, but can't boot the vm.<br><div>[root@yinyin qemu-glusterfs]# virsh create gluster-libvirt.xml </div>
<div>´íÎó£º´Ó gluster-libvirt.xml ´´½¨Óòʧ°Ü</div><div>´íÎó£ºUnable to read from monitor: Connection reset by peer</div>
<div><br></div><div>the libvirt build the qemu/gluster command correctly, the qemu-kvm try to run, but faile after a while, that cause the libvirt monitor connect failed.</div><div><br></div><div>the /var/libvirt/qemu/gluster-vm.log follow:</div>
<div> </div><div><div>2012-08-30 01:03:08.418+0000: starting up</div><div>LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name gluster-vm -uuid f65bd812-45fb-cc2d-75fd-84206248e026 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/gluster-vm.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=gluster://<a href="http://10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=native" target="_blank">10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=native</a> -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -device usb-tablet,id=input0 -spice port=30038,addr=0.0.0.0,disable-ticketing -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5</div>
<div>2012-08-30 01:03:08.423+0000: 4452: debug : virCommandHook:2041 : Run hook 0x48f160 0x7f433ba0e570</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : qemuProcessHook:2475 : Obtaining domain lock</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : virDomainLockManagerNew:123 : plugin=0x7f43300b7980 dom=0x7f43240022b0 withResources=1</div>
<div>2012-08-30 01:03:08.423+0000: 4452: debug : virLockManagerNew:291 : plugin=0x7f43300b7980 type=0 nparams=4 params=0x7f433ba0d9d0 flags=0</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : virLockManagerLogParams:98 : key=uuid type=uuid value=f65bd812-45fb-cc2d-75fd-84206248e026</div>
<div>2012-08-30 01:03:08.423+0000: 4452: debug : virLockManagerLogParams:94 : key=name type=string value=gluster-vm</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : virLockManagerLogParams:82 : key=id type=uint value=1</div>
<div>2012-08-30 01:03:08.423+0000: 4452: debug : virLockManagerLogParams:82 : key=pid type=uint value=4452</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : virDomainLockManagerNew:135 : Adding leases</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : virDomainLockManagerNew:140 : Adding disks</div>
<div>2012-08-30 01:03:08.423+0000: 4452: debug : virLockManagerAcquire:337 : lock=0x7f4324001ba0 state='(null)' flags=3 fd=0x7f433ba0db3c</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : virLockManagerFree:374 : lock=0x7f4324001ba0</div>
<div>2012-08-30 01:03:08.423+0000: 4452: debug : qemuProcessHook:2500 : Moving process to cgroup</div><div>2012-08-30 01:03:08.423+0000: 4452: debug : virCgroupNew:603 : New group /libvirt/qemu/gluster-vm</div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupDetect:262 : Detected mount/mapping 0:cpu at /cgroup/cpu in </div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupDetect:262 : Detected mount/mapping 1:cpuacct at /cgroup/cpuacct in </div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupDetect:262 : Detected mount/mapping 2:cpuset at /cgroup/cpuset in </div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupDetect:262 : Detected mount/mapping 3:memory at /cgroup/memory in </div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupDetect:262 : Detected mount/mapping 4:devices at /cgroup/devices in </div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupDetect:262 : Detected mount/mapping 5:freezer at /cgroup/freezer in </div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupDetect:262 : Detected mount/mapping 6:blkio at /cgroup/blkio in </div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:524 : Make group /libvirt/qemu/gluster-vm</div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:546 : Make controller /cgroup/cpu/libvirt/qemu/gluster-vm/</div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:546 : Make controller /cgroup/cpuacct/libvirt/qemu/gluster-vm/</div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:546 : Make controller /cgroup/cpuset/libvirt/qemu/gluster-vm/</div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:546 : Make controller /cgroup/memory/libvirt/qemu/gluster-vm/</div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:546 : Make controller /cgroup/devices/libvirt/qemu/gluster-vm/</div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:546 : Make controller /cgroup/freezer/libvirt/qemu/gluster-vm/</div><div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupMakeGroup:546 : Make controller /cgroup/blkio/libvirt/qemu/gluster-vm/</div>
<div>2012-08-30 01:03:08.424+0000: 4452: debug : virCgroupSetValueStr:320 : Set value '/cgroup/cpu/libvirt/qemu/gluster-vm/tasks' to '4452'</div><div>2012-08-30 01:03:08.426+0000: 4452: debug : virCgroupSetValueStr:320 : Set value '/cgroup/cpuacct/libvirt/qemu/gluster-vm/tasks' to '4452'</div>
<div>2012-08-30 01:03:08.429+0000: 4452: debug : virCgroupSetValueStr:320 : Set value '/cgroup/cpuset/libvirt/qemu/gluster-vm/tasks' to '4452'</div><div>2012-08-30 01:03:08.432+0000: 4452: debug : virCgroupSetValueStr:320 : Set value '/cgroup/memory/libvirt/qemu/gluster-vm/tasks' to '4452'</div>
<div>2012-08-30 01:03:08.435+0000: 4452: debug : virCgroupSetValueStr:320 : Set value '/cgroup/devices/libvirt/qemu/gluster-vm/tasks' to '4452'</div><div>2012-08-30 01:03:08.437+0000: 4452: debug : virCgroupSetValueStr:320 : Set value '/cgroup/freezer/libvirt/qemu/gluster-vm/tasks' to '4452'</div>
<div>2012-08-30 01:03:08.439+0000: 4452: debug : virCgroupSetValueStr:320 : Set value '/cgroup/blkio/libvirt/qemu/gluster-vm/tasks' to '4452'</div><div>2012-08-30 01:03:08.442+0000: 4452: debug : qemuProcessInitCpuAffinity:1731 : Setting CPU affinity</div>
<div>2012-08-30 01:03:08.443+0000: 4452: debug : qemuProcessInitCpuAffinity:1760 : Set CPU affinity with specified cpuset</div><div>2012-08-30 01:03:08.443+0000: 4452: debug : qemuProcessHook:2512 : Setting up security labelling</div>
<div>2012-08-30 01:03:08.443+0000: 4452: debug : virSecurityDACSetProcessLabel:637 : Dropping privileges of DEF to 107:107</div><div>2012-08-30 01:03:08.443+0000: 4452: debug : qemuProcessHook:2519 : Hook complete ret=0</div>
<div>2012-08-30 01:03:08.443+0000: 4452: debug : virCommandHook:2043 : Done hook 0</div><div>2012-08-30 01:03:08.443+0000: 4452: debug : virCommandHook:2056 : Notifying parent for handshake start on 24</div><div>2012-08-30 01:03:08.443+0000: 4452: debug : virCommandHook:2077 : Waiting on parent for handshake complete on 25</div>
<div>2012-08-30 01:03:08.495+0000: 4452: debug : virCommandHook:2093 : Hook is done 0</div><div>Gluster connection failed for server=10.1.81.111 port=24007 volume=dht image=windows7-32-DoubCards-iotest-qcow2.img transport=socket</div>
<div>qemu-kvm: -drive file=gluster://<a href="http://10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=native" target="_blank">10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=native</a>: could not open disk image gluster://<a href="http://10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img" target="_blank">10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img</a>: No data available</div>
<div>2012-08-30 01:03:11.565+0000: shutting down</div></div><div><br></div><div>I can boot the vm with the command:</div><div>LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name gluster-vm -uuid f65bd812-45fb-cc2d-75fd-84206248e026 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/gluster-vm.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=gluster://<a href="http://10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=native" target="_blank">10.1.81.111:24007/dht/windows7-32-DoubCards-iotest-qcow2.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=native</a> -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -device usb-tablet,id=input0 -spice port=30038,addr=0.0.0.0,disable-ticketing -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5</div>
<div><br></div><div>my question:</div><div>1.what's the libvirt hook funciton? could it affect the qemu-kvm command?</div><div>2.It's hard to debug the qemu-kvm progress from libvirt, I try to hang the glusterd for a moment, then to gdb the qemu-kvm, do your have better methods?</div>
<div><br></div><div>Best Regards,</div><div>Yin Yin</div><div><br></div><div>You have new mail in /var/spool/mail/root</div><div><div class="h5"><div class="gmail_quote">On Fri, Aug 24, 2012 at 5:44 PM, Deepak C Shetty <span dir="ltr"><<a href="mailto:deepakcs@linux.vnet.ibm.com" target="_blank">deepakcs@linux.vnet.ibm.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On 08/24/2012 12:22 PM, Harsh Bora wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 08/24/2012 12:05 PM, Daniel Veillard wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Thu, Aug 23, 2012 at 04:31:50PM +0530, Harsh Prateek Bora wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This patchset provides support for Gluster protocol based network disks.<br>
It is based on the proposed gluster support in Qemu on qemu-devel:<br>
<a href="http://lists.gnu.org/archive/html/qemu-devel/2012-08/msg01539.html" target="_blank">http://lists.gnu.org/archive/<u></u>html/qemu-devel/2012-08/<u></u>msg01539.html</a><br>
</blockquote>
<br>
Just to be clear, that qemu feature didn't make the deadline for 1.2,<br>
right ? I don't think we can add support at the libvirt level until<br>
the patches are commited in QEmu, but that doesn't prevent reviewing<br>
them in advance . Right now we are in freeze for 0.10.0,<br>
</blockquote>
<br>
</blockquote>
<br></div>
I am working on enabling oVirt/VDSM to be able to exploit this, using harsh's RFC patches.<br>
VDSM patch @ <a href="http://gerrit.ovirt.org/#/c/6856/" target="_blank">http://gerrit.ovirt.org/#/c/<u></u>6856/</a><br>
<br>
An early feedback would help me, especially on the xml spec posted here. My VDSM patch<br>
depends on it.<br>
<br>
thanx,<br>
deepak<div><div><br>
<br>
<br>
______________________________<u></u>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/<u></u>mailman/listinfo/gluster-devel</a><br>
</div></div></blockquote></div><br></div></div></div>
</blockquote></div><br></div></div>