<br><br><div class="gmail_quote">On Thu, Jun 27, 2013 at 2:30 PM, Deepak C Shetty <span dir="ltr"><<a href="mailto:deepakcs@linux.vnet.ibm.com" target="_blank">deepakcs@linux.vnet.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div><div class="h5">
<div>On 06/27/2013 12:34 PM, Kanagaraj M
wrote:<br>
</div>
<blockquote type="cite"><br>
<br>
<div class="gmail_quote">On Thu, Jun 27, 2013 at 11:20 AM, Deepak
C Shetty <span dir="ltr"><<a href="mailto:deepakcs@linux.vnet.ibm.com" target="_blank">deepakcs@linux.vnet.ibm.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 06/24/2013 06:21 PM, Vijay Bellur wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 06/20/2013 07:28 PM, M. Mohan Kumar wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Vijay Bellur <<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>>
writes:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 06/19/2013 09:51 PM, M. Mohan Kumar wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<br>
<br>
When qemu is invoked by a non-root user with -drive<br>
file=gluster://server/volname/imagename option,
unprivileged port is<br>
used for gluster rpc and by default glusterd and
gluster brick process<br>
deny the request if the request is from a unprivileged
port. The option<br>
"rpc-auth-allow-insecure" needs to be enabled in
glusterd.vol so that<br>
non privileged ports can be used to access Gluster
volumes.<br>
<br>
In a typical environment VDSM might want to enable
rpc-auth-allow-insecure<br>
option and the administrator has to edit the
glusterd.vol manually and<br>
restart glusterd process.<br>
<br>
CLI options available to enable volume specific
options to work with<br>
unprivileged ports by using gluster volume set
<volname> <option><br>
<value>. For example per volume
server.allow-insecure option can be<br>
enabled so that unprivileged users can mount a
GlusterFS volume.<br>
<br>
But as of now there is no CLI option available to set
glusterd.vol<br>
options. How about adding a gluster CLI set option to
configure<br>
glusterd.vol options? Can following CLI command line
'gluster volume set<br>
all <glusterd.option> <value>" be used for
setting glusterd options?<br>
IIUC "all" is a reserved volume name and we can use
this reserved name<br>
for setting glusterd option.<br>
</blockquote>
<br>
'volume set all' is mostly used for options that are
applicable to all<br>
volumes. Since glusterd options are beyond the scope of
a volume, tying<br>
them to the peer entity might be a good idea. We can
introduce 'peer set<br>
all <key> <value>' which sets a particular
option on all peers.<br>
<br>
</blockquote>
<br>
You mean by 'gluster peer set all rpc-auth-allow-insecure
on' will<br>
enable insecured port access to all glusterds in the peer
environment?<br>
</blockquote>
<br>
<br>
Yes.<br>
</blockquote>
<br>
This still doesn't help the VDSM usecase, when VDSM host ( aka
hypervisor host ) is not part of gluster peer.<br>
One of the goal here was to provide a cli way to modify
glusterd options so that VDSM can exploit it, when Gluster
volume is used as a storage domain, and VDSM needs
rpc-auth-allow-insecure to be ON as VMs accessing Gluster
volume natively via libgfapi will be running as non-root.<br>
<br>
On the same lines.. how does oVirt Engine 'Volumes' GUI manage
Gluster volumes.... when the oVirt host is not part of the
Gluster peer ? Just wondering....<br>
<br>
</blockquote>
<div><br>
</div>
<div>Why oVirt Engine host needs to be a gluster peer to be able
to communicate with gluster(through vdsm)? As super-vdsm is
running as root in the gluster node and it should be able to
communicate with underlying gluster cli and can respond to
oVirt engine.</div>
</div>
</blockquote>
<br></div></div>
Right, I forgot the fact that gluster clis are executed thru
super-vdsm, so non-priv / insecure issue won't be present. But the
VDSM usecase still is a issue... when VDSM host (hypervisor node) is
not a gluster peer, we need a way to set the glusterd option...
currently one has to do that manually, which is not a good choice.<div class="im"><br>
<br>
<blockquote type="cite">
<div class="gmail_quote">
<div><br>
</div>
<div>If not, are you talking about the scenario where gluster
node doesn't have vdsm installed also the storage cluster is
not managed through oVirt.</div>
</div>
</blockquote>
<br></div>
So, as i see it, there are couple of scenarios.... from VDSM
perspective...<br>
<br>
For the sake of discussion, lets say <b>VDSM (aka hypervisor host)
is hostA & Gluster node/host is hostB</b><b>.</b><br>
<br>
<b>Case 1:</b> hostA is not a gluster peer. hostB is a gluster peer
(obviously) , part of Gluster volume which is managed by oVirt
Engine.<br>
<br>
hostA will access Gluster volume as non-root (as part of VM
running on Gluster storage domain, using libgfapi). Since hostA is
not a gluster peer but since the Gluster volume is managed by oVirt,
it should be possible for the engine to first execute 'gluster peer
set all rpc-auth-allow-insecure on the gluster volume, then ask VDSM
to create the gluster storage domain. From the perspective of "User
friendliness", its better to have User click on something like
"Allow insecure access" or "Make this gluster volume work with VDSM
Gluster storage domain" which will execute 'gluster peer set all
..insecure.. ' option, so that User is aware of enabling the
insecure option, instead of engine doing it w/o User's knowledge as
part of gluster storage domain flow.<br>
<br>
This should work and for this to happen seamless, the CLI support
for setting glusterd is needed, so this usecase is covered.<br>
<br>
<b>Case 2:</b> hostA is not a gluster peer. hostB is a gluster peer
(obviously), but the Gluster volume is not managed by oVirt.<br>
<br>
The same as Case 1 applies, with the exception that, since the
Gluster volume is not managed by oVirt, engine cannot do 'gluster
peer set all rpc-auth-allow-insecure'. Ther are again 2 sub-cases
here ...<br>
<br>
<b>2a)</b> Have the user import the gluster volume into oVirt,
post which it becomes like Case 1... problem solved.<br>
<br>
<b>2b)</b> For some reason, user doesn't want to import the
gluster volume into oVirt.. in which case, the only way is for
someone to _manually_ do 'gluster peer set all
rpc-auth-allow-insecure' from one of the gluster peers, which is not
a good thing since it involved manual intervention. One can argue
that hostA can use --remote-host option and do 'gluster peer set
all...', which will work today, but that would be insecure way of
doing it, since you are allowing a non-peer to set insecure option
for all peers. Also Mohan's another patch
(<a href="http://review.gluster.org/#/c/5092/" target="_blank">http://review.gluster.org/#/c/5092/</a>) adds support for IP based
access control, which also disables any 'set' kind of actions using
--remote-host, so --remote-host anyways won't be possible in VDSM
once that patch is thru. Net, there is no way in VDSM to enable
glusterd insecure option for this usecase.<br>
<br>
<b>Case 3: </b>hostA is a gluster peer. hostB is a gluster peer
(obviously) , part of Gluster volume which is managed by oVirt
Engine.<br>
<br>
Since oVirt is managing is gluster volume, the same as Case 1
applies<br>
<b><br>
</b><b>Case 4: </b>hostA is a gluster peer. hostB is a gluster peer
(obviously) , but the Gluster volume is not managed by oVirt. <br>
<br>
(Someone manually set up the Gluster volume, using hypervisor
hosts as peers, for using the un-used disk space on hyp host for
storage)<br>
This is similar to Case 2, but since hostA and hostB are part of
gluster peer, VDSM can do 'gluster peer set all ...insecure..'
before creating storage domain. Problem happens, when new hosts are
added to the datacentre which are not part of gluster peer. Note in
a virt env. hosts can be dynamically added / remove. So when a VM
backed by gluster storage domain migrates to a host that is not a
gluster peer, this usecase is broken like in Case 2b<br></div></blockquote><div><br></div><div><br></div><div>I am not much familiar with storage domain migrations, please correct me in case i am wrong.</div><div><br>
</div><div>From the above case 4, i assume </div><div> </div><div>- cluster1 is having hostA and hostB</div><div>- SD is created on hostA</div><div>- a new cluster cluster2 is created and hostC is added to it</div><div>- SD is moved to hostC</div>
<div><br></div><div>in this hostC is not a peer. this is same as <b>case 2b. </b>Also if a new peer is added to cluster1, insecure option will be automatically set in the new node as well AFAIK. </div><div><br></div><div>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<br>
<b>To Summarize:</b><br>
<br>
1) 'gluster peer set all ...insecure...' cli option will definitely
help and it will help more if the oVirt Volumes tab provides an UI
way of invoking this on User request. This will cover some usecases
as desc above<br>
<br>
2) Case 2b & 4 - how to handle this scenario ?<br>
<br>
thanx,<br>
deepak<div class="im"><br>
<br>
<br>
<blockquote type="cite">
<div class="gmail_quote">
<div><br>
</div>
<div>Thanks,</div>
<div>Kanagaraj</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
thanx,<br>
deepak<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
IIUC <a href="http://glusterd.info" target="_blank">glusterd.info</a>
file can be used to store about these parameters<br>
similar to how volume specific options are stored in
vols/<volname>/info<br>
file?<br>
<br>
</blockquote>
<br>
We can persist this in glusterd.vol referred by the
respective glusterd<br>
instance.<br>
</blockquote>
<br>
So glusterd.vol is not [re]generated during glusterd init?<br>
</blockquote>
<br>
<br>
No, glusterd.vol does not get altered during init.<br>
<br>
-Vijay<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
-Vijay<br>
</blockquote>
<br>
<br>
<br>
</blockquote>
<br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br>
<br>
<br>
</blockquote>
<br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
</blockquote>
</div>
<br>
</blockquote>
<br>
</div></div>
</blockquote></div><br>