<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I think we're basically talking about ODX.
<a class="moz-txt-link-freetext" href="http://www.google.com/patents/US20120102561">http://www.google.com/patents/US20120102561</a><br>
<div class="moz-cite-prefix">On 8/22/2014 7:29 AM, Giacomo Fazio
wrote:<br>
</div>
<blockquote
cite="mid:CAO0_EMLVr69vDePS2ukKyU2U_EJ20F-VFQkfS_0gqYaBCzk3+w@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>Hello Niels,<br>
<br>
</div>
Thanks for your explanation. I'm happy you consider my
proposal doable and that many ideas came to your mind. I would
like to contribute to it, but I really don't know anything
about the GlusterFS code and APIs, so I don't know where to
start and, in this condition, I think it is not an easy job to
do. I had a rapid look at the functions already existent in
the APIs, thinking that perhaps a low level copy function was
already present and I could use it (I saw that, for example,
the rename function uses this approach), but could not find
anything. <br>
Therefore I hope someone has the time and feels like
implementing this idea, I agree it would be a good
improvement, not just for me.<br>
<br>
</div>
Giacomo<br>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div dir="ltr">
<div
style="font-family:Arial,Helvetica,sans-serif;font-size:11px;color:rgb(96,96,96);line-height:14px"
align="left">
<div align="left">
<div align="left">
<div align="left"><b style="font-size:12px">Giacomo
Fazio</b><br>
IT Engineer<br>
<br>
Tel. +41 91 910 7690<br>
E-mail: <a moz-do-not-send="true"
href="mailto:giacomo.fazio@wcpmediaservices.com"
style="color:rgb(227,6,19);text-decoration:none"
target="_blank">giacomo.fazio@wcpmediaservices.com</a>
| Web: <a moz-do-not-send="true"
href="http://www.wcpmediaservices.com"
style="color:rgb(227,6,19);text-decoration:none"
target="_blank">www.wcpmediaservices.com</a><br>
<br>
<span style="color:rgb(227,6,19)">Europe Office:</span> Via
Zurigo 35, 6900 Lugano, Switzerland<br>
<span style="color:rgb(227,6,19)">USA Office:</span> 7083
Hollywood Boulevard Los Angeles, CA 90028</div>
<div style="color:rgb(0,0,0);font-family:'Times New
Roman';font-size:medium;line-height:normal;margin-top:10px">
<img moz-do-not-send="true"
src="http://wcpapp.wcpmediaservices.net/images/firma-wcpmediaservices.png"
alt="" height="130" width="380"></div>
</div>
</div>
</div>
</div>
</div>
<br>
<br>
<div class="gmail_quote">On Fri, Aug 22, 2014 at 3:27 PM, Niels
de Vos <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:ndevos@redhat.com" target="_blank">ndevos@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="">On Fri, Aug 22, 2014 at 01:26:02PM +0200,
Giacomo Fazio wrote:<br>
> Hi there,<br>
><br>
> Thanks to both Soumya and Prashanth. Actually you are
both right. With the<br>
> approach proposed by Soumya I would avoid the FUSE
overhead but, as<br>
> Prashanth says, the network transfer overhead would
be always present. This<br>
> is particularly important for me because I deal with
very big files<br>
> (usually around 100 GB and even more), so that
network transfer have a big<br>
> impact, while I don't think the impact of the FUSE
overhead is that big.<br>
> That's why what I would like to get is a "brick to
brick" copy (just server<br>
> side), so I would like to use the APIs to order the
server to make a copy,<br>
> so that the network transfer can be avoided.<br>
><br>
> As far as I understood, it is not currently possible
with libgfapi. Do you<br>
> think it would be difficult to implement? Are there
any other ways?<br>
> Thank you and best regards,<br>
<br>
</div>
"Difficult" is always relative, it depends on many factors
:) But<br>
I think implementing server-side copy is quite doable. You
should start<br>
with thinking of, and proposing a design. Some ideas that
would work:<br>
<br>
a.<br>
Have a server-side daemon (maybe glusterd) handle the
copy. Some new<br>
libgfapi or gluster-cli function can then connect to the
daemon and<br>
pass the instruction on (src brick + dst volume +
src+dst filename).<br>
This daemon can then connect to it's instance on the
server hosting<br>
the source-brick, and initiate the copy.<br>
<br>
b.<br>
Add a new file operation to the GlusterFS protocol,
something like<br>
copy-to-brick. This operation would receive the request
from the<br>
client (the client talks to the src-brick hosting the
src-file as<br>
usual), and the brick process needs to learn how to
connect to an<br>
other brick (from a different volume) and create/write
the file<br>
there. The client application should be smart enough to
pass the<br>
path to the dst-brick that should contain the dst-file.<br>
<br>
While writing this, I have convinced myself that (a) would
surely be<br>
easier to do. GlusterD could spawn a special copy process
(like<br>
a libgfapi client) that connects to the source and
destination volumes,<br>
do the copy, and exit.<br>
<br>
This also makes it much easier to start contributing!<br>
<br>
1.<br>
A relatively simple libgfapi binary that implements "cp"
with<br>
volume:/path/to/file as parameters should not be too
difficult. Of<br>
course, you may need to mkdir the (parent) structure on
the<br>
destination too, possibly adding a "-r" option for
recursive<br>
copying.<br>
<br>
2.<br>
A second step could then integrate this cp/libgfapi
implementation<br>
in some gluster-cli/glusterd procedures.<br>
<br>
3.<br>
Making it smarter and initiate the copy from one of the
source<br>
bricks, can then be an other step.<br>
<br>
<br>
For (1), it could be easier to extend some available
copy-tool.<br>
Something like rsync already supports different protocols.
Maybe it is<br>
possible to teach rsync how and what functions to call from
libgfapi.<br>
rsync supports many useful options already, writing a new
cp/libgfapi<br>
from scratch that matches only a subset from the features
that rsync<br>
has, will be a major project.<br>
<br>
The above are just some ideas, thinking out loud... But,
starting with<br>
integrating libgfapi in rsync or similar sounds like a major
usability<br>
improvement for many Gluster users.<br>
<br>
Niels<br>
<div class=""><br>
<br>
><br>
> *Giacomo Fazio*<br>
> IT Engineer<br>
><br>
> Tel. <a moz-do-not-send="true"
href="tel:%2B41%2091%20910%207690" value="+41919107690">+41
91 910 7690</a><br>
</div>
> E-mail: <a moz-do-not-send="true"
href="mailto:giacomo.fazio@wcpmediaservices.com">giacomo.fazio@wcpmediaservices.com</a>
| Web: <a moz-do-not-send="true"
href="http://www.wcpmediaservices.com" target="_blank">www.wcpmediaservices.com</a><br>
<div class="im HOEnZb">><br>
> Europe Office: Via Zurigo 35, 6900 Lugano,
Switzerland<br>
> USA Office: 7083 Hollywood Boulevard Los Angeles, CA
90028<br>
><br>
><br>
</div>
<div class="HOEnZb">
<div class="h5">> On Fri, Aug 22, 2014 at 9:36 AM,
Prashanth Pai <<a moz-do-not-send="true"
href="mailto:ppai@redhat.com">ppai@redhat.com</a>>
wrote:<br>
><br>
> > Hi,<br>
> ><br>
> > Even with that approach, data would still be
read (over the n/w) at the<br>
> > client (the app using libgfapi). I think what
he is looking for is a server<br>
> > side copy (brick to brick) or within same
brick _without_ the need for data<br>
> > to go through client.<br>
> ><br>
> > Swift has this feature[1] and it would be
really cool for glusterfs to<br>
> > have it (may be as an external tool or as a
API in libgfapi) :)<br>
> ><br>
> > # gluster-copy <src> <dest><br>
> > or<br>
> > glfs_copy(src,dest)<br>
> ><br>
> > [1]<br>
> > <a moz-do-not-send="true"
href="http://programmerthoughts.com/openstack/server-side-object-copy-in-openstack-storage/"
target="_blank">http://programmerthoughts.com/openstack/server-side-object-copy-in-openstack-storage/</a><br>
> ><br>
> ><br>
> ><br>
> > Regards,<br>
> > -Prashanth Pai<br>
> ><br>
> > ----- Original Message -----<br>
> > From: "Soumya Koduri" <<a
moz-do-not-send="true"
href="mailto:skoduri@redhat.com">skoduri@redhat.com</a>><br>
> > To: "Giacomo Fazio" <<a
moz-do-not-send="true"
href="mailto:giacomo.fazio@wcpmediaservices.com">giacomo.fazio@wcpmediaservices.com</a>>,
"John Mark<br>
> > Walker" <<a moz-do-not-send="true"
href="mailto:johnmark@gluster.org">johnmark@gluster.org</a>><br>
> > Cc: <a moz-do-not-send="true"
href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a>,
"Giovanni Contri" <<br>
> > <a moz-do-not-send="true"
href="mailto:giovanni.contri@wcpmediaservices.com">giovanni.contri@wcpmediaservices.com</a>>,
<a moz-do-not-send="true"
href="mailto:forge-admin@gluster.org">forge-admin@gluster.org</a><br>
> > Sent: Friday, August 22, 2014 12:40:01 PM<br>
> > Subject: Re: [Gluster-devel] Question about
file copy through libgfapi<br>
> ><br>
> > Hi Giacomo,<br>
> ><br>
> > If your requirement is to get away with
fuse/protocol clients and do<br>
> > server-side operations, I think its doable by
writing a simple libgfapi<br>
> > application. But since there is no libgfapi
API equivalent to "cp"<br>
> > command, you may need to implement that
functionality using "glfs_open,<br>
> > glfs_read & glfs_write" APIs.<br>
> ><br>
> > Here are the few links which Humble has
documented on how to use<br>
> > libgfapi and different APIs supported by it-<br>
> ><br>
> > <a moz-do-not-send="true"
href="http://humblec.com/libgfapi-interface-glusterfs/"
target="_blank">http://humblec.com/libgfapi-interface-glusterfs/</a><br>
> > <a moz-do-not-send="true"
href="https://github.com/gluster/glusterfs/blob/master/doc/features/libgfapi.md"
target="_blank">https://github.com/gluster/glusterfs/blob/master/doc/features/libgfapi.md</a><br>
> ><br>
> ><br>
> > Few sample examples (written in 'C' and
'python') are copied to -<br>
> > <a moz-do-not-send="true"
href="https://github.com/gluster/glusterfs/tree/master/api/examples"
target="_blank">https://github.com/gluster/glusterfs/tree/master/api/examples</a><br>
> ><br>
> ><br>
> > Thanks,<br>
> > Soumya<br>
> ><br>
> ><br>
> ><br>
> > On 08/21/2014 08:45 PM, Giacomo Fazio wrote:<br>
> > > Hi John,<br>
> > ><br>
> > > Thanks for your quick answer. Do you mean
that my question can be<br>
> > > summarized in "can we do server-only
operations?"? Yes, I think so.<br>
> > > Please let me know as soon as you receive
any answer or provide me a<br>
> > > link where I can follow directly this
case.<br>
> > > Thanks in advance and best regards,<br>
> > ><br>
> > > *Giacomo Fazio*<br>
> > > IT Engineer<br>
> > ><br>
> > > Tel. <a moz-do-not-send="true"
href="tel:%2B41%2091%20910%207690"
value="+41919107690">+41 91 910 7690</a><br>
> > > E-mail:Â <a moz-do-not-send="true"
href="mailto:giacomo.fazio@wcpmediaservices.com">giacomo.fazio@wcpmediaservices.com</a><br>
> > > <mailto:<a moz-do-not-send="true"
href="mailto:giacomo.fazio@wcpmediaservices.com">giacomo.fazio@wcpmediaservices.com</a>>Â
|Â Â Web:Â<br>
> > > <a moz-do-not-send="true"
href="http://www.wcpmediaservices.com" target="_blank">www.wcpmediaservices.com</a>
<<a moz-do-not-send="true"
href="http://www.wcpmediaservices.com" target="_blank">http://www.wcpmediaservices.com</a>><br>
> > ><br>
> > > Europe Office:Â Via Zurigo 35, 6900
Lugano, Switzerland<br>
> > > USA Office:Â 7083 Hollywood Boulevard Los
Angeles, CA 90028<br>
> > ><br>
> > ><br>
> > > On Thu, Aug 21, 2014 at 5:04 PM, John
Mark Walker <<a moz-do-not-send="true"
href="mailto:johnmark@gluster.org">johnmark@gluster.org</a><br>
> > > <mailto:<a moz-do-not-send="true"
href="mailto:johnmark@gluster.org">johnmark@gluster.org</a>>>
wrote:<br>
> > ><br>
> > > Thanks, Giacomo. I'm sending this to
the gluster-devel list - it's<br>
> > > an interesting question. Basically,
can we do server-only operations?<br>
> > ><br>
> > > -JM<br>
> > ><br>
> > ><br>
> > ><br>
> >
------------------------------------------------------------------------<br>
> > ><br>
> > > Hello,<br>
> > ><br>
> > > I am currently using GlusterFS
version 3.5 with two bricks. What<br>
> > > I currently do is mounting the
whole storage in some Linux<br>
> > > clients (RedHat) through
fuse.glusterfs that (I think) uses NFS<br>
> > > in the background.<br>
> > > What I would like to do is
copying a file from a directory to<br>
> > > another one in the storage in the
quickest way. Using a "cp<br>
> > > file1 file2" from my RedHat
client is not the best option<br>
> > > because the data flows from the
storage to my RedHat client<br>
> > > through the network and then back
to the storage. I would like<br>
> > > instead to avoid this waste of
time and copy the file directly<br>
> > > from the 1st directory to the 2nd
one. So, in a nutshell, I<br>
> > > would like to have file1 ->
file2Â , instead of file1 -><br>
> > > RedHatclient -> file2<br>
> > > Do you think is it possible, for
example using libgfapi? Any<br>
> > > example to show me?<br>
> > > Thank you in advance and best
regards,<br>
> > ><br>
> > > *Giacomo Fazio*<br>
> > > IT Engineer<br>
> > ><br>
> > > Tel. +41 91 910 7690
<a class="moz-txt-link-rfc2396E" href="tel:%2B41%2091%20910%207690"><tel:%2B41%2091%20910%207690></a><br>
> > > E-mail:Â <a
moz-do-not-send="true"
href="mailto:giacomo.fazio@wcpmediaservices.com">giacomo.fazio@wcpmediaservices.com</a><br>
> > > <mailto:<a
moz-do-not-send="true"
href="mailto:giacomo.fazio@wcpmediaservices.com">giacomo.fazio@wcpmediaservices.com</a>>Â
|Â Â Web:Â<br>
> > > <a moz-do-not-send="true"
href="http://www.wcpmediaservices.com" target="_blank">www.wcpmediaservices.com</a>
<<a moz-do-not-send="true"
href="http://www.wcpmediaservices.com" target="_blank">http://www.wcpmediaservices.com</a>><br>
> > ><br>
> > > Europe Office:Â Via Zurigo 35,
6900 Lugano, Switzerland<br>
> > > USA Office:Â 7083 Hollywood
Boulevard Los Angeles, CA 90028<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > >
_______________________________________________<br>
> > > Gluster-devel mailing list<br>
> > > <a moz-do-not-send="true"
href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
> > > <a moz-do-not-send="true"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-devel</a><br>
> > ><br>
> >
_______________________________________________<br>
> > Gluster-devel mailing list<br>
> > <a moz-do-not-send="true"
href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
> > <a moz-do-not-send="true"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-devel</a><br>
> ><br>
<br>
> _______________________________________________<br>
> Gluster-devel mailing list<br>
> <a moz-do-not-send="true"
href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
> <a moz-do-not-send="true"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-devel</a><br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel">http://supercolony.gluster.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
<br>
</body>
</html>