<div dir="ltr">Hi fog,<br><div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_quote">On Mon, Apr 29, 2013 at 5:03 PM, fog - <span dir="ltr"><<a href="mailto:fog_is_my_name@hotmail.com" target="_blank">fog_is_my_name@hotmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr">Hello Krish,<br><br>yes, no deadlock occurs without blocking (... somewhat obviously). However, if I can't block I do not gain anything regarding code readability. It makes more sense to use the standard STACK_WIND / UNWIND pairs instead of creating a syncthread with a callback function.<br>
</div></div></blockquote><div><br></div><div>This comment made me think what are the advantages of syncop framework over STACK_WIND/UNWIND framework (even when you are using synctask_new in non-blocking mode by providing a cbk). After discussion with Avati, it became clear that the actual benefit is not really whether synctask_new is blocking or non-blocking. It is actually that you can do synchronous operations in your synctask (or to put it crudely in your function that you pass as an argument to synctask). Some of these operations are already implemented in the form of synchronous fops (like syncop_lookup etc). But the core benefit of syncop framework is that you can "yield" (from executing your task's code to execute a different task's code) and "wake" the thread at any point in your code. This more fine grained "control" (which is basically continuations) is lacking in STACK_WIND/UNWIND framework.<br>
<br></div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="ltr"><br>I could use this approach if I start the syncthread in the FOP instead of the CBK function (and use syncops for everything). The problem is that in my scenario only the return of the FOP will tell me if additional FOPs need to be executed (and 99%+ of the time this won't be the case). This makes spawning a syncthread every time sound like a bad idea. <br>
<br>I think you identified the occurring deadlock in your other reply correctly by the way. Seems it's a bit more complicated to use syncops correctly than I originally assumed, I'll probably go back to STACK_WIND / UNWIND chains even if the resulting code is quite messy. <br>
<br>Thanks for your insight <br><span class="HOEnZb"><font color="#888888"> ~fog <br></font></span><div class="hm HOEnZb"><br></div><div><div class="hm HOEnZb"><div></div><hr>Date: Mon, 29 Apr 2013 14:30:50 +0530<br>From: <a href="mailto:kparthas@redhat.com" target="_blank">kparthas@redhat.com</a><br>
To: <a href="mailto:fog_is_my_name@hotmail.com" target="_blank">fog_is_my_name@hotmail.com</a><br>CC: <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a></div><div><div class="h5"><br>Subject: Re: [Gluster-devel] rpc problems when using syncops in callbacks<br>
<br>
<div>Fog,<br>
<br>
On 04/29/2013 01:57 PM, fog - wrote:<br>
</div>
<blockquote>
<div dir="ltr">Hello Avati,<br>
<br>
I am wrapping the syncop call in a synctask_new (otherwise
glusterFS will run into a null pointer @ synctask_get in the
SYNCOP macro & crash). Below is some code to show how I do
it currently to test the syncops. <br>
<br>
typedef struct{<br>
xlator_t *this; loc_t *loc; dict_t *dic;<br>
}syncstore_args;<br>
<br>
int32_t __xattr_store_sync(void* data)<br>
{<br>
syncstore_args *args = (syncstore_args*)data;<br>
return syncop_setxattr(FIRST_CHILD(args->this),
args->loc, args->dic, 0);<br>
}<br>
<br>
int32_t xattr_store_sync(xlator_t *this, call_frame_t *frame,
loc_t *loc, dict_t *dic)<br>
{<br>
syncstore_args args = {this, loc, dic};<br>
return synctask_new(this->ctx->env,
__xattr_store_sync, NULL, NULL, &args);<br>
</div>
</blockquote>
If you don't provide a synctask_cbk_t to synctask_new, you are using
synctask in a 'blocking' mode.<br>
That is, the thread calling synctask_new would block until the
synctask_fn_t function (ie, __xattr_store_sync) returns.<br>
An alternative way to do this would be,<br>
<br>
int32_t xattr_store_sync(xlator_t *this, call_frame_t *frame, loc_t
*loc, dict_t *dic)<br>
{<br>
syncstore_args args = {this, loc, dic};<br>
return synctask_new(this->ctx->env, __xattr_store_sync,
__xattr_store_sync_cbk, NULL, &args);<br>
}<br>
<br>
int32_t __xattr_store_sync_cbk (int ret, /*and the other args*/) <br>
{<br>
// Your code goes here<br>
return ret;<br>
}<br>
<br>
Now, all file operations performed using syncop_* inside
__xattr_store_sync would have the synchronous flavour, while leaving
the calling thread (thread calling xattr_store_sync fn) 'free'. This
should avoid the hang issue.<br>
<br>
HTH,<br>
krish<br>
<br>
<br>
<blockquote>
<div dir="ltr">}<br>
<br>
<div>
<hr>Date: Mon, 29 Apr 2013 00:19:11 -0700<br>
Subject: Re: [Gluster-devel] rpc problems when using syncops
in callbacks<br>
From: <a href="mailto:anand.avati@gmail.com" target="_blank">anand.avati@gmail.com</a><br>
To: <a href="mailto:fog_is_my_name@hotmail.com" target="_blank">fog_is_my_name@hotmail.com</a><br>
CC: <a href="mailto:gluster-devel@nongnu.org" target="_blank">gluster-devel@nongnu.org</a><br>
<br>
<div dir="ltr">Note that you need to place your syncop code in
a synctask function strictly within a syncenv (by calling
synctask_new(). You're probably calling syncop_XXX()
directly in your xlator code?
<div><br>
</div>
<div>Avati</div>
</div>
<div><br>
<br>
<div>On Fri, Apr 26, 2013 at 2:40 AM,
fog - <span dir="ltr"><<a href="mailto:fog_is_my_name@hotmail.com" target="_blank">fog_is_my_name@hotmail.com</a>></span>
wrote:<br>
<blockquote style="border-left:1px #ccc solid;padding-left:1ex">
<div>
<div dir="ltr">Hello everyone,<br>
<div>
<div dir="ltr">
<div>
<div><br>
</div>
I am trying to use syncops in a custom
translator to keep my code at least borderline
readable, but I am having limited success. <br>
<br>
Problem Symptoms: <br>
Using a syncop in a regular fop is fine.
However, in a callback it causes a 'freeze'
(synctask_yield called by the SYNCOP macro
doesn't return). <br>
<br>
</div>
<div>What seems to be the Problem: <br>
</div>
<div>Looking at the traces, there is no
corresponding trace from rpc_clnt_reply_init
on the client to the trace from
rpcsvc_submit_generic on the server. In other
words, the rpc reply gets sent but isn't
correctly received. Obviously this is not
really a networking problem but something
else... I'd guess it's a deadlock somewhere on
the client?<br>
</div>
<div>From the point of the syncop call onwards
the client doesn't 'get' any rpc replies any
more (the next GlusterFS Handshake sent by the
client, which is received by the server and
replied to, leads to a disconnection
accordingly).<br>
</div>
<div><br>
</div>
<div>Again: This problem is only occurring when
calling a syncop from a callback function
inside my translator, if I call the same
syncop in a fop call it completes fine.<br>
<br>
I hope you can make sense out of the above
problem description. <br>
</div>
<div>Thanks for your time ~<br>
</div>
<br>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Gluster-devel mailing list
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
<br></div></div></div> </div></div>
<br>_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Raghavendra G<br>
</div></div></div>