<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Fog,<br>
<br>
I hope I have caught your attention before you have decided to
start over using STACK_WIND/STACK_UNWIND (async way)<br>
macros and drop the syncop approach.<br>
<br>
<br>
On 04/29/2013 05:03 PM, fog - wrote:<br>
</div>
<blockquote cite="mid:BAY171-W5970E7E6DDF32DC1086950F7B20@phx.gbl"
type="cite">
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style>
<div dir="ltr">Hello Krish,<br>
<br>
yes, no deadlock occurs without blocking (... somewhat
obviously). However, if I can't block I do not gain anything
regarding code readability. It makes more sense to use the
standard STACK_WIND / UNWIND pairs instead of creating a
syncthread with a callback function. <br>
<br>
I could use this approach if I start the syncthread in the FOP
instead of the CBK function (and use syncops for everything).
The problem is that in my scenario only the return of the FOP
will tell me if additional FOPs need to be executed (and 99%+ of
the time this won't be the case). This makes spawning a
syncthread every time sound like a bad idea. <br>
</div>
</blockquote>
Making the fop use the synctask framework DOESN'T mean you are going
to spawn a new thread everytime!It actually means you are scheduling
another task into the synctask environment. All you need to ensure
is that you don't 'yield' while on the epoll thread.<br>
You could take a look at how mgmt/glusterd[1] xlator uses the
synctask framework to provide synchronous wrappers to the mgmt
operations. In glusterd, the rpc programs have been marked for using
synctask at the rpcsvc layer. What this means is, that each rpc
request would be run in a synctask and the epoll thread returns to
listening for newer network events. In this approach, you have the
guarantee that epoll is not held hostage when you yield (never!).
And all your code looks pretty and synchronous.<br>
<br>
This approach may not work too well for you. But I am just saying
how I got across the synctask 'barrier' (pun not intended) while
making glusterd's mgmt operations (appear) synchronous.<br>
<br>
[1] - xlators/mgmt/glusterd/src/glusterd-syncop.c<br>
<br>
HTH,<br>
krish<br>
<blockquote cite="mid:BAY171-W5970E7E6DDF32DC1086950F7B20@phx.gbl"
type="cite">
<div dir="ltr"><br>
I think you identified the occurring deadlock in your other
reply correctly by the way. Seems it's a bit more complicated to
use syncops correctly than I originally assumed, I'll probably
go back to STACK_WIND / UNWIND chains even if the resulting code
is quite messy. <br>
<br>
Thanks for your insight <br>
~fog <br>
<br>
<div>
<hr id="stopSpelling">Date: Mon, 29 Apr 2013 14:30:50 +0530<br>
From: <a class="moz-txt-link-abbreviated" href="mailto:kparthas@redhat.com">kparthas@redhat.com</a><br>
To: <a class="moz-txt-link-abbreviated" href="mailto:fog_is_my_name@hotmail.com">fog_is_my_name@hotmail.com</a><br>
CC: <a class="moz-txt-link-abbreviated" href="mailto:gluster-devel@nongnu.org">gluster-devel@nongnu.org</a><br>
Subject: Re: [Gluster-devel] rpc problems when using syncops
in callbacks<br>
<br>
<div class="ecxmoz-cite-prefix">Fog,<br>
<br>
On 04/29/2013 01:57 PM, fog - wrote:<br>
</div>
<blockquote
cite="mid:BAY171-W930585B6245B591348F89EF7B20@phx.gbl">
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">Hello Avati,<br>
<br>
I am wrapping the syncop call in a synctask_new (otherwise
glusterFS will run into a null pointer @ synctask_get in
the SYNCOP macro & crash). Below is some code to show
how I do it currently to test the syncops. <br>
<br>
typedef struct{<br>
xlator_t *this; loc_t *loc; dict_t *dic;<br>
}syncstore_args;<br>
<br>
int32_t __xattr_store_sync(void* data)<br>
{<br>
syncstore_args *args = (syncstore_args*)data;<br>
return syncop_setxattr(FIRST_CHILD(args->this),
args->loc, args->dic, 0);<br>
}<br>
<br>
int32_t xattr_store_sync(xlator_t *this, call_frame_t
*frame, loc_t *loc, dict_t *dic)<br>
{<br>
syncstore_args args = {this, loc, dic};<br>
return synctask_new(this->ctx->env,
__xattr_store_sync, NULL, NULL, &args);<br>
</div>
</blockquote>
If you don't provide a synctask_cbk_t to synctask_new, you are
using synctask in a 'blocking' mode.<br>
That is, the thread calling synctask_new would block until the
synctask_fn_t function (ie, __xattr_store_sync) returns.<br>
An alternative way to do this would be,<br>
<br>
int32_t xattr_store_sync(xlator_t *this, call_frame_t *frame,
loc_t *loc, dict_t *dic)<br>
{<br>
syncstore_args args = {this, loc, dic};<br>
return synctask_new(this->ctx->env,
__xattr_store_sync, __xattr_store_sync_cbk, NULL, &args);<br>
}<br>
<br>
int32_t __xattr_store_sync_cbk (int ret, /*and the other
args*/) <br>
{<br>
// Your code goes here<br>
return ret;<br>
}<br>
<br>
Now, all file operations performed using syncop_* inside
__xattr_store_sync would have the synchronous flavour, while
leaving the calling thread (thread calling xattr_store_sync
fn) 'free'. This should avoid the hang issue.<br>
<br>
HTH,<br>
krish<br>
<br>
<br>
<blockquote
cite="mid:BAY171-W930585B6245B591348F89EF7B20@phx.gbl">
<div dir="ltr">}<br>
<br>
<div>
<hr id="ecxstopSpelling">Date: Mon, 29 Apr 2013 00:19:11
-0700<br>
Subject: Re: [Gluster-devel] rpc problems when using
syncops in callbacks<br>
From: <a moz-do-not-send="true"
class="ecxmoz-txt-link-abbreviated"
href="mailto:anand.avati@gmail.com">anand.avati@gmail.com</a><br>
To: <a moz-do-not-send="true"
class="ecxmoz-txt-link-abbreviated"
href="mailto:fog_is_my_name@hotmail.com">fog_is_my_name@hotmail.com</a><br>
CC: <a moz-do-not-send="true"
class="ecxmoz-txt-link-abbreviated"
href="mailto:gluster-devel@nongnu.org">gluster-devel@nongnu.org</a><br>
<br>
<div dir="ltr">Note that you need to place your syncop
code in a synctask function strictly within a syncenv
(by calling synctask_new(). You're probably calling
syncop_XXX() directly in your xlator code?
<div><br>
</div>
<div>Avati</div>
</div>
<div class="ecxgmail_extra"><br>
<br>
<div class="ecxgmail_quote">On Fri, Apr 26, 2013 at
2:40 AM, fog - <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:fog_is_my_name@hotmail.com"
target="_blank">fog_is_my_name@hotmail.com</a>></span>
wrote:<br>
<blockquote class="ecxgmail_quote"
style="border-left:1px #ccc
solid;padding-left:1ex;">
<div>
<div dir="ltr">Hello everyone,<br>
<div>
<div dir="ltr">
<div>
<div><br>
</div>
I am trying to use syncops in a custom
translator to keep my code at least
borderline readable, but I am having
limited success. <br>
<br>
Problem Symptoms: <br>
Using a syncop in a regular fop is fine.
However, in a callback it causes a
'freeze' (synctask_yield called by the
SYNCOP macro doesn't return). <br>
<br>
</div>
<div>What seems to be the Problem: <br>
</div>
<div>Looking at the traces, there is no
corresponding trace from
rpc_clnt_reply_init on the client to the
trace from rpcsvc_submit_generic on the
server. In other words, the rpc reply
gets sent but isn't correctly received.
Obviously this is not really a
networking problem but something else...
I'd guess it's a deadlock somewhere on
the client?<br>
</div>
<div>From the point of the syncop call
onwards the client doesn't 'get' any rpc
replies any more (the next GlusterFS
Handshake sent by the client, which is
received by the server and replied to,
leads to a disconnection accordingly).<br>
</div>
<div><br>
</div>
<div>Again: This problem is only occurring
when calling a syncop from a callback
function inside my translator, if I call
the same syncop in a fop call it
completes fine.<br>
<br>
I hope you can make sense out of the
above problem description. <br>
</div>
<div>Thanks for your time ~<br>
</div>
<br>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a moz-do-not-send="true"
href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a><br>
<a moz-do-not-send="true"
href="https://lists.nongnu.org/mailman/listinfo/gluster-devel"
target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
<fieldset class="ecxmimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Gluster-devel mailing list
<a moz-do-not-send="true" class="ecxmoz-txt-link-abbreviated" href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a>
<a moz-do-not-send="true" class="ecxmoz-txt-link-freetext" href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>