Comments are inlined.<br><br><div class="gmail_quote">On Thu, Apr 8, 2010 at 9:21 PM, Olivier Le Cam <span dir="ltr"><<a href="mailto:Olivier.LeCam@crdp.ac-versailles.fr">Olivier.LeCam@crdp.ac-versailles.fr</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi again,<br>
<br>
Thanks for information.<div class="im"><br>
<br>
Raghavendra G wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Metadata caching is done in stat-prefetch. But that is not suitable to your use-case. in stat-prefetch stats of dentries are stored from the period of readdir to close on directory fd.<br>
</blockquote>
<br></div>
Although this translators is not easy to understand for a new comer in glusterfs devel, I'll carefully read its code.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Did you try with a very high cache-timeout value (say 5 minutes or more)? Also a bigger cache-size might help.<br>
</blockquote>
<br></div>
Yes, I did. In my situation, the problem is indeed the latency caused by tons of stats. My guess is that it should be possible to reach better improvement with metadata caching than with the io-cache (files are small and can usually be fetched with only a single frame).<br>
<br>
Well, now I have a crutial question regarding the way to wrote a translator. I used the rot-13 code as a starting point.<br>
<br>
I have a "lookup" xlator fop and its associated callback function. Here is a part my (very simple) speudo code:<br>
<br>
int32_t<br>
mdc_lookup_cbk (call_frame_t *frame, void *cookie, xlator_t *this,<br>
int32_t op_ret, int32_t op_errno, inode_t *inode,<br>
struct stat *stbuf, dict_t *dict,<br>
struct stat *postparent)<br>
{<br>
/* TODO: cache metadata */<br>
<br>
STACK_UNWIND_STRICT (lookup, frame, op_ret, op_errno, inode,<br>
stbuf, dict, postparent);<br>
}<br>
<br>
int32_t<br>
mdc_lookup (call_frame_t *frame, xlator_t *this, loc_t *loc,<br>
dict_t *xattr_req)<br>
{<br>
uint8_t cached = 0;<br>
<br>
if (xattr_req == NULL)<br>
goto out;<br>
<br>
/* TODO: check if we have cached metadata for "loc->inode"<br>
If so, prepare data and set cache to 1 */<br>
<br>
if (cached) {<br>
/* How to "re-inject" metadata in the engine? */<br>
STACK_UNWIND_STRICT (lookup, frame, -1, 0, NULL, NULL,<br>
NULL, NULL);<br>
return 0;<br>
}<br>
<br>
out :<br>
STACK_WIND (frame, mdc_lookup_cbk, FIRST_CHILD (this),<br>
FIRST_CHILD (this)->fops->lookup, loc, xattr_req);<br>
return 0;<br>
}<br>
<br>
In mdc_lookup, I think I can stop going through the lower-level xlators with the help of STACK_UNWIND_STRICT. But how can I deliver cached metadata back to the engine at this step?<br></blockquote><div><br>you can do that by sending the cached stats (here stat of file, stat of parent directory) through STACK_UNWIND.<br>
<br>STACK_UNWIND_STRICT (lookup, frame, 0, 0, loc->inode, cached_stat, NULL, parent_stat);<br><br>you can look into libglusterfs/src/defaults.c for default definitions of each of fop (file operations) and their call backs.<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Thanks,<div><div></div><div class="h5"><br>
-- <br>
Olivier<br>
<br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="http://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
</div></div></blockquote></div><br><br clear="all">regards,<br>-- <br>Raghavendra G<br><br>