<div dir="ltr">How does the NetBSD nfs server provide stable directory offsets, for the NFS client to resume reading from at a later point in time? Very similar problems are present in that scenario and it might be helpful to see what approaches are taken there (which are probably more tried and tested)<div><br></div><div>Thanks</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Sep 13, 2014 at 12:02 PM, Emmanuel Dreyfus <span dir="ltr"><<a href="mailto:manu@netbsd.org" target="_blank">manu@netbsd.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In <<a href="mailto:1lrx1si.n8tms1igmi5pM%25manu@netbsd.org">1lrx1si.n8tms1igmi5pM%manu@netbsd.org</a>> I explained why NetBSD<br>
currently fails self-heald.t, but since the subjet is burried deep in a<br>
thread, it might be worth starting a new one to talk about how to fix.<br>
<br>
In 3 places within glusterfs code (features/index,<br>
features/snapview-server and storage/posix), a server component answers<br>
readdir requests on a directory which may be split in mulitple calls.<br>
<br>
To answer one call, we have the following library calls:<br>
- opendir()<br>
- seekdir() to resume where the previous request was<br>
- readdir()<br>
- telldir() to record where we are for the next request<br>
- closedir()<br>
<br>
This relies on unspecified behavior, as POSIX says: "The value of loc<br>
should have been returned from an earlier call to telldir() using the<br>
same directory stream."<br>
<a href="http://pubs.opengroup.org/onlinepubs/9699919799/functions/seekdir.html" target="_blank">http://pubs.opengroup.org/onlinepubs/9699919799/functions/seekdir.html</a><br>
<br>
Since we do opendir() and closedir() at each time, we do not use the<br>
same directory stream. It causes an infinite loop on NetBSD because it<br>
badly resume from previous request, and in the general case it will<br>
break badly if an entry is added in the directory between two requests.<br>
<br>
How can we fix that?<br>
<br>
1) we can keep the directory stream open. The change is intrusive since<br>
we will need a chained list of open contexts, and we need to clean them<br>
if they timeout.<br>
<br>
2) in order to keep state between requests, we can use the entry index<br>
(first encoutered is 1, and so on) instead of values returned by<br>
telldir(). That works around the unspecified behavior, but it still<br>
breaks if directory content is changed between two requests<br>
<br>
3) make sure the readdir is done in a single request. That means trying<br>
with bigger buffers until it works. For instance in<br>
xlator/cluster/afr/src/afr-self-heald.c we have:<br>
while ((ret = syncop_readdir (subvol, fd, 131072, offset, &entries)))<br>
<br>
We would use -1 instead of 131072 to tell that we want everything<br>
without a size limit, and the server component (here features/index)<br>
would either return everyting or fail, whithout playing with<br>
telldir/seekdir.<br>
<br>
Opinions? The third solution seems the best to me since it is not very<br>
intrusive and it makes things simplier. Indeed we allow unbound data<br>
size to come back from the brick to glustershd, but we trust the brick,<br>
right?<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Emmanuel Dreyfus<br>
<a href="http://hcpnet.free.fr/pubz" target="_blank">http://hcpnet.free.fr/pubz</a><br>
<a href="mailto:manu@netbsd.org">manu@netbsd.org</a><br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-devel</a><br>
</font></span></blockquote></div><br></div>