Hi Dan,<br> These problems poped up when we did some bug fixes in our hashing algorithm. (it was alright in 1.4.0preX releases, and got corrected from rc2). This behavior can happen for all those who used dht in rc1, and now shifted to higher rcX releases. <br>
If you notice the name length is multiple of 16 (32 in this case), and to fix this, we have two approach as of now. Have a separate '/mnt/debug' mountpoint with 'option lookup-unhashed yes' in dht. Now stat the files which are of filename length 16 or multiple of 16 over this debug mountpoint. This should fix your missing file problem on main mountpoint also (as it creates a proper linkfile in proper hashed volume).<br>
<br>Sorry for this in-convenience.<br><br>Regards,<br>Amar<br><br><br><div class="gmail_quote">On Mon, Mar 9, 2009 at 10:40 PM, Dan Parsons <span dir="ltr"><<a href="mailto:dparsons@nyip.net">dparsons@nyip.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">I'm getting the below error messages in rc4. Like my previous email, there doesn't seem to be any pattern as to which server/client it's happening on, though the errors are occurring fairly frequently.<div>
<br>
</div><div><div>2009-03-09 17:32:26 E [unify.c:585:unify_lookup] unify: returning ESTALE for /bio/data/fast-hmmsearch-all/tmpP986E__fast-hmmsearch-all_job/result.tigrfam.TIGR02622.hmmhits: file count is 1</div><div>2009-03-09 17:32:26 E [unify.c:591:unify_lookup] unify: /bio/data/fast-hmmsearch-all/tmpP986E__fast-hmmsearch-all_job/result.tigrfam.TIGR02622.hmmhits: found on unify-switch-ns</div>
<div>2009-03-09 17:32:26 W [fuse-bridge.c:301:need_fresh_lookup] fuse-bridge: revalidate of /bio/data/fast-hmmsearch-all/tmpP986E__fast-hmmsearch-all_job/result.tigrfam.TIGR02622.hmmhits failed (Stale NFS file handle)</div>
<div><br></div><div>2009-03-09 17:32:28 E [unify.c:360:unify_lookup_cbk] unify: child(dht0): path(/bio/data/fast-hmmsearch-all/tmpP986E__fast-hmmsearch-all_job/result.tigrfam.TIGR01420.hmmhits): No such file or directory</div>
<div>2009-03-09 17:32:28 E [unify.c:360:unify_lookup_cbk] unify: child(unify-switch-ns): path(/bio/data/fast-hmmsearch-all/tmpP986E__fast-hmmsearch-all_job/result.tigrfam.TIGR01420.hmmhits): No such file or directory</div>
<div><br></div><div>As you can see, there are two separate sets of errors for two different files, though both errors are troubling. This problem has persisted from rc2 to rc4, though I can't say for certain that it was introduced in rc2 (I think it was there prior to that as well). No matching errors in server logs.</div>
<div><br></div><div>Any suggestions? My configs are below. Thanks!</div><div><br></div><div><span style="border-collapse: collapse;"><div>CLIENT CONFIG:</div><div><br></div><div><div>volume unify-switch-ns</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.51</div><div> option remote-subvolume posix-unify-switch-ns</div><div>end-volume</div><div><br></div><div>#volume distfs01-ns-readahead</div>
<div># type performance/read-ahead</div><div># option page-size 1MB</div><div># option page-count 8</div><div># subvolumes distfs01-ns-brick</div><div>#end-volume</div><div><br></div><div>#volume unify-switch-ns</div>
<div># type performance/write-behind</div><div># option block-size 1MB</div><div># option cache-size 3MB</div><div># subvolumes distfs01-ns-readahead</div><div>#end-volume</div><div><br></div><div>volume distfs01-unify</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.51</div><div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs02-unify</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.52</div><div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs03-unify</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.53</div><div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs04-unify</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.54</div><div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs01-stripe</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.51</div><div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume distfs02-stripe</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.52</div><div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume distfs03-stripe</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.53</div><div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume distfs04-stripe</div>
<div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.54</div><div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume stripe0</div>
<div><span style="white-space: pre;"> </span>type cluster/stripe</div><div><span style="white-space: pre;"> </span>option block-size *.jar,*.pin:1MB,*:2MB</div><div><span style="white-space: pre;"> </span>subvolumes distfs01-stripe distfs02-stripe distfs03-stripe distfs04-stripe</div>
<div>end-volume</div><div><br></div><div>volume dht0</div><div><span style="white-space: pre;"> </span>type cluster/dht</div><div>#<span style="white-space: pre;"> </span>option lookup-unhashed yes</div><div><span style="white-space: pre;"> </span>subvolumes distfs01-unify distfs02-unify distfs03-unify distfs04-unify</div>
<div>end-volume</div><div><br></div><div>volume unify</div><div><span style="white-space: pre;"> </span>type cluster/unify</div><div><span style="white-space: pre;"> </span>option namespace unify-switch-ns</div><div><span style="white-space: pre;"> </span>option self-heal off</div>
<div><span style="white-space: pre;"> </span>option scheduler switch</div><div># send *.phr/psq/pnd etc to stripe0, send the rest to hash</div><div># extensions have to be *.foo* and not simply *.foo or rsync's tmp file naming will prevent files from being matched</div>
<div><span style="white-space: pre;"> </span>option scheduler.switch.case *.phr*:stripe0;*.psq*:stripe0;*.pnd*:stripe0;*.psd*:stripe0;*.pin*:stripe0;*.nsi*:stripe0;*.nin*:stripe0;*.nsd*:stripe0;*.nhr*:stripe0;*.nsq*:stripe0;*.tar*:stripe0;*.tar.gz*:stripe0;*.jar*:stripe0;*.img*:stripe0;*.perf*:stripe0;*.tgz*:stripe0;*.fasta*:stripe0;*.huge*:stripe0</div>
<div><span style="white-space: pre;"> </span>subvolumes stripe0 dht0</div><div>end-volume</div><div><br></div><div>volume ioc</div><div> type performance/io-cache</div><div> subvolumes unify</div><div> option cache-size 3000MB</div>
<div><span style="white-space: pre;"> </span>option cache-timeout 3600</div><div>end-volume</div><div><br></div><div>volume filter</div><div> type features/filter</div><div> option fixed-uid 0</div><div> option fixed-gid 900</div>
<div> subvolumes ioc</div><div>end-volume</div><div><br></div><div><br></div><div><br></div><div><br></div><div>SERVER CONFIG:</div><div>volume posix-unify-brick</div><div><span style="white-space: pre;"> </span>type storage/posix</div>
<div><span style="white-space: pre;"> </span>option directory /distfs-storage-space/glusterfs/unify</div><div># the below line is here to make the output of 'df' accurate, as both volumes are served from the same local drive</div>
<div><span style="white-space: pre;"> </span>option export-statfs-size off</div><div>end-volume</div><div><br></div><div>volume posix-stripe-brick</div><div> type storage/posix</div><div> option directory /distfs-storage-space/glusterfs/stripe</div>
<div>end-volume</div><div><br></div><div>volume posix-unify-switch-ns-brick</div><div> type storage/posix</div><div> option directory /distfs-storage-space/glusterfs/unify-switch-ns</div><div>end-volume</div>
<div><br></div><div>volume posix-unify</div><div><span style="white-space: pre;"> </span>type performance/io-threads</div><div><span style="white-space: pre;"> </span>option thread-count 4</div><div><span style="white-space: pre;"> </span>subvolumes posix-unify-brick</div>
<div>end-volume</div><div><br></div><div>volume posix-stripe</div><div><span style="white-space: pre;"> </span>type performance/io-threads</div><div><span style="white-space: pre;"> </span>option thread-count 4</div><div>
<span style="white-space: pre;"> </span>subvolumes posix-stripe-brick</div><div>end-volume</div><div><br></div><div>volume posix-unify-switch-ns</div><div><span style="white-space: pre;"> </span>type performance/io-threads</div>
<div><span style="white-space: pre;"> </span>option thread-count 2</div><div><span style="white-space: pre;"> </span>subvolumes posix-unify-switch-ns-brick</div><div>end-volume</div><div><br></div><div>volume server</div>
<div><span style="white-space: pre;"> </span>type protocol/server</div><div><span style="white-space: pre;"> </span>option transport-type tcp</div><div><span style="white-space: pre;"> </span>option auth.addr.posix-unify.allow 10.8.101.*,10.8.15.50</div>
<div><span style="white-space: pre;"> </span>option auth.addr.posix-stripe.allow 10.8.101.*,10.8.15.50</div><div><span style="white-space: pre;"> </span>option auth.addr.posix-unify-switch-ns.allow 10.8.101.*,10.8.15.50</div>
<div><span style="white-space: pre;"> </span>subvolumes posix-unify posix-stripe posix-unify-switch-ns</div><div>end-volume</div><div><br></div></div></span></div>
</div>
<br>_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a><br>
<a href="http://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Amar Tumballi<br><br>