I just received this error message using rc4:<div><br></div><div><div>2009-03-09 21:58:16 E [client-protocol.c:505:client_ping_timer_expired] distfs03-stripe: ping timer expired! bailing transport</div><div>2009-03-09 21:58:16 N [client-protocol.c:6607:notify] distfs03-stripe: disconnected</div>
<div><br></div><div>It happened a total of 7 times across my 33 client nodes. It doesn't seem to be related to any particular client, but the errors did happen mostly (though not always) on the unify-ns server. The gluster servers are under pretty heavy network utilization, however it doesn't seem to be near the link capacity and in any case, i/o should just block if it's slow to respond, correct? Fortunately, gluster is automatically reconnecting after the error. I don't remember seeing this in rc2. The only corresponding errors in the server logs are simply showing the client disconnecting. I've also ruled out any interconnect faults.</div>
<div><br></div><div>Any suggestions? My configs are below.</div><br>Dan<br>
</div><div><br></div><div>CLIENT CONFIG:</div><div><br></div><div><div>volume unify-switch-ns</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.51</div><div>
option remote-subvolume posix-unify-switch-ns</div><div>end-volume</div><div><br></div><div>#volume distfs01-ns-readahead</div><div># type performance/read-ahead</div><div># option page-size 1MB</div><div># option page-count 8</div>
<div># subvolumes distfs01-ns-brick</div><div>#end-volume</div><div><br></div><div>#volume unify-switch-ns</div><div># type performance/write-behind</div><div># option block-size 1MB</div><div># option cache-size 3MB</div>
<div># subvolumes distfs01-ns-readahead</div><div>#end-volume</div><div><br></div><div>volume distfs01-unify</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.51</div>
<div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs02-unify</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.52</div>
<div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs03-unify</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.53</div>
<div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs04-unify</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.54</div>
<div> option remote-subvolume posix-unify</div><div>end-volume</div><div><br></div><div>volume distfs01-stripe</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.51</div>
<div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume distfs02-stripe</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.52</div>
<div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume distfs03-stripe</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.53</div>
<div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume distfs04-stripe</div><div> type protocol/client</div><div> option transport-type tcp</div><div> option remote-host 10.8.101.54</div>
<div> option remote-subvolume posix-stripe</div><div>end-volume</div><div><br></div><div>volume stripe0</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>type cluster/stripe</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option block-size *.jar,*.pin:1MB,*:2MB</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>subvolumes distfs01-stripe distfs02-stripe distfs03-stripe distfs04-stripe</div><div>end-volume</div><div><br></div><div>volume dht0</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>type cluster/dht</div>
<div>#<span class="Apple-tab-span" style="white-space:pre"> </span>option lookup-unhashed yes</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>subvolumes distfs01-unify distfs02-unify distfs03-unify distfs04-unify</div>
<div>end-volume</div><div><br></div><div>volume unify</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>type cluster/unify</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option namespace unify-switch-ns</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>option self-heal off</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option scheduler switch</div><div># send *.phr/psq/pnd etc to stripe0, send the rest to hash</div>
<div># extensions have to be *.foo* and not simply *.foo or rsync's tmp file naming will prevent files from being matched</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option scheduler.switch.case *.phr*:stripe0;*.psq*:stripe0;*.pnd*:stripe0;*.psd*:stripe0;*.pin*:stripe0;*.nsi*:stripe0;*.nin*:stripe0;*.nsd*:stripe0;*.nhr*:stripe0;*.nsq*:stripe0;*.tar*:stripe0;*.tar.gz*:stripe0;*.jar*:stripe0;*.img*:stripe0;*.perf*:stripe0;*.tgz*:stripe0;*.fasta*:stripe0;*.huge*:stripe0</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>subvolumes stripe0 dht0</div><div>end-volume</div><div><br></div><div>volume ioc</div><div> type performance/io-cache</div><div> subvolumes unify</div>
<div> option cache-size 3000MB</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option cache-timeout 3600</div><div>end-volume</div><div><br></div><div>volume filter</div><div> type features/filter</div>
<div> option fixed-uid 0</div><div> option fixed-gid 900</div><div> subvolumes ioc</div><div>end-volume</div><div><br></div><div><br></div><div><br></div><div><br></div><div>SERVER CONFIG:</div><div>volume posix-unify-brick</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>type storage/posix</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option directory /distfs-storage-space/glusterfs/unify</div><div>
# the below line is here to make the output of 'df' accurate, as both volumes are served from the same local drive</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option export-statfs-size off</div>
<div>end-volume</div><div><br></div><div>volume posix-stripe-brick</div><div> type storage/posix</div><div> option directory /distfs-storage-space/glusterfs/stripe</div><div>end-volume</div><div><br></div><div>
volume posix-unify-switch-ns-brick</div><div> type storage/posix</div><div> option directory /distfs-storage-space/glusterfs/unify-switch-ns</div><div>end-volume</div><div><br></div><div>volume posix-unify</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>type performance/io-threads</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option thread-count 4</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>subvolumes posix-unify-brick</div>
<div>end-volume</div><div><br></div><div>volume posix-stripe</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>type performance/io-threads</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option thread-count 4</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>subvolumes posix-stripe-brick</div><div>end-volume</div><div><br></div><div>volume posix-unify-switch-ns</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>type performance/io-threads</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>option thread-count 2</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>subvolumes posix-unify-switch-ns-brick</div><div>end-volume</div>
<div><br></div><div>volume server</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>type protocol/server</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option transport-type tcp</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>option auth.addr.posix-unify.allow 10.8.101.*,10.8.15.50</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>option auth.addr.posix-stripe.allow 10.8.101.*,10.8.15.50</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>option auth.addr.posix-unify-switch-ns.allow 10.8.101.*,10.8.15.50</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>subvolumes posix-unify posix-stripe posix-unify-switch-ns</div>
<div>end-volume</div><div><br></div></div>