<p>I noticed you are mounting with direct-io-mode=enable. That disables kernel readahead and can affect read perf.</p>
<p>Avati</p>
<div class="gmail_quote">On Oct 18, 2012 6:44 PM, "Doug Schouten" <<a href="mailto:dschoute@sfu.ca">dschoute@sfu.ca</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
I am noticing a rather slow read performance using GlusterFS 3.3 with the following configuration:<br>
<br>
Number of Bricks: 4<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: server1:/srv/data<br>
Brick2: server2:/srv/data<br>
Brick3: server3:/srv/data<br>
Brick4: server4:/srv/data<br>
Options Reconfigured:<br>
features.quota: off<br>
features.quota-timeout: 1800<br>
performance.flush-behind: on<br>
performance.io-thread-count: 64<br>
performance.quick-read: on<br>
performance.stat-prefetch: on<br>
performance.io-cache: on<br>
performance.write-behind: on<br>
performance.read-ahead: on<br>
performance.write-behind-<u></u>window-size: 4MB<br>
performance.cache-refresh-<u></u>timeout: 1<br>
performance.cache-size: 4GB<br>
nfs.rpc-auth-allow: none<br>
network.frame-timeout: 60<br>
nfs.disable: on<br>
performance.cache-max-file-<u></u>size: 1GB<br>
<br>
<br>
The servers are connected with bonded 1Gb ethernet, and have LSI MegaRAID arrays with 12x1 TB disks in RAID-6 array, using XFS file system mounted like:<br>
<br>
xfs logbufs=8,logbsize=32k,<u></u>noatime,nodiratime 0 0<br>
<br>
and we use the FUSE client<br>
<br>
localhost:/global /global glusterfs defaults,direct-io-mode=<u></u>enable,log-level=WARNING,log-<u></u>file=/var/log/gluster.log 0 0<br>
<br>
Our files are all >= 2MB. When rsync-ing we see about 50MB/s read performance which improves to 250MB/s after the first copy. This indicates to me that the disk caching is working as expected. However I am rather surprised by the low 50MB/s read speed; this is too low to be limited by network, and the native disk read performance is way better. Is there some configuration that can improve this situation?<br>
<br>
thanks,<br>
<br>
<br>
-- <br>
<br>
<br>
Doug Schouten<br>
Research Associate<br>
TRIUMF<br>
______________________________<u></u>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.<u></u>org/mailman/listinfo/gluster-<u></u>users</a><br>
</blockquote></div>