Just find out set features.quota-timeout to an acceptable value like 30 significantly helps.<div>for example:</div><div><br></div><div>sudo gluster volume set test_replica2_dis4 features.quota-timeout 30</div><div><br></div>
<div><div>-sh-4.1$ sudo dd if=/dev/zero of=/mnt/iotest/xferd bs=1M count=2000</div><div>2000+0 records in</div><div>2000+0 records out</div><div>2097152000 bytes (2.1 GB) copied, 10.4415 s, 201 MB/s</div><br><div class="gmail_quote">
2012/9/29 ·ûÓÀÌÎ <span dir="ltr"><<a href="mailto:yongtaofu@gmail.com" target="_blank">yongtaofu@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Dear gluster experts,<div><br></div><div>We have encountered a severe performance issue related to quota feature of gluster.</div><div><br></div><div>My underlying fs is lvm with xfs format.</div><div>The problem is if quota is enabled the io performance is about 26MB/s but with quota disabled the io performance is 216MB/s.</div>
<div>Any one known what's the problem? BTW I have reproduce it several times and it is related to quota indeed.</div><div><br></div><div>Here's the test:</div><div><br></div><div>__________________quota enabled________________________________________</div>
<div><div><br></div></div><div><div>-sh-4.1$ sudo gluster volume info test_replica2_dis4</div><div> </div><div>Volume Name: test_replica2_dis4</div><div>Type: Distributed-Replicate</div><div>Volume ID: 9004b9d8-88f9-4312-9c22-1f1a62fd9784</div>
<div>Status: Started</div><div>Number of Bricks: 4 x 2 = 8</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: bjzw.miaoyan.cluster1.node1.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick2: bjzw.miaoyan.cluster1.node2.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick3: bjzw.miaoyan.cluster1.node3.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick4: bjzw.miaoyan.cluster1.node4.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick5: bjzw.miaoyan.cluster1.node5.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick6: bjzw.miaoyan.cluster1.node6.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick7: bjzw.miaoyan.cluster1.node7.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick8: bjzw.miaoyan.cluster1.node8.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Options Reconfigured:</div><div>features.limit-usage: /:5TB</div><div>features.quota: enable</div></div><div><div>-sh-4.1$ sudo dd if=/dev/zero of=iotest/testfff2 bs=1M count=1000</div><div>1000+0 records in</div><div>
1000+0 records out</div><div>1048576000 bytes (1.0 GB) copied, 40.3801 s, 26.0 MB/s</div><div><div>______________________________________________________________________</div></div><div><br></div><div><br></div><div>__________________quota disabled________________________________________</div>
<div><br></div><div><div><div>-sh-4.1$ sudo gluster volume info test_replica2_dis4</div><div> </div><div>Volume Name: test_replica2_dis4</div><div>Type: Distributed-Replicate</div><div>Volume ID: 9004b9d8-88f9-4312-9c22-1f1a62fd9784</div>
<div>Status: Started</div><div>Number of Bricks: 4 x 2 = 8</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: bjzw.miaoyan.cluster1.node1.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick2: bjzw.miaoyan.cluster1.node2.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick3: bjzw.miaoyan.cluster1.node3.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick4: bjzw.miaoyan.cluster1.node4.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick5: bjzw.miaoyan.cluster1.node5.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick6: bjzw.miaoyan.cluster1.node6.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick7: bjzw.miaoyan.cluster1.node7.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick8: bjzw.miaoyan.cluster1.node8.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Options Reconfigured:</div><div>features.limit-usage: /:5TB</div><div>features.quota: disable</div></div><div>-sh-4.1$ sudo dd if=/dev/zero of=iotest/testfff2 bs=1M count=1000</div><div>1000+0 records in</div><div>1000+0 records out</div>
<div>1048576000 bytes (1.0 GB) copied, 4.86384 s, 216 MB/s</div></div><div><div>______________________________________________________________________</div></div><div><br></div><div>Thank you!</div><span class="HOEnZb"><font color="#888888"><div>
<br></div>-- <br>·ûÓÀÌÎ<br>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>·ûÓÀÌÎ<br>
</div>