Dear gluster experts,<div><br></div><div>We have encountered a severe performance issue related to quota feature of gluster.</div><div><br></div><div>My underlying fs is lvm with xfs format.</div><div>The problem is if quota is enabled the io performance is about 26MB/s but with quota disabled the io performance is 216MB/s.</div>
<div>Any one known what's the problem? BTW I have reproduce it several times and it is related to quota indeed.</div><div><br></div><div>Here's the test:</div><div><br></div><div>__________________quota enabled________________________________________</div>
<div><div><br></div></div><div><div>-sh-4.1$ sudo gluster volume info test_replica2_dis4</div><div> </div><div>Volume Name: test_replica2_dis4</div><div>Type: Distributed-Replicate</div><div>Volume ID: 9004b9d8-88f9-4312-9c22-1f1a62fd9784</div>
<div>Status: Started</div><div>Number of Bricks: 4 x 2 = 8</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: bjzw.miaoyan.cluster1.node1.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick2: bjzw.miaoyan.cluster1.node2.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick3: bjzw.miaoyan.cluster1.node3.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick4: bjzw.miaoyan.cluster1.node4.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick5: bjzw.miaoyan.cluster1.node5.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick6: bjzw.miaoyan.cluster1.node6.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick7: bjzw.miaoyan.cluster1.node7.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick8: bjzw.miaoyan.cluster1.node8.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Options Reconfigured:</div><div>features.limit-usage: /:5TB</div><div>features.quota: enable</div></div><div><div>-sh-4.1$ sudo dd if=/dev/zero of=iotest/testfff2 bs=1M count=1000</div><div>1000+0 records in</div><div>
1000+0 records out</div><div>1048576000 bytes (1.0 GB) copied, 40.3801 s, 26.0 MB/s</div><div><div>______________________________________________________________________</div></div><div><br></div><div><br></div><div>__________________quota disabled________________________________________</div>
<div><br></div><div><div><div>-sh-4.1$ sudo gluster volume info test_replica2_dis4</div><div> </div><div>Volume Name: test_replica2_dis4</div><div>Type: Distributed-Replicate</div><div>Volume ID: 9004b9d8-88f9-4312-9c22-1f1a62fd9784</div>
<div>Status: Started</div><div>Number of Bricks: 4 x 2 = 8</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: bjzw.miaoyan.cluster1.node1.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick2: bjzw.miaoyan.cluster1.node2.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick3: bjzw.miaoyan.cluster1.node3.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick4: bjzw.miaoyan.cluster1.node4.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick5: bjzw.miaoyan.cluster1.node5.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Brick6: bjzw.miaoyan.cluster1.node6.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick7: bjzw.miaoyan.cluster1.node7.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div><div>Brick8: bjzw.miaoyan.cluster1.node8.qiyi.domain:/mnt/xfsd/test_replica2_dis4</div>
<div>Options Reconfigured:</div><div>features.limit-usage: /:5TB</div><div>features.quota: disable</div></div><div>-sh-4.1$ sudo dd if=/dev/zero of=iotest/testfff2 bs=1M count=1000</div><div>1000+0 records in</div><div>1000+0 records out</div>
<div>1048576000 bytes (1.0 GB) copied, 4.86384 s, 216 MB/s</div></div><div><div>______________________________________________________________________</div></div><div><br></div><div>Thank you!</div><div><br></div>-- <br>·ûÓÀÌÎ<br>
</div>