<HTML><HEAD>
<META content="text/html; charset=utf-8" http-equiv=Content-Type></HEAD>
<BODY>
<DIV>
<DIV style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif">In a replicated cluster, the client writes to all replicas at the same time. This is likely while you are only getting half the speed for writes as its going to two servers and therefore maxing your gigabit network. That is, unless I am misunderstanding how you are measuring the 60MB/s write speed.<BR><BR>I don't have any advice on the other bits...sorry.<BR><BR>Todd</DIV></DIV>
<DIV dir=ltr>
<HR>
<SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">From: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif"><A href="mailto:redshift@telenet.be">RedShift</A></SPAN><BR><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">Sent: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif">10/6/2013 12:41 AM</SPAN><BR><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">To: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif"><A href="mailto:gluster-users@gluster.org">Gluster-users@gluster.org</A></SPAN><BR><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">Subject: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif">[Gluster-users] Options to turn off/on for reliable virtual machinewrites & write performance</SPAN><BR><BR></DIV>Hi all, I'm building a cluster to host virtual machines to ESXi hosts (using NFS). The point of the cluster is that it should survive an unclean node death (test scenario by hard removing disks or cutting power, etc...), by which I need to make sure all writes are completed on both nodes before gluster returns the operation as completed. For now, I have this: <BR><BR><BR>gluster> volume info ha-ds1 <BR>Volume Name: ha-ds1 <BR>Type: Replicate <BR>Volume ID: da2fb668-2f3e-4839-a5da-4a51d5fcba05 <BR>Status: Started <BR>Number of Bricks: 1 x 2 = 2 <BR>Transport-type: tcp <BR>Bricks: <BR>Brick1: 10.255.255.1:/vol/gluster/ha-ds1 <BR>Brick2: 10.255.255.2:/vol/gluster/ha-ds1 <BR>Options Reconfigured: <BR>cluster.self-heal-daemon: on <BR>performance.flush-behind: Off <BR>network.frame-timeout: 30 <BR>network.ping-timeout: 15 <BR>cluster.heal-timeout: 300 <BR>gluster> volume status all detail <BR>Status of volume: ha-ds1 <BR>------------------------------------------------------------------------------ <BR>Brick : Brick 10.255.255.1:/vol/gluster/ha-ds1 <BR>Port : 49153 <BR>Online : Y <BR>Pid : 2252 <BR>File System : ext4 <BR>Device : /dev/mapper/stor--node1-gluster <BR>Mount Options : rw,noatime,nodiratime,journal_checksum,data=journal,errors=panic,nodelalloc <BR>Inode Size : 256 <BR>Disk Space Free : 219.3GB <BR>Total Disk Space : 269.1GB <BR>Inode Count : 17924096 <BR>Free Inodes : 17923263 <BR>------------------------------------------------------------------------------ <BR>Brick : Brick 10.255.255.2:/vol/gluster/ha-ds1 <BR>Port : 49152 <BR>Online : Y <BR>Pid : 2319 <BR>File System : ext4 <BR>Device : /dev/mapper/stor--node2-gluster <BR>Mount Options : rw,noatime,nodiratime,journal_checksum,data=journal,errors=panic,nodelalloc <BR>Inode Size : 256 <BR>Disk Space Free : 221.3GB <BR>Total Disk Space : 269.1GB <BR>Inode Count : 17924096 <BR>Free Inodes : 17923162 <BR>gluster> <BR><BR><BR>(I would also like to grab your attention to the mount options - are those OK or can I do better?) <BR><BR><BR>Is this enough to garantuee a proper cluster failover (data is consistent at all times) to the second node without interruption to the virtual machines? In my testing it appears to be, but I want to make sure - maybe someone else has something to add or something to look out for? <BR><BR><BR>Second, I'd like to improve the write performance of this cluster. Reads are good (> 110 MB/s, the ESXi servers are connected via gigabit so that'll be the maximum) but writes are only half that (~60 MB/s). The hardware can definitely do more - a simple dd 16 GB filewrite to the underlying filesystem nets ~227 MB/s. I gathered some statistics during sequential write tests, I see the load going to ~15 and some CPU usage but it looks like one CPU core is spendings its majority in IO wait. I know the hardware can perform better - are there any other places I should start looking? <BR><BR><BR>Thanks,<BR><BR>Glenn<BR>_______________________________________________<BR>Gluster-users mailing list<BR>Gluster-users@gluster.org<BR>http://supercolony.gluster.org/mailman/listinfo/gluster-users<BR></BODY></HTML>