Ya I've had that set scene I started, any other suggested tweaks?<br><br>Thanks<br><br><div class="gmail_quote">On Fri, Jan 23, 2009 at 11:12 AM, Keith Freedman <span dir="ltr"><<a href="mailto:freedman@freeformit.com">freedman@freeformit.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">dont forget to set :<br>
option read-subvolume BRICKNAME<br>
<br>
in your AFR config, that'll improve your read performance significantly.<div class="Ih2E3d"><br>
<br>
<br>
At 11:06 AM 1/23/2009, Evan wrote:<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d">
Ideal what I'm trying to accomplish is to have multiple samba servers connected with a 1.544Mbps pipe stay in sync with each other. So its important for me to be able to have near really disk access speeds when reading and writing to the local gluster Node in the AFR group but as far as getting the data propagated out to the other nodes I know the 1.544Mbps pipe can't keep up so I'll take the fastest I can get as long as it doesn't affect the local performance (which is what I am seeing).<br>
<br>
<br>
<br></div><div><div></div><div class="Wj3C7c">
On Fri, Jan 23, 2009 at 10:56 AM, Keith Freedman <<mailto:<a href="mailto:freedman@freeformit.com" target="_blank">freedman@freeformit.com</a>><a href="mailto:freedman@freeformit.com" target="_blank">freedman@freeformit.com</a>> wrote:<br>
At 10:18 AM 1/23/2009, Evan wrote:<br>
I added the following to the bottom of my spec file:<br>
<br>
volume writebehind<br>
type performance/write-behind<br>
option aggregate-size 10MB # default is 0bytes<br>
option flush-behind off # default is 'off'<br>
subvolumes afr<br>
end-volume<br>
<br>
which gives me the following results when making a 10MB file<br>
# time dd if=/dev/zero of=/tmp/disktest count=10240 bs=1024<br>
10240+0 records in<br>
10240+0 records out<br>
10485760 bytes (10 MB) copied, 0.173179 s, 60.5 MB/s<br>
<br>
real 0m0.183s<br>
user 0m0.000s<br>
sys 0m0.204s<br>
<br>
# time dd if=/dev/zero of=/mnt/gluster/disktest count=10240 bs=1024<br>
10240+0 records in<br>
10240+0 records out<br>
10485760 bytes (10 MB) copied, 5.50861 s, 1.9 MB/s<br>
<br>
real 0m5.720s<br>
user 0m0.000s<br>
sys 0m0.060s<br>
<br>
Although this is better than I had before is there anyway to have gluster write the data to the localBrick and then sync/afr in the background so I could expect to see something closer to the 60 MB/s I see when writing to the local disk directly?<br>
<br>
<br>
what you really want is a delayed replication. I've asked for this in this mailing list recently, and was told that it's something they've considered (more as a DR feature than an HA feature), but it's not currently on the list of priorities.<br>
<br>
The issue, as I see it, is if it's an HA feature, then you really need to insure that the data is replicated before you let your application think the data is written. If the replication was delayed, and the server went down, the data is lost forever. This is bad for HA.<br>
if it's a DR feature, then you're probably ok, because usually disaster recovery scenarios can probably withstand some data loss, and you're more interested in a point-in-time snapshot of the data.<br>
<br>
FUSE is a problem, and TCP/IP is a problem with much overhead and large blocksizes.<br>
<br>
Ideally, glusters write-behind would be smart enough to aggregate smaller blocks of data into a large write. I think this would solve a lot of the problem you're having in your tests.<br>
<br>
Thanks<br>
<br>
<br></div></div><div class="Ih2E3d">
aghavendra G <<mailto:<a href="mailto:raghavendra@zresearch.com" target="_blank">raghavendra@zresearch.com</a>><mailto:<a href="mailto:raghavendra@zresearch.com" target="_blank">raghavendra@zresearch.com</a>><a href="mailto:raghavendra@zresearch.com" target="_blank">raghavendra@zresearch.com</a>> wrote:<br>
above afr with afr as a subvolume<br>
<br></div><div class="Ih2E3d">
On Fri, Jan 23, 2009 at 12:59 AM, Evan <_<mailto:<a href="mailto:Gluster@devnada.com" target="_blank">Gluster@devnada.com</a>><mailto:<a href="mailto:Gluster@devnada.com" target="_blank">Gluster@devnada.com</a>><a href="mailto:Gluster@devnada.com" target="_blank">Gluster@devnada.com</a>> wrote:<br>
Where should I put the write-behind translator?<br>
Just above afr with afr as a subvolume? Or should I put it just above my localBrick volume and below afr?<br>
<br>
<br>
Here is the output using /dev/zero:<br>
# time dd if=/dev/zero of=/mnt/gluster/disktest count=1024 bs=1024<br>
<br>
1024+0 records in<br>
1024+0 records out<br>
1048576 bytes (1.0 MB) copied, 1.90119 s, 552 kB/s<br>
<br>
real 0m2.098s<br>
user 0m0.000s<br>
sys 0m0.016s<br>
<br>
# time dd if=/dev/zero of=/tmp/disktest count=1024 bs=1024<br>
<br>
1024+0 records in<br>
1024+0 records out<br>
1048576 bytes (1.0 MB) copied, 0.0195388 s, 53.7 MB/s<br>
<br>
real 0m0.026s<br>
user 0m0.000s<br>
sys 0m0.028s<br>
<br>
<br>
Thanks<br>
<br>
<br></div><div class="Ih2E3d">
On Thu, Jan 22, 2009 at 12:52 PM, Anand Avati <<mailto:<a href="mailto:avati@zresearch.com" target="_blank">avati@zresearch.com</a>><mailto:<a href="mailto:avati@zresearch.com" target="_blank">avati@zresearch.com</a>><a href="mailto:avati@zresearch.com" target="_blank">avati@zresearch.com</a>> wrote:<br>
Do you have write-behind loaded on the client side? For IO testing,<br>
use /dev/zero instead of /dev/urandom.<br>
<br>
avati<br>
<br></div><div class="Ih2E3d">
On Fri, Jan 23, 2009 at 2:14 AM, Evan <_<mailto:<a href="mailto:Gluster@devnada.com" target="_blank">Gluster@devnada.com</a>><mailto:<a href="mailto:Gluster@devnada.com" target="_blank">Gluster@devnada.com</a>><a href="mailto:Gluster@devnada.com" target="_blank">Gluster@devnada.com</a>> wrote:<br>
> I have a 2 node single process AFR setup with 1.544Mbps bandwidth between<br>
> the 2 nodes. When I write a 1MB file to the gluster share it seems to AFR to<br>
> the other node in real time killing my disk IO speeds on the gluster mount<br>
> point. Is there anyway to fix this? Ideally I would like to see near real<br>
> disk IO speeds from/to the local gluster mount point and let the afr play<br>
> catch up in the background as the bandwidth becomes available.<br>
><br></div>
> Gluster Spec File (same on both nodes) <<<a href="http://pastebin.com/m58dc49d4" target="_blank">http://pastebin.com/m58dc49d4</a>><a href="http://pastebin.com/m58dc49d4" target="_blank">http://pastebin.com/m58dc49d4</a>><a href="http://pastebin.com/m58dc49d4" target="_blank">http://pastebin.com/m58dc49d4</a><div class="Ih2E3d">
<br>
<br>
> IO speed tests:<br>
> # time dd if=/dev/urandom of=/mnt/gluster/disktest count=1024 bs=1024<br>
> 1024+0 records in<br>
> 1024+0 records out<br>
> 1048576 bytes (1.0 MB) copied, 8.34701 s, 126 kB/s<br>
><br>
> real 0m8.547s<br>
> user 0m0.000s<br>
> sys 0m0.372s<br>
><br>
> # time dd if=/dev/urandom of=/tmp/disktest count=1024 bs=1024<br>
> 1024+0 records in<br>
> 1024+0 records out<br>
> 1048576 bytes (1.0 MB) copied, 0.253865 s, 4.1 MB/s<br>
><br>
> real 0m0.259s<br>
> user 0m0.000s<br>
> sys 0m0.284s<br>
><br>
><br>
> Thanks<br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br></div>
> <mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>><mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><div class="Ih2E3d">
<br>
<br>
> <a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br>
><br>
><br>
<br>
<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br></div>
<mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>><mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><div class="Ih2E3d">
<br>
<br>
<a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br>
<br>
<br>
<br>
<br>
--<br>
Raghavendra G<br>
<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br></div><div class="Ih2E3d">
<mailto:<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br>
<br>
<br>
</div></blockquote>
<br>
</blockquote></div><br>