I've upgraded GlusterFS from 1.3.12 to 2.0rc1 and ran into the same duplicated messages. According to the packet captures the duplicate messages problem looks to be more client related as the operation is taking too long and times out then re-connects to issue the same IMAP COPY command. I haven't been able to recreate this with an NFS backend.<br>
<br>Also, Is there anything specific that can be done to improve small file performance (14k) on GlusterFS? So far my benchmarks are showing read/write performance on a few large (100Mb) files is quite a bit better than on hundreds of small 14Kb files.<br>
<br>There was some talk of using GlusterFS for email storage last November. Is anyone else successfully using this in a production enviroment?<br><br>Thanks,<br><br>Cory<br><br><div class="gmail_quote">On Thu, Jan 22, 2009 at 2:58 AM, Keith Freedman <span dir="ltr"><<a href="mailto:freedman@freeformit.com">freedman@freeformit.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">we ran into this problem.<br>
it seems related to timestamps being off by microseconds.<br>
when someone would check their email on one machine, then hit another whose time was off by even microseconds, it would think all the messages are suddenly new or different from ones it's checked.<br>
<br>
my guess is this problem doesn't exist in 2.0 because of the way it manages timestamps on files in HA.<br>
<br>
However, I can't tell you for sure, since we switched to dovecot which tracks messages by message id instead of timestamps and is also much more efficient.<br>
<br>
hope that helps.<div><div></div><div class="Wj3C7c"><br>
<br>
At 08:28 PM 1/21/2009, Cory Meyer wrote:<br>
</div></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="Wj3C7c">
I'm working on a project to use GlusterFS as the backend for email storage to replace the current NFS implementation. The goal is to configure GlusterFS w/AFR to replicate the files across all 3 storage nodes. Each storage node will also act a an email server configured behind a load balancer running Sendmail, Maildrop, and Courier-Imap.<br>
<br>
Main issue so far seems to be related with Courier-Imap in that when moving messages between IMAP folders some messages are duplicated with the io-threads enabled on the client side. Issue looks to be on the Courier-IMAP side though I haven't seen this with NFS and the duplicate messages within the Maildir have unique file names.<br>
<br>
Any other simular experiences with email services backended with GlusterFS?<br>
<br>
Node test hardware x 3:<br>
Quad core Xeon 2Ghz w/ 4x7200rpm SATA drives.<br>
Raid5 across all 4 drives. (Raid0 will also be tested if additional speed is necessary)<br>
8Gb Ram<br>
<br>
Network: Currently 100Mbit though production will be 1Gbit.<br>
<br>
Os: Debian Etch (2.6.18-6-686-bigmem)<br>
GlusterFS: 1.3.12<br>
Fuse: 2.7.3glfs10<br>
Courier-imap 4.1.1.20060828-5<br>
<br>
Partition Layout:<br>
sda1 --> Os (ext3)<br>
sda2 --> Swap<br>
sda5 --> glusterfs_data (ext3)<br>
<br>
GlusterFS patched Fuse kernel module, util, and libraries have been installed on each of my 3 storage nodes.<br>
<br>
Here is the basics of my Gluster configuration.<br>
Server: Brick --> TCP<br>
Client: Bricks --> AFR --> io-thread --> write-behind --> io-cache --> read-ahead<br>
<br>
Follow the Pastebin URL for my raw config file:<br></div></div>
<<a href="http://glusterfs.pastebin.com/f7814657c" target="_blank">http://glusterfs.pastebin.com/f7814657c</a>><a href="http://glusterfs.pastebin.com/f7814657c" target="_blank">http://glusterfs.pastebin.com/f7814657c</a><br>
<br>
Any suggestions?<br>
<br>
-- Cory<br>
<br><div class="Ih2E3d">
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br>
</div></blockquote>
<br>
</blockquote></div><br>