<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Oh, Amar.. thankyou!</div><div>connected straight away!</div><div><br></div><div>Many thanks.</div><br><div><div>On 18 Jun 2008, at 05:58, Amar S. Tumballi wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hi Daniel,<br> I have fixed a bug in mainline-2.5 branch (from which 1.3.x releases are made), which addresses this issue. Let me know if you can try out the latest patchset from tla. Or you can use the below link too. (note the same version number, you need to make sure you did 'make uninstall', or 'rpm -e glusterfs' before installing this.<br> <br><a href="http://gnu.zresearch.com/~amar/qa-releases/glusterfs-1.3.9.tar.gz">http://gnu.zresearch.com/~amar/qa-releases/glusterfs-1.3.9.tar.gz</a><br><br>Regards,<br><br><div class="gmail_quote">On Tue, Jun 10, 2008 at 8:44 AM, Daniel Jordan Bambach <<a href="mailto:dan@lateral.net">dan@lateral.net</a>> wrote:<br> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div style=""><div>Thanks to Anand, I have some serious speed ups on local machine performance, by combining the server and client within the client config. This removed a few overheads, and both write and read speeds are up on each individual machine</div> <div><br></div><div>However - using the attached spec files, neither server is able to connect to the other, and I am stumped as to why, each log file reads the equiv of:</div><div><br></div><div><div>2008-06-10 13:07:32 E [tcp-client.c:171:tcp_connect] latsrv2-local: non-blocking connect() returned: 111 (Connection refused)</div> <div><br></div><div>This simply looks like there is no protocol/sever to connect to for the other client.</div><div><br></div><div>Is anyone able to spot a howler in here, or is it something more fundamental?</div><div><br> </div><div>P.S. Apologies to Anand for sending this to you twice!</div><div><br></div></div><div>The two client specs are here (as there are no longer any server specs!)</div><div><br></div><div><div>LATSRV1:</div><div><div> #Start with server defs in our client conf.</div><div>#We can save on the overhead of a seperate glusterfsd</div><div>#Becuase we are always running a server+client pair </div><div class="Ih2E3d"><div><br></div><div>volume posix </div> <div> type storage/posix</div></div><div> option directory /home/export2</div><div>end-volume</div><div><br></div><div>volume plocks</div><div class="Ih2E3d"><div> type features/posix-locks</div><div> subvolumes posix</div> <div>end-volume</div><div><br></div></div><div>volume latsrv1-local</div><div class="Ih2E3d"><div> type performance/io-threads</div><div> option thread-count 8</div><div> option cache-size 64MB</div></div><div> subvolumes plocks </div> <div class="Ih2E3d"><div>end-volume</div><div><br></div><div>volume server</div><div> type protocol/server</div><div> option transport-type tcp/server<span style="white-space: pre;"> </span># For TCP/IP transport</div></div> <div> option auth.ip.latsrv1-local.allow *<span style="white-space: pre;"> </span># Allow access to "brick" volume</div><div> subvolumes latsrv1-local</div><div>end-volume</div><div><br></div><div>#Continue with the client spec..</div> <div><br></div><div>volume latsrv2-local</div><div class="Ih2E3d"><div> type protocol/client</div><div> option transport-type tcp/client</div><div> option remote-host latsrv2</div></div><div># option remote-subvolume <a href="http://195.224.189.148" target="_blank">195.224.189.148</a> #fake to model unavailable server</div> <div> option remote-subvolume latsrv2-local</div><div>end-volume</div><div><br></div><div>volume data-afr</div><div> type cluster/afr</div><div> subvolumes latsrv1-local latsrv2-local</div><div> option read-subvolume latsrv1-local</div> <div> option self-heal on</div><div>end-volume</div><div><br></div><div>volume data</div><div class="Ih2E3d"><div> type performance/read-ahead</div><div> option page-size 128kB<span style="white-space: pre;"> </span># 256KB is the default option</div> <div> option page-count 4<span style="white-space: pre;"> </span># 2 is default option</div><div> option force-atime-update off<span style="white-space: pre;"> </span># default is off</div></div><div> subvolumes data-afr</div> <div>end-volume</div><div><br></div><div>#we will mount the volume data.</div><div><br></div></div><div><br></div><div><br></div><div>LATSRV2:</div><div><div>#Start with server defs in our client conf.</div><div>#We can save on the overhead of a seperate glusterfsd</div> <div>#Becuase we are always running a server+client pair </div><div class="Ih2E3d"><div><br></div><div>volume posix </div><div> type storage/posix</div></div><div> option directory /home/export2</div><div>end-volume</div> <div><br></div><div>volume plocks</div><div class="Ih2E3d"><div> type features/posix-locks</div><div> subvolumes posix</div><div>end-volume</div><div><br></div></div><div>volume latsrv2-local</div><div class="Ih2E3d"><div> type performance/io-threads</div><div> option thread-count 8</div><div> option cache-size 64MB</div></div><div> subvolumes plocks </div><div class="Ih2E3d"><div>end-volume</div><div><br></div><div>volume server</div><div> type protocol/server</div><div> option transport-type tcp/server<span style="white-space: pre;"> </span># For TCP/IP transport</div></div><div> option auth.ip.latsrv2-local.allow *<span style="white-space: pre;"> </span># Allow access to "brick" volume</div> <div> subvolumes latsrv2-local</div><div>end-volume</div><div><br></div><div>#Continue with the client spec..</div><div><br></div><div>volume latsrv1-local</div><div class="Ih2E3d"><div> type protocol/client</div><div> option transport-type tcp/client</div> <div> option remote-host latsrv1</div></div><div># option remote-subvolume <a href="http://195.224.189.148" target="_blank">195.224.189.148</a> #fake to model unavailable server</div><div> option remote-subvolume latsrv1-local</div> <div>end-volume</div><div><br></div><div>volume data-afr</div><div> type cluster/afr</div><div> subvolumes latsrv1-local latsrv2-local</div><div> option read-subvolume latsrv2-local</div><div> option self-heal on</div><div> end-volume</div><div><br></div><div>volume data</div><div class="Ih2E3d"><div> type performance/read-ahead</div><div> option page-size 128kB<span style="white-space: pre;"> </span># 256KB is the default option</div><div> option page-count 4<span style="white-space: pre;"> </span># 2 is default option</div><div> option force-atime-update off<span style="white-space: pre;"> </span># default is off</div></div><div> subvolumes data-afr</div> <div>end-volume</div><div><br></div><div>#we will mount the volume data.</div><div><br></div></div></div><div><br></div><br><div><div><div></div><div class="Wj3C7c"><div>On 5 Jun 2008, at 20:51, Anand Babu Periasamy wrote:</div> <br></div></div><blockquote type="cite"><div><div><div></div><div class="Wj3C7c">There is lot of scope for improvement in performance and simplicity.<br><br>Booster translator will help only when you LD_PRELOAD glusterfs-booster.so<br> before launching your applications. It bypasses kernel-fuse for reads and<br>writes. Even in that case, it makes sense to load booster translator on<br>the<br>server side.<br><br>In your setup, you have 2 servers acting as complete mirror for each other<br> (server and client for each other). You can merge client and server into<br>one process by loading protocol/server into the client space. It will be a<br>lot simpler and faster. Just 2 vol spec files.<br><br>In upcoming 1.4, you will also be able to use the web embeddable glusterfs<br> client to directly access the storage from apache addresses space (or even<br>run the whole file system inside apache or lighttpd). It also has binary<br>protocol (fast and efficient) and non-blocking I/O functionalities.<br> <br>Please see the attached PDF. It will give you a good idea.<br><br>--<br>Anand Babu Periasamy<br>GPG Key ID: 0x62E15A31<br>Blog [<a href="http://ab.freeshell.org" target="_blank">http://ab.freeshell.org</a>]<br>The GNU Operating System [<a href="http://www.gnu.org" target="_blank">http://www.gnu.org</a>]<br> Z RESEARCH Inc [<a href="http://www.zresearch.com" target="_blank">http://www.zresearch.com</a>]<br><br><br><br>Daniel Jordan Bambach wrote:<br><blockquote type="cite">Hiya all..<br></blockquote><blockquote type="cite"><br> </blockquote><blockquote type="cite">A scenario that seems to be a very neat solution to a basic high <br></blockquote>availability Webserver set up (Apache, Mysql, Python+Django) is to set <br>up two machines, configure master<->master replication between the two <br> MySQL databases, and then set up GlusterFS to mirror a filesystem <br>between the machine that carries the Apache config, Django<br><blockquote type="cite">applications, and file upload folders between the machines. You can <br> </blockquote>pull the plug on either, and things should keep running on the other.<br><blockquote type="cite"><br></blockquote><blockquote type="cite">With this in mind, I have set up an arrangement whereby each box runs <br> </blockquote>GlusterFSD, and has a client running on them that connects the local <br>server. AFR is set up at the server level, so that perhaps when/if the <br>other machine goes down, the client happily carries on dealing with <br> read/ write requests while the server deals with the non-existence of <br>the other server.<br><blockquote type="cite"><br></blockquote><blockquote type="cite">I've set this up in a test environment, and all is working peachy, and <br> </blockquote>we are thinking of moving to deploy this to a new production<br><blockquote type="cite">environment.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">With this in mind, I wanted to poll the collective knowledge of this <br> </blockquote>list to see if there are any gotchas to this set up I might have <br>missed, or any obvious performance features I should be using that I <br>am not.<br><blockquote type="cite"><br></blockquote><blockquote type="cite"> Any help or advise would be greatly appreciated!!<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Here are the current server and client configs for the two machines:<br></blockquote><blockquote type="cite"> <br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote></div></div><div class="Ih2E3d"><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite"> Gluster-users mailing list<br></blockquote><blockquote type="cite"><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br></blockquote><blockquote type="cite"><a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br> </blockquote><blockquote type="cite"><br></blockquote><br><br></div><span><GlusterFS-Layout.pdf></span><span><GlusterFS-Layout.odp></span></div></blockquote></div><br></div><br>_______________________________________________<br> Gluster-users mailing list<br> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br> <a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br> <br></blockquote></div><br><br clear="all"><br>-- <br>Amar Tumballi<br>Gluster/GlusterFS Hacker<br>[bulde on #gluster/<a href="http://irc.gnu.org">irc.gnu.org</a>]<br><a href="http://www.zresearch.com">http://www.zresearch.com</a> - Commoditizing Super Storage!</blockquote></div><br></body></html>