<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Thanks to Anand, I have some serious speed ups on local machine performance, by combining the server and client within the client config. This removed a few overheads, and both write and read speeds are up on each individual machine</div><div><br></div><div>However - using the attached spec files, neither server is able to connect to the other, and I am stumped as to why, each log file reads the equiv of:</div><div><br></div><div><div>2008-06-10 13:07:32 E [tcp-client.c:171:tcp_connect] latsrv2-local: non-blocking connect() returned: 111 (Connection refused)</div><div><br></div><div>This simply looks like there is no protocol/sever to connect to for the other client.</div><div><br></div><div>Is anyone able to spot a howler in here, or is it something more fundamental?</div><div><br></div><div>P.S. Apologies to Anand for sending this to you twice!</div><div><br></div></div><div>The two client specs are here (as there are no longer any server specs!)</div><div><br></div><div><div>LATSRV1:</div><div><div>#Start with server defs in our client conf.</div><div>#We can save on the overhead of a seperate glusterfsd</div><div>#Becuase we are always running a server+client pair </div><div><br></div><div>volume posix </div><div> type storage/posix</div><div> option directory /home/export2</div><div>end-volume</div><div><br></div><div>volume plocks</div><div> type features/posix-locks</div><div> subvolumes posix</div><div>end-volume</div><div><br></div><div>volume latsrv1-local</div><div> type performance/io-threads</div><div> option thread-count 8</div><div> option cache-size 64MB</div><div> subvolumes plocks </div><div>end-volume</div><div><br></div><div>volume server</div><div> type protocol/server</div><div> option transport-type tcp/server<span class="Apple-tab-span" style="white-space: pre; "> </span># For TCP/IP transport</div><div> option auth.ip.latsrv1-local.allow *<span class="Apple-tab-span" style="white-space: pre; "> </span># Allow access to "brick" volume</div><div> subvolumes latsrv1-local</div><div>end-volume</div><div><br></div><div>#Continue with the client spec..</div><div><br></div><div>volume latsrv2-local</div><div> type protocol/client</div><div> option transport-type tcp/client</div><div> option remote-host latsrv2</div><div># option remote-subvolume 195.224.189.148 #fake to model unavailable server</div><div> option remote-subvolume latsrv2-local</div><div>end-volume</div><div><br></div><div>volume data-afr</div><div> type cluster/afr</div><div> subvolumes latsrv1-local latsrv2-local</div><div> option read-subvolume latsrv1-local</div><div> option self-heal on</div><div>end-volume</div><div><br></div><div>volume data</div><div> type performance/read-ahead</div><div> option page-size 128kB<span class="Apple-tab-span" style="white-space: pre; "> </span># 256KB is the default option</div><div> option page-count 4<span class="Apple-tab-span" style="white-space: pre; "> </span># 2 is default option</div><div> option force-atime-update off<span class="Apple-tab-span" style="white-space: pre; "> </span># default is off</div><div> subvolumes data-afr</div><div>end-volume</div><div><br></div><div>#we will mount the volume data.</div><div><br></div></div><div><br></div><div><br></div><div>LATSRV2:</div><div><div>#Start with server defs in our client conf.</div><div>#We can save on the overhead of a seperate glusterfsd</div><div>#Becuase we are always running a server+client pair </div><div><br></div><div>volume posix </div><div> type storage/posix</div><div> option directory /home/export2</div><div>end-volume</div><div><br></div><div>volume plocks</div><div> type features/posix-locks</div><div> subvolumes posix</div><div>end-volume</div><div><br></div><div>volume latsrv2-local</div><div> type performance/io-threads</div><div> option thread-count 8</div><div> option cache-size 64MB</div><div> subvolumes plocks </div><div>end-volume</div><div><br></div><div>volume server</div><div> type protocol/server</div><div> option transport-type tcp/server<span class="Apple-tab-span" style="white-space: pre; "> </span># For TCP/IP transport</div><div> option auth.ip.latsrv2-local.allow *<span class="Apple-tab-span" style="white-space: pre; "> </span># Allow access to "brick" volume</div><div> subvolumes latsrv2-local</div><div>end-volume</div><div><br></div><div>#Continue with the client spec..</div><div><br></div><div>volume latsrv1-local</div><div> type protocol/client</div><div> option transport-type tcp/client</div><div> option remote-host latsrv1</div><div># option remote-subvolume 195.224.189.148 #fake to model unavailable server</div><div> option remote-subvolume latsrv1-local</div><div>end-volume</div><div><br></div><div>volume data-afr</div><div> type cluster/afr</div><div> subvolumes latsrv1-local latsrv2-local</div><div> option read-subvolume latsrv2-local</div><div> option self-heal on</div><div>end-volume</div><div><br></div><div>volume data</div><div> type performance/read-ahead</div><div> option page-size 128kB<span class="Apple-tab-span" style="white-space: pre; "> </span># 256KB is the default option</div><div> option page-count 4<span class="Apple-tab-span" style="white-space: pre; "> </span># 2 is default option</div><div> option force-atime-update off<span class="Apple-tab-span" style="white-space: pre; "> </span># default is off</div><div> subvolumes data-afr</div><div>end-volume</div><div><br></div><div>#we will mount the volume data.</div><div><br></div></div></div><div><br></div><br><div><div>On 5 Jun 2008, at 20:51, Anand Babu Periasamy wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>There is lot of scope for improvement in performance and simplicity.<br><br>Booster translator will help only when you LD_PRELOAD glusterfs-booster.so<br>before launching your applications. It bypasses kernel-fuse for reads and<br>writes. Even in that case, it makes sense to load booster translator on<br>the<br>server side.<br><br>In your setup, you have 2 servers acting as complete mirror for each other<br>(server and client for each other). You can merge client and server into<br>one process by loading protocol/server into the client space. It will be a<br>lot simpler and faster. Just 2 vol spec files.<br><br>In upcoming 1.4, you will also be able to use the web embeddable glusterfs<br>client to directly access the storage from apache addresses space (or even<br>run the whole file system inside apache or lighttpd). It also has binary<br>protocol (fast and efficient) and non-blocking I/O functionalities.<br><br>Please see the attached PDF. It will give you a good idea.<br><br>--<br>Anand Babu Periasamy<br>GPG Key ID: 0x62E15A31<br>Blog [<a href="http://ab.freeshell.org">http://ab.freeshell.org</a>]<br>The GNU Operating System [<a href="http://www.gnu.org">http://www.gnu.org</a>]<br>Z RESEARCH Inc [<a href="http://www.zresearch.com">http://www.zresearch.com</a>]<br><br><br><br>Daniel Jordan Bambach wrote:<br><blockquote type="cite">Hiya all..<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">A scenario that seems to be a very neat solution to a basic high <br></blockquote>availability Webserver set up (Apache, Mysql, Python+Django) is to set <br>up two machines, configure master<->master replication between the two <br>MySQL databases, and then set up GlusterFS to mirror a filesystem <br>between the machine that carries the Apache config, Django<br><blockquote type="cite">applications, and file upload folders between the machines. You can <br></blockquote>pull the plug on either, and things should keep running on the other.<br><blockquote type="cite"><br></blockquote><blockquote type="cite">With this in mind, I have set up an arrangement whereby each box runs <br></blockquote>GlusterFSD, and has a client running on them that connects the local <br>server. AFR is set up at the server level, so that perhaps when/if the <br>other machine goes down, the client happily carries on dealing with <br>read/ write requests while the server deals with the non-existence of <br>the other server.<br><blockquote type="cite"><br></blockquote><blockquote type="cite">I've set this up in a test environment, and all is working peachy, and <br></blockquote>we are thinking of moving to deploy this to a new production<br><blockquote type="cite">environment.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">With this in mind, I wanted to poll the collective knowledge of this <br></blockquote>list to see if there are any gotchas to this set up I might have <br>missed, or any obvious performance features I should be using that I <br>am not.<br><blockquote type="cite"><br></blockquote><blockquote type="cite">Any help or advise would be greatly appreciated!!<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Here are the current server and client configs for the two machines:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">Gluster-users mailing list<br></blockquote><blockquote type="cite"><a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br></blockquote><blockquote type="cite"><a href="http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users">http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users</a><br></blockquote><blockquote type="cite"><br></blockquote><br><br><span><GlusterFS-Layout.pdf></span><span><GlusterFS-Layout.odp></span></div></blockquote></div><br></body></html>