<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>Right now I’m testing things out, trying to get Xen
going using gluster as the storage back end.<o:p></o:p></p>
<p class=MsoNormal>I have a pretty simple setup at the moment. <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>1 server running Xen 3.3.1/Debian Etch(gluster client)<o:p></o:p></p>
<p class=MsoNormal>1 server running openfiler 2.3 (gluster server)<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Networking on gluster client:<o:p></o:p></p>
<p class=MsoNormal>2 gigabit nics in bond0 (mode4)<o:p></o:p></p>
<p class=MsoNormal>bond0 is bridged with xenbr0<o:p></o:p></p>
<p class=MsoNormal>Xen adds virtual interfaces to the xenbr0 bridge for the virtual
machines.. <o:p></o:p></p>
<p class=MsoNormal>Basically all externally bound traffic goes through xenbr0,
then bond0 which is then balanced over eth0 and eth1<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Networking on gluster server:<o:p></o:p></p>
<p class=MsoNormal>2 gigabit nics in bond0 (mode4)<o:p></o:p></p>
<p class=MsoNormal>All external traffic goes through bond0<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Gluster version on server and client:<o:p></o:p></p>
<p class=MsoNormal># glusterfs --version<o:p></o:p></p>
<p class=MsoNormal>glusterfs 2.0.0rc1 built on Feb 17 2009 10:28:23<o:p></o:p></p>
<p class=MsoNormal>Repository revision: glusterfs--mainline--3.0--patch-844<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Fuse on gluster client:<o:p></o:p></p>
<p class=MsoNormal>I believe I’m using the Gluster patched fuse, but I’m
not sure.<o:p></o:p></p>
<p class=MsoNormal>When I tried to compile fuse, it said that the module was in
the kernel.<o:p></o:p></p>
<p class=MsoNormal>So I tried to compile gluster, and it didn’t see fuse.<o:p></o:p></p>
<p class=MsoNormal>So I tried to modprobe fuse but it wasn’t there.<o:p></o:p></p>
<p class=MsoNormal>Then compiled fuse using the –enable-kernel-module
option to force fuse to compile.<o:p></o:p></p>
<p class=MsoNormal>Then I compiled gluster.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Fuse on gluster server:<o:p></o:p></p>
<p class=MsoNormal>I think I’m using the fuse that came with Openfiler
2.3<o:p></o:p></p>
<p class=MsoNormal>(again, not sure how to check at this point)<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Ok, so basically everything is generally working great as
far as I can tell - no errors in logs, etc. <o:p></o:p></p>
<p class=MsoNormal>I’m running windows 2008 on Xen and all is functioning…
except one thing.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Whenever I put a decent bit of Disk I/O on the virtual
machine.. such as downloading a large file from the internet… <o:p></o:p></p>
<p class=MsoNormal>Then the virtual machine seems to hang 2-5 seconds,
continues, then hangs, then continues, then hangs… as it’s
downloading the file.<o:p></o:p></p>
<p class=MsoNormal>The whole VM doesn’t hang, however… any
operation that does not require disk access will continue to run smoothly.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>If I move the VM disk off of the gluster mount onto local
storage, then everything runs fine. Downloading a file runs quite
smoothly then.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>So I think it’s gluster/fuse/networking related (maybe)
but I don’t know how to figure it out from here.<o:p></o:p></p>
<p class=MsoNormal>I’ve played with various performance translators and
also disabling all performance translators. <o:p></o:p></p>
<p class=MsoNormal>All variations do not seem to have an impact on this issue
at all.<o:p></o:p></p>
<p class=MsoNormal>I suspect it’s something deeper or perhaps I’m
just missing something with the performance translators... <o:p></o:p></p>
<p class=MsoNormal>Anyway, I will be willing to try any and all suggestions.<o:p></o:p></p>
<p class=MsoNormal>I would really love to have gluster working – it’s
the most innovative solution to storage that I’ve found for what I’d
like to do.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Thanks!!<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Richard Williams<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>My .vol files:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal># file: /etc/glusterfs/glusterfs-client.vol<o:p></o:p></p>
<p class=MsoNormal>volume remote<o:p></o:p></p>
<p class=MsoNormal> type protocol/client<o:p></o:p></p>
<p class=MsoNormal> option transport-type tcp<o:p></o:p></p>
<p class=MsoNormal> option remote-host 64.16.220.101 # can be IP or
hostname<o:p></o:p></p>
<p class=MsoNormal> option remote-subvolume tempvm-brick<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume writebehind<o:p></o:p></p>
<p class=MsoNormal> type performance/write-behind<o:p></o:p></p>
<p class=MsoNormal> option block-size 128KB<o:p></o:p></p>
<p class=MsoNormal> subvolumes remote<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal># file: /usr/local/etc/glusterfs/glusterfs-server.vol<o:p></o:p></p>
<p class=MsoNormal>volume posix-tempvm-brick<o:p></o:p></p>
<p class=MsoNormal> type storage/posix<o:p></o:p></p>
<p class=MsoNormal> option directory
/mnt/nasblock1vg/nasblock1ext3/tempvm-brick<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume locks-tempvm-brick<o:p></o:p></p>
<p class=MsoNormal> type features/locks<o:p></o:p></p>
<p class=MsoNormal> subvolumes posix-tempvm-brick<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume tempvm-brick<o:p></o:p></p>
<p class=MsoNormal> type performance/io-threads<o:p></o:p></p>
<p class=MsoNormal> option thread-count 8<o:p></o:p></p>
<p class=MsoNormal> subvolumes locks-tempvm-brick<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>volume server<o:p></o:p></p>
<p class=MsoNormal> type protocol/server<o:p></o:p></p>
<p class=MsoNormal> option transport-type tcp<o:p></o:p></p>
<p class=MsoNormal>option auth.addr.tempvm-brick.allow <masked ip
addresses><o:p></o:p></p>
<p class=MsoNormal>subvolumes tempvm-brick<o:p></o:p></p>
<p class=MsoNormal>end-volume<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>