<div dir="ltr">Hello all!<div><br></div><div>This problem is solved by me today.</div><div style>Root of all in the incompatibility of gluster cache and kvm cache.</div><div style><br></div><div style>Bug reproduces if KVM virtual machine created with cache=writethrough (default for OpenStack) option and hosted on GlusterFS volume. If any other (cache=writeback or cache=none with direct-io) cacher used, performance of writing to existing file inside VM is equal to bare storage (from host machine) write performance.</div>
<div style><br></div><div style>I think, it must be documented in Gluster and maybe filled a bug.</div><div style><br></div><div style>Other question. Where I can read something about gluster tuning (optimal cache size, write-behind, flush-behind use cases and other)? I found only options list, without any how-to or tested cases.</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/3/5 Toby Corkindale <span dir="ltr"><<a href="mailto:toby.corkindale@strategicdata.com.au" target="_blank">toby.corkindale@strategicdata.com.au</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 01/03/13 21:12, Brian Candler wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Fri, Mar 01, 2013 at 03:30:07PM +0600, Nikita A Kardashin wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
If I try to execute above command inside virtual machine (KVM), first<br>
time all going right - about 900MB/s (cache effect, I think), but if I<br>
run this test again on existing file - task (dd) hungs up and can be<br>
stopped only by Ctrl+C.<br>
Overall virtual system latency is poor too. For example, apt-get<br>
upgrade upgrading system very, very slow, freezing on "Unpacking<br>
replacement" and other io-related steps.<br>
Does glusterfs have any tuning options, that can help me?<br>
</blockquote>
<br>
If you are finding that processes hang or freeze indefinitely, this is not<br>
a question of "tuning", this is simply "broken".<br>
<br>
Anyway, you're asking the wrong person - I'm currently in the process of<br>
stripping out glusterfs, although I remain interested in the project.<br>
<br>
I did find that KVM performed very poorly, but KVM was not my main<br>
application and that's not why I'm abandoning it. I'm stripping out<br>
glusterfs primarily because it's not supportable in my environment, because<br>
there is no documentation on how to analyse and recover from failure<br>
scenarios which can and do happen. This point in more detail:<br>
<a href="http://www.gluster.org/pipermail/gluster-users/2013-January/035118.html" target="_blank">http://www.gluster.org/<u></u>pipermail/gluster-users/2013-<u></u>January/035118.html</a><br>
<br>
The other downside of gluster was its lack of flexibility, in particular the<br>
fact that there is no usage scaling factor on bricks, so that even with a<br>
simple distributed setup all your bricks have to be the same size. Also,<br>
the object store feature which I wanted to use, has clearly had hardly any<br>
testing (even the RPM packages don't install properly).<br>
<br>
I *really* wanted to deploy gluster, because in principle I like the idea of<br>
a virtual distribution/replication system which sits on top of existing<br>
local filesystems. But for storage, I need something where operational<br>
supportability is at the top of the pile.<br>
</blockquote>
<br></div></div>
I have to agree; GlusterFS has been in use here in production for a while, and while it mostly works, it's been fragile and documentation has been disappointing. Despite 3.3 being in beta for a year, it still seems to have been poorly tested. For eg, I can't believe almost no-one else noticed that the log files were busted.. nor that the bug report has been around for quarter of a year without being responded to or fixed.<br>
<br>
I have to ask -- what are you moving to now, Brian?<span class="HOEnZb"><font color="#888888"><br>
<br>
-Toby</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
______________________________<u></u>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.<u></u>org/mailman/listinfo/gluster-<u></u>users</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>With best regards,<br>differentlocal (<a href="http://www.differentlocal.ru">www.differentlocal.ru</a> | <a href="mailto:differentlocal@gmail.com">differentlocal@gmail.com</a>),<br>
System administrator.
</div>