<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I would want tests of all the posix operations. Need a difference
not just in throughput, but in max iops for the various ops.<br>
<br>
<div class="moz-cite-prefix">On 07/27/2014 08:27 AM, Vipul Nayyar
wrote:<br>
</div>
<blockquote
cite="mid:1406474844.38055.YahooMailNeo@web194902.mail.sg3.yahoo.com"
type="cite">
<div style="color:#000; background-color:#fff;
font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial,
Lucida Grande, Sans-Serif;font-size:10pt">
<div class="" style=""><span class="" style="">Hi</span></div>
<div class="" style="color: rgb(0, 0, 0); font-size: 13px;
font-family: HelveticaNeue, 'Helvetica Neue', Helvetica,
Arial, 'Lucida Grande', sans-serif; background-color:
transparent; font-style: normal;"><span class="" style=""><br
class="" style="">
</span></div>
<div class="" style="color: rgb(0, 0, 0); font-size: 13px;
font-family: HelveticaNeue, 'Helvetica Neue', Helvetica,
Arial, 'Lucida Grande', sans-serif; background-color:
transparent; font-style: normal;"><span class="" style="">As
guided by you, I performed the experiment regarding
measurement of the effect of always enabled profiling. I
performed two write tests, one with a 20 MB file and the
other with a 730 MB file. Each file was written 20 times to
the mounted volume after clearing the buffers on every
iteration and the time taken measured with the time command.
I ran the following bash script for this purpose.</span></div>
<div class="" style="color: rgb(0, 0, 0); font-size: 13px;
font-family: HelveticaNeue, 'Helvetica Neue', Helvetica,
Arial, 'Lucida Grande', sans-serif; background-color:
transparent; font-style: normal;"><br>
</div>
<div class="" style="background-color: transparent;">i=1</div>
<div class="" style="background-color: transparent;">while [[ $i
-lt 21 ]]; do</div>
<div class="" style="background-color: transparent;"><span
class="" style="white-space:pre"> </span>sync &&
echo 3 > /proc/sys/vm/drop_caches</div>
<div class="" style="background-color: transparent;"><span
class="" style="white-space:pre"> </span>path="/mnt/write_test"$i</div>
<div class="" style="background-color: transparent;"><span
class="" style="white-space:pre"> </span>out=$( time cp
/home/vipul/test.avi $path)</div>
<div class="" style="background-color: transparent;"><span
class="" style="background-color: transparent; font-size:
10pt; white-space: pre;"> </span><span
style="background-color: transparent; font-size: 10pt;">i=$((i+1))</span><br>
</div>
<div class="" style="background-color: transparent;"><span
class="" style=""></span></div>
<div class="" style="background-color: transparent;">done</div>
<div class="" style="background-color: transparent;"><br>
</div>
<div class="" style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida Grande',
sans-serif; font-style: normal;">Since the values at different
times for writing the same file are quite varied, I plotted a
graph using the obtained values(Y-axis represents seconds)
which can be found attached. As you might see in these images,
there is no clear pattern found in the variation of values
obtained while writing.</div>
<div class="" style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida Grande',
sans-serif; font-style: normal;"><br>
</div>
<div class="" style="background-color: transparent; color:
rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida Grande',
sans-serif; font-style: normal;">So according to me, values in
both the conditions are quite near to each other and equally
capable of going quite high or low than the mean value and
hence, there is no negative effect seen due to the change
proposed. I hope someone else can shed more light on whether
setting the option(always enabled profiling) really decreased
the performance or not.</div>
<div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial,
helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;"
class=""><br class="" style="">
</div>
<div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial,
helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;"
class="">Regards</div>
<div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial,
helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;"
class=""><span
style="background-color:transparent;line-height:1.22;"
class="">Vipul Nayyar </span></div>
<div style="color:rgb(0, 0,
0);font-size:12.727272033691406px;font-family:arial,
helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;"
class=""><br class="" style="">
</div>
<div class="qtdSeparateBR"><br>
<br>
</div>
<div class="yahoo_quoted" style="display: block;">
<div style="font-family: HelveticaNeue, Helvetica Neue,
Helvetica, Arial, Lucida Grande, Sans-Serif; font-size:
10pt;" class="">
<div style="font-family: times new roman, new york, times,
serif; font-size: 12pt;" class="">
<div dir="ltr" class="" style=""> <font class="" style=""
face="Arial" size="2"> On Wednesday, 16 July 2014
12:50 PM, Krishnan Parthasarathi
<a class="moz-txt-link-rfc2396E" href="mailto:kparthas@redhat.com"><kparthas@redhat.com></a> wrote:<br class=""
style="">
</font> </div>
<br class="" style="">
<br class="" style="">
<div class="" style="">
<div id="yiv9374482142" class="" style="">
<div class="" style="">
<div style="font-family:times new roman, new york,
times, serif;font-size:12pt;color:#000000;"
class="">
<div class="" style="">Vipul,<br class="" style=""
clear="none">
</div>
<div class="" style=""><br class="" style=""
clear="none">
</div>
<hr id="yiv9374482142zwchr" class="" style="">
<blockquote style="border-left:2px solid
#1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,
Arial, sans-serif;font-size:12pt;" class="">
<div
style="color:#000;background-color:#fff;font-family:HelveticaNeue,
Helvetica Neue, Helvetica, Arial, Lucida
Grande, Sans-Serif;font-size:10pt;" class="">
<div class="" style=""><span class="" style="">Hello,</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style=""><br class="" style=""
clear="none">
</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style="">Following is a proposal
for modifying the io profiling capability
of the io-stats xlator. I recently sent in
a patch(review.gluster.org/#/c/8244/)
regarding that, which uses the already
written latency related functions in
io-stats to dump info through meta and
added some more data containers which
would track some more fops related info
each time a request goes through io-stats.
Currently, before the io-stats' custom
latency functions can run, the
measure_latency and count_fop_hits option
should be enabled. I propose to remove
these two options entirely from io-stats.</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style=""><br class="" style=""
clear="none">
</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style="">In order to track io
performance, these options should be
enabled all the time, or removed entirely,
so that a record of io requests can be
kept since mount time, else enabling these
options only when it is required will not
give you the average statistics over the
whole period since the start. This is
based on the methodology of Linux kernel
itself, since it internally maintains the
io statistics data structures all the time
and presents it via /proc filesystem
whenever required. Enabling of any option
is not required, and the data available
represents statistics since the boot time.</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style=""><br class="" style=""
clear="none">
</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style="">I would like to know the
views over this, if having io-stats
profiling info available all the time
would be a good thing?</span></div>
</div>
</blockquote>
<div class="" style="">Could you run the following
experiment to measure the effect of profiling
being enabled always?<br class="" style=""
clear="none">
</div>
<div class="" style="">- Fix the I/O workload to
be run.<br class="" style="" clear="none">
</div>
<div class="" style="">- Setup 1 (control group) :
Run the fixed workload on a volume with both the
profiling options NOT set.<br class="" style=""
clear="none">
</div>
<div class="" style="">- Setup 2 : Run the (same)
fixed workload on the same volume with the
profiling options set.<br class="" style=""
clear="none">
</div>
<div class="" style="">- In both setup, measure
the latencies observed by the said workload. You
could use time(1) command<br class="" style=""
clear="none">
</div>
<div class="" style=""> for a crude measurement.<br
class="" style="" clear="none">
</div>
<div class="" style=""><br class="" style=""
clear="none">
This should allow us to make an informed
decision on whether there is any performance
effect<br class="" style="" clear="none">
</div>
<div class="" style="">when profiling is enabled
on a volume by default.<br class="" style=""
clear="none">
</div>
<blockquote style="border-left:2px solid
#1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,
Arial, sans-serif;font-size:12pt;" class="">
<div
style="color:#000;background-color:#fff;font-family:HelveticaNeue,
Helvetica Neue, Helvetica, Arial, Lucida
Grande, Sans-Serif;font-size:10pt;" class="">
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style=""><br class="" style=""
clear="none">
</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial, 'Lucida
Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style="">Apart from this, I was
going over latency.c in libglusterfs,
which does a fine job of maintaining
latency info for every xlator and
encountered an anomaly which I thought
should be dealt with. The
function gf_proc_dump_latency_info which
dumps the latency array for the specified
xlator consists of a last line which in
the end flushes this array through memset
after every dump. That means, you get
different latency info every time you read
the profile file in meta. I think,
flushing the data structure after every
dump is wrong since, you don't get overall
stats since one enabled the option at the
top of meta, and more importantly,
multiple applications reading this file
can get wrong info, since it gets cleared
after one read only.</span></div>
</div>
</blockquote>
<div class="" style="">Clearing of the statistics
on every request sounds incorrect to me. Could
you please send a patch to fix this?<br class=""
style="" clear="none">
</div>
<div class="" style=""><br class="" style=""
clear="none">
</div>
<div class="" style="">thanks,<br class=""
style="" clear="none">
</div>
<div class="" style="">Krish
<div class="" id="yiv9374482142yqtfd37805"
style=""><br class="" style="" clear="none">
</div>
</div>
<div class="" id="yiv9374482142yqtfd95429"
style="">
<blockquote style="border-left:2px solid
#1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,
Arial, sans-serif;font-size:12pt;" class="">
<div
style="color:#000;background-color:#fff;font-family:HelveticaNeue,
Helvetica Neue, Helvetica, Arial, Lucida
Grande, Sans-Serif;font-size:10pt;" class="">
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial,
'Lucida Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style=""><br class="" style=""
clear="none">
</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial,
'Lucida Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
class="" style="">If my reasons seem apt
for you, I'll send a patch over for
evaluation.</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:HelveticaNeue,
'Helvetica Neue', Helvetica, Arial,
'Lucida Grande',
sans-serif;background-color:transparent;font-style:normal;"><span
style="background-color:transparent;font-family:arial,
helvetica, clean, sans-serif;" class=""><br
class="" style="" clear="none">
</span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:arial,
helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;"><span
style="background-color:transparent;font-family:arial,
helvetica, clean, sans-serif;" class="">Regards</span><br
class="" style="" clear="none">
</div>
<div class="" style="color:rgb(0, 0,
0);font-size:13px;font-family:arial,
helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;"><span
class=""
style="background-color:transparent;line-height:1.22;">Vipul
Nayyar </span></div>
<div class="" style="color:rgb(0, 0,
0);font-size:12.727272033691406px;font-family:arial,
helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;"><br
class="" style="" clear="none">
</div>
</div>
</blockquote>
<div class="" style=""><br class="" style=""
clear="none">
</div>
</div>
</div>
</div>
</div>
<br class="" style="">
<br class="" style="">
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel">http://supercolony.gluster.org/mailman/listinfo/gluster-devel</a></pre>
</blockquote>
<br>
</body>
</html>