<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, Sans-Serif;font-size:10pt"><div class="" style="">Hello,</div><div class="" style=""><br></div><div class="" style="">Basing my first patch to Gluster as a stepping stone, I've written a small utility glusterfsiostat, in python which you can find attached with this email. Currently, the modifications done by my patch to io-stats which is under review as of now, dumps private information from the xlator object to the proper file for private info in the meta directory. This includes total bytes read/written along with read/write speed in the previous 10 seconds. The speed at every 1 second is identified by it's respective unix timestamp and hence given out in bytes/second. These values at discrete points of time can be used to generate a graph. </div><div class="" style=""><br class="" style=""></div><div class=""
style="">The python tool first identifies all gluster mounts in the system, identifies the mount path and parses the meta xlator output in order to generate output similar to the iostat tool. Passing '-j' option gives you extra information in a consumable json format. By default, the tool pretty prints the basic stats which are human readable. This tool is supposed to be a framework on which other applications can be built upon. I'm putting this out for community feedback so as to improve it further.</div><div class="" style=""><br class="" style=""></div><div class="" style=""><span class="" style=""></span></div><div class="" style="">Do you think the stats and the way they're generated as of now is something that might be usable by someone? Any other implementation suggestions or addition of some more stats that I can/should possibly provide with the utility?</div><div class="" style=""><br class="" style=""></div><div class="" style="color: rgb(0,
0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;">Note: In order to test this, you need to apply my patch(<a href="http://review.gluster.org/#/c/8030/" style="font-size: 10pt;" class="">http://review.gluster.org/#/c/8030/</a><span style="background-color: transparent;" class="">) in your repo first, build and then mount a volume. Preferably perform a big read/write operation with a file on your Gluster mount before executing the python script. Then run it as 'python stat.py' or 'python stat.py -j'</span></div><div class="" style=""></div><div class="" style=""><br class="" style=""></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial, helvetica, clean, sans-serif;background-color:transparent;font-style:normal;" class="">Regards</div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial, helvetica, clean,
sans-serif;background-color:transparent;font-style:normal;" class=""><span style="background-color:transparent;line-height:1.22;" class="">Vipul Nayyar </span></div> <div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" style="display: block;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, Sans-Serif; font-size: 10pt;" class=""> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;" class=""> <div dir="ltr" class="" style=""> <font size="2" face="Arial" class="" style=""> On Sunday, 8 June 2014 8:51 PM, Vipul Nayyar <nayyar_vipul@yahoo.com> wrote:<br class="" style=""> </font> </div> <br class="" style=""><br class="" style=""> <div class="" style=""><div id="yiv1349009113" class="" style=""><div class="" style=""><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande,
Sans-Serif;font-size:10pt;" class=""><div class="" style=""><span class="" style="">Hi,</span></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;background-color:transparent;font-style:normal;" class=""><span class="" style=""><br clear="none" class="" style=""></span></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;background-color:transparent;font-style:normal;" class=""><span class="" style="">If I have a say in this decision, then I'd like to go with the way of keeping the io-stats xlator in it's place with the addition of dumpops. Or atleast postpone duplicating the functionality of io-stats in latency.c. I think focusing on my framework without the worry of a major upheaval in the xlator world would help me better in achieving my
goals.</span></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial, helvetica, clean, sans-serif;background-color:transparent;font-style:normal;" class=""><br clear="none" class="" style=""></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial, helvetica, clean, sans-serif;background-color:transparent;font-style:normal;" class="">Regards</div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:arial, helvetica, clean, sans-serif;background-color:transparent;font-style:normal;" class=""><span style="background-color:transparent;line-height:1.22;" class="">Vipul Nayyar </span></div><div style="color:rgb(0, 0, 0);font-size:12.727272033691406px;font-family:arial, helvetica, clean, sans-serif;background-color:transparent;font-style:normal;" class=""><br clear="none" class="" style=""></div> <div class="" style=""><br clear="none" class="" style=""><br clear="none" class="" style=""></div><div class=""
id="yiv1349009113yqt19279" style=""><div class="" style="display:block;"> <div style="font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;font-size:10pt;" class=""> <div style="font-family:'times new roman', 'new york', times, serif;font-size:12pt;" class=""> <div dir="ltr" class="" style=""> <font size="2" face="Arial" class="" style=""> On Saturday, 7 June 2014 1:05 AM, Anand Avati <avati@gluster.org> wrote:<br clear="none" class="" style=""> </font> </div> <br clear="none" class="" style=""><br clear="none" class="" style=""> <div class="" style=""><div id="yiv1349009113" class="" style=""><div class="" style=""><div dir="ltr" class="" style=""><br clear="none" class="" style=""><div class="" style=""><br clear="none" class="" style=""><div class="" style="">On Fri, Jun 6, 2014 at 10:13 AM, Vipul Nayyar <span dir="ltr" class="" style=""><<a rel="nofollow" shape="rect"
ymailto="mailto:nayyar_vipul@yahoo.com" target="_blank" href="mailto:nayyar_vipul@yahoo.com" class="" style="">nayyar_vipul@yahoo.com</a>></span> wrote:<br clear="none" class="" style="">
<blockquote class="" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="" style=""><div style="color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;font-size:10pt;" class="">
<div class="" style=""><span class="" style="">Hello,</span></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;background-color:transparent;font-style:normal;" class="">
<span class="" style=""><br clear="none" class="" style=""></span></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;background-color:transparent;font-style:normal;" class=""><span class="" style="">I'm Vipul and I'll be working on a tool called glusterfsiostat under GSOC this summer with KP as my mentor. Based on our discussion, the plan for the future is to build an initial working version of the tool in python and improve it later based on feedback. This tool will display i/o information about every glusterfs mount in the system. The primary
source for getting stats would be the .meta folder accessible in every mount. As of now, the meta xlator is a good resource to get the hierarchy and basic information about every translator being used in a mount. But in terms of statistics, it only shows latency of every FOP for each xlator. </span></div>
<div style="color:rgb(0, 0, 0);font-size:13px;font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;background-color:transparent;font-style:normal;" class=""><span class="" style=""><br clear="none" class="" style=""></span></div><div style="color:rgb(0, 0, 0);font-size:13px;font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;background-color:transparent;font-style:normal;" class="">
<span class="" style="">Since, our aim is to provide information similar to the tool nfsiostat, we're hungry for more information, which is available in the private data structures maintained by io-stats. One way that I see to achieve this is to define a
dumpops structure in io-stats and a priv function in it, which can be used to dump custom info into the private file in the .meta folder. If you feel that there's a better way for doing this, please do guide me.</span></div>
</div></div></blockquote><div class="" style=""><br clear="none" class="" style=""></div><div class="" style="">You could do that, or enhance the latency capturing functionality with more stats, essentially making io-stats xlator redundant on the client side.</div><div class="" id="yiv1349009113yqtfd15662" style=""><div class="" style=""> </div><blockquote class="" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="" style=""><div style="color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;font-size:10pt;" class=""><div style="color:rgb(0, 0, 0);font-size:13px;font-family:HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif;background-color:transparent;font-style:normal;" class="">
<span class="" style="">On a second note, the tool would provide stats in properly formatted manner by default for human consumption and also in json format if it is needed by any other application.</span></div></div></div></blockquote></div><div class="" style="">
<br clear="none" class="" style=""></div><div class="" style="">That sounds good.</div><div class="" style=""><br clear="none" class="" style=""></div><div class="" style="">Thanks</div></div></div></div></div></div><br clear="none" class="" style=""><br clear="none" class="" style=""></div> </div> </div> </div></div> </div></div></div><br class="" style=""><br class="" style=""></div> </div> </div> </div> </div></body></html>