<html><body><div style="font-family: lucida console,sans-serif; font-size: 12pt; color: #000000"><div>No worries.<br></div><div><br></div><div>Is this something that could be prototyped in python, tp generate feedback/requirements from the community and then made more permanent in 'c'?<br></div><div><br></div><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Krishnan Parthasarathi" <kparthas@redhat.com><br><b>To: </b>"Paul Cuzner" <pcuzner@redhat.com><br><b>Cc: </b>"Vipul Nayyar" <nayyar_vipul@yahoo.com>, "Vijay Bellur" <vbellur@redhat.com>, "gluster-devel" <gluster-devel@nongnu.org><br><b>Sent: </b>Wednesday, 12 February, 2014 4:37:25 PM<br><b>Subject: </b>Re: [Gluster-devel] New project on the Forge - gstatus<br><div><br></div>Sorry for the spam, I pressed the send button too soon.<br>Note to self, your editor short-cuts don't work on your<br>email client :-(<br><div><br></div>The premise under which rrdtool integration with io-stats was<br>suggested to offset the current deficiency of io-stats. io-stats<br>logs its counters to the log file or presents it to glusterd/cli<br>via an RPC. These are not really good interfaces for reporting<br>tools, for instance glusterfsiostat.<br><div><br></div>I think we should move this discussion to another thread and give it some<br>time for more requirements to come by. Once we identify what GlusterFS<br>needs to provide in the form of statistics. How it should can<br>be determined having in mind the available set of statistics analysis<br>and collection framework.<br><div><br></div>thanks,<br>Krish <br><div><br></div>----- Original Message -----<br>> The premise under which rrdtool integration with io-stats was<br>> suggested to offset the current deficiency of io-stats. io-stats<br>> logs its counters to the log file or presents it to glusterd/cli<br>> via an RPC. These are not really good interfaces for external monitoring<br>> <br>> ----- Original Message -----<br>> > glusterfsiostat is a great idea!<br>> > <br>> > I do wonder if the inclusion of rrd in the design is adding complication<br>> > though. For example, does cifsiostat and nfsiostat do this?<br>> > <br>> > As an admin, would I not just run the glusterfsiostat command with an<br>> > interval/count - and if I want to see the stats over a time period,<br>> > couldn't<br>> > I just pipe it to a file and background the process? I could get the high<br>> > level perf ctrs for any time period that way and not be bound to the fixed<br>> > rrd file size.<br>> > <br>> > For my money - longer term time series observations don't belong in rrd,<br>> > but<br>> > should be forwarded to a "management layer" - and in that context would we<br>> > get the value out of the addtional integration work with rrd?<br>> > <br>> > Just another point of view to consider<br>> > <br>> > ----- Original Message -----<br>> > <br>> > > From: "Vipul Nayyar" <nayyar_vipul@yahoo.com><br>> > > To: "Vijay Bellur" <vbellur@redhat.com>, "Paul Cuzner"<br>> > > <pcuzner@redhat.com>,<br>> > > "gluster-devel" <gluster-devel@nongnu.org><br>> > > Cc: "Krishnan Parthasarathi" <kparthas@redhat.com><br>> > > Sent: Wednesday, 12 February, 2014 3:54:10 AM<br>> > > Subject: Re: [Gluster-devel] New project on the Forge - gstatus<br>> > <br>> > > Hello,<br>> > <br>> > > I'm Vipul, a Computer Engg. student studying in New Delhi. I have some<br>> > > past<br>> > > experience in contributing to open source and I'm interested in<br>> > > contributing<br>> > > to Gluster along with learning from the community.<br>> > <br>> > > For the past couple of weeks, I've been in constant contact with Krishnan<br>> > > Parthasarathi, regarding building a tool named glusterfsiostat, an<br>> > > nfsiostat<br>> > > clone integrated with rrdtool(a data logging system)[1]. We believe that<br>> > > storing io-stats data in a database will be a great improvement over<br>> > > dumping<br>> > > it in a log file. Since it's a Round Robin database, so it's more<br>> > > suitable<br>> > > for our cause as we'll be generating time-series data and our window size<br>> > > would also be fixed. Plus, the data can be easily accessed from the db<br>> > > and<br>> > > processed/modified with a perl/bash script according to the consumer's<br>> > > requirement.<br>> > <br>> > > As for the issue of integrating the io-stats xlator code with rrdtool,<br>> > > Krish<br>> > > asked me to explore two aspects of it. First, compiling rrdtool code in<br>> > > io-stats optionally, only when the user provides a --enable-rrdtool like<br>> > > parameter during configure, can be done similar to how<br>> > > --disable-xml-output<br>> > > option is dealt with by configure and code compiled optionally by<br>> > > checking<br>> > > certain macros defined in confdefs.h.<br>> > <br>> > > On the second note, rrdtool provides a C API which works quite similar to<br>> > > the<br>> > > rrd comand line tool. So including the rrd C api in io-stats will do the<br>> > > work of storing stats in db. As written earlier, getting/displaying the<br>> > > data<br>> > > would just be a simple task of querying the rrd database.<br>> > <br>> > > Although I've spent some considerable time studying the io-stats code and<br>> > > the<br>> > > data structures being used, I think starting to work on a prototype along<br>> > > with everyone's criticism will help me a lot. Is there anywhere written,<br>> > > exactly what data is currently being logged and dumped in the log file ??<br>> > > This will help me in identifying the important data structures and place<br>> > > the<br>> > > rrdtool code in the proper place, where it needs to be.<br>> > <br>> > > I know the draft above, seems quite simple and maybe it doesn't cover too<br>> > > many aspects that need to be dealt with beforehand, but that's where as<br>> > > an<br>> > > amateur contributor, I need the community's help.<br>> > <br>> > > I'll send a patch soon, your way, if you think that the direction in<br>> > > which<br>> > > I'm planning to tread is good for the community.<br>> > <br>> > > [1] http://oss.oetiker.ch/rrdtool/<br>> > <br>> > > Hoping to hear your views soon.<br>> > <br>> > > Regards<br>> > > Vipul Nayyar<br>> > <br>> > > On Monday, 10 February 2014 8:29 PM, Vijay Bellur <vbellur@redhat.com><br>> > > wrote:<br>> > > On 02/10/2014 02:00 AM, Paul Cuzner wrote:<br>> > > ><br>> > > > Hi,<br>> > > ><br>> > > > I've started a new project on the forge, called gstatus.- wiki page is<br>> > > > https://forge.gluster.org/gstatus/pages/Home<br>> > > ><br>> > > > The idea is to provide admins with a single command to assess the state<br>> > > > of the components of a cluster - nodes, bricks and volume states -<br>> > > > together with capacity information.<br>> > > ><br>> > > > It's the kind of feature that would be great (IMO) as a sub command of<br>> > > > gluster i.e. gluster status - but as a stop gap here's the python<br>> > > > project (we could even use this as a prototype!)<br>> > > ><br>> > > > On the wiki page, you'll find some additional volume status definitions<br>> > > > that I've dreamt up - online-degraded, online-partial, to describe the<br>> > > > effect brick down events have on a volume's data availability. There<br>> > > > are<br>> > > > output examples on the wiki, but here's some examples to show you what<br>> > > > you currently get from the tool<br>> > > ><br>> > > > On my test 4-way cluster, this is what a healthy state looks like<br>> > > ><br>> > > > [ root@rhs1-1 gstatus]# ./gstatus.py<br>> > > > Analysis complete<br>> > > ><br>> > > > Cluster Summary:<br>> > > > Version - 3.4.0.44rhs Nodes - 4/ 4 Bricks - 4/ 4 Volumes - 1/ 1<br>> > > ><br>> > > > Volume Summary<br>> > > > myvol ONLINE (4/4 bricks online) - Distributed-Replicate<br>> > > > Capacity: 64.53 MiB/19.97 GiB (used,total)<br>> > > ><br>> > > > Status Messages<br>> > > > Cluster is healthy, all checks successful<br>> > > ><br>> > > > And then if I take a *two nodes* down, that provide bricks to the *same<br>> > > > replica set*, I see;<br>> > > ><br>> > > > Analysis complete<br>> > > ><br>> > > ><br>> > > > Cluster Summary:<br>> > > > Version - 3.4.0.44rhs Nodes - 2/ 4 Bricks - 2/ 4 Volumes - 0/ 1<br>> > > ><br>> > > > Volume Summary<br>> > > > myvol ONLINE_PARTIAL (2/4 bricks online) - Distributed-Replicate<br>> > > > Capacity: 32.27 MiB/9.99 GiB (used,total)<br>> > > ><br>> > > ><br>> > > > Status Messages<br>> > > > - rhs1-4 is down<br>> > > > - rhs1-2 is down<br>> > > > - Brick rhs1-4:/gluster/brick1 is down/unavailable<br>> > > > - Brick rhs1-2:/gluster/brick1 is down/unavailable<br>> > > ><br>> > > ><br>> > > ><br>> > <br>> > > This is great!<br>> > <br>> > > I think adding one more for the client stack would be neat. A tool<br>> > > similar to nfsstat/nfsiostat which can expose various counters in<br>> > > iostats xlator and also status information like brick connectivity from<br>> > > the client perspective. I also have a cool name for that - glusteriostat<br>> > > ;)<br>> > <br>> > > Cheers,<br>> > > Vijay<br>> > <br>> > > _______________________________________________<br>> > > Gluster-devel mailing list<br>> > > Gluster-devel@nongnu.org<br>> > > https://lists.nongnu.org/mailman/listinfo/gluster-devel<br>> > <br>> <br></blockquote><div><br></div></div></body></html>