phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: monitoring
Date Tue, 01 Jul 2014 18:47:54 GMT
+2 Awesome job, Jesse!

On Tuesday, July 1, 2014, Eli Levine <elilevine@gmail.com> wrote:

> +1 for only supporting Hadoop2 initially (great work, Jesse!). It's
> important that Phoenix started implementing monitoring and metrics
> features. An incremental approach is the way to go IMHO.
>
> Thanks,
>
> Eli
>
>
> On Tue, Jul 1, 2014 at 11:13 AM, James Taylor <jamestaylor@apache.org
> <javascript:_e(%7B%7D,'cvml','jamestaylor@apache.org');>> wrote:
>
>> Seems like an excellent feature to have in 4.1 which I'm hoping we can
>> do by end of the month. I'd rather the feature make it in and only
>> support hadoop2 than have it not make the release.
>>
>> Any one else have thoughts on this?
>>
>> On Tue, Jul 1, 2014 at 8:01 PM, Jesse Yates <jesse.k.yates@gmail.com
>> <javascript:_e(%7B%7D,'cvml','jesse.k.yates@gmail.com');>> wrote:
>> > That's certainly possible, though somewhat unsatisfying :) Its not
>> terribly
>> > difficult to the Hadoop1 code - I've modularized a majority of the code
>> so
>> > all that is really needed is replicating the reading of the Hadoop1
>> metrics2
>> > classes to something the phoenix writer understands (just like in the
>> > Hadoop2 impl).
>> >
>> > Happy to push the code somewhere so people can take a look... or they
>> just
>> > wait a couple weeks :)
>> >
>> > -------------------
>> > Jesse Yates
>> > @jesse_yates
>> > jyates.github.com
>> >
>> >
>> > On Tue, Jul 1, 2014 at 10:53 AM, James Taylor <jamestaylor@apache.org
>> <javascript:_e(%7B%7D,'cvml','jamestaylor@apache.org');>>
>> > wrote:
>> >>
>> >> How about if we make it a hadoop2 only feature?
>> >>
>> >>
>> >> On Tuesday, July 1, 2014, Jesse Yates <jesse.k.yates@gmail.com
>> <javascript:_e(%7B%7D,'cvml','jesse.k.yates@gmail.com');>> wrote:
>> >>>
>> >>> I was working on a patch to support using Cloudera's HTrace
>> >>> (https://github.com/cloudera/htrace) library for phoenix queries.
>> There was
>> >>> a preliminary patch for the 2.X series, but it never got committed.
>> Two
>> >>> weeks ago I almost finished porting it to the 4.X line, but haven't
>> finished
>> >>> the piece that will support Hadoop1*. I can't seem to find the JIRA
>> right
>> >>> now though...
>> >>>
>> >>> I've gotten sidetracked on some internal work, so probably not going
>> to
>> >>> have time to wrap it up for a couple weeks.
>> >>>
>> >>> -J
>> >>>
>> >>> * It uses the Hadoop metrics2 library to collect the traces and then
>> has
>> >>> a default implementation (as a metrics sink) that writes them to a
>> phoenix
>> >>> table so it can be analyzed later. Because of the changes between
>> Hadoop1
>> >>> and 2, there is some reflection funkiness that is necessary to
>> support both,
>> >>> but just haven't finished the hadoop1 side of it.
>> >>> -------------------
>> >>> Jesse Yates
>> >>> @jesse_yates
>> >>> jyates.github.com
>> >>>
>> >>>
>> >>> On Tue, Jul 1, 2014 at 9:29 AM, Stephen Sprague <spragues@gmail.com
>> <javascript:_e(%7B%7D,'cvml','spragues@gmail.com');>>
>> >>> wrote:
>> >>>>
>> >>>> following up to my own message i did see reference to something
>> called
>> >>>> "Hannibal" (https://github.com/sentric/hannibal) in the Phoenix
>> doc.  Looks
>> >>>> like this helps with identifying imbalance within your cluster.
>> >>>>
>> >>>> I'll keep searching around though - a Phoenix log file looks like
>> >>>> perhaps the best way to monitor phoenix queries at this point as
i
>> don't
>> >>>> think there is any self-aware queries to ask of it.  That's a jdbc
>> driver
>> >>>> issue anyway - not phoenix.
>> >>>>
>> >>>> Cheers,
>> >>>> Stephen.
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Mon, Jun 30, 2014 at 10:17 PM, Stephen Sprague <
>> spragues@gmail.com <javascript:_e(%7B%7D,'cvml','spragues@gmail.com');>>
>> >>>> wrote:
>> >>>>>
>> >>>>> Hi guys,
>> >>>>>
>> >>>>> May i ask what tools you use to monitor (active) Phoenix queries?
>> >>>>>
>> >>>>> looking at the web ui for hbase master and region servers it
isn't
>> >>>>> exactly intuitive as to what *phoenix* is doing.  Its probably
a
>> stretch to
>> >>>>> ask if there's something like "show processlist" (ala mysql)
out
>> there
>> >>>>> somewhere.
>> >>>>>
>> >>>>> so is there a log file with metrics in it that perhaps i could
tail
>> or
>> >>>>> analyze in realtime?   perhaps even feed it back into hbase?
:)
>>  hey, wait a
>> >>>>> sec, anything in those system tables that is realtime-ish?
>> >>>>>
>> >>>>> regardless, kinda curious how people monitor the Phoenix plant.
>> >>>>>
>> >>>>> thanks,
>> >>>>> Stephen.
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >
>>
>
>

Mime
View raw message