phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey Zhong <jzh...@hortonworks.com>
Subject Re: monitoring
Date Tue, 01 Jul 2014 22:39:24 GMT

There is a plan to drop
hadoop1(https://issues.apache.org/jira/browse/PHOENIX-103).  So only
supporting hadoop2 is good enough.

-Jeffrey

From:  James Taylor <jamestaylor@apache.org>
Reply-To:  <user@phoenix.apache.org>
Date:  Tuesday, July 1, 2014 11:47 AM
To:  "user@phoenix.apache.org" <user@phoenix.apache.org>
Subject:  Re: monitoring

+2 Awesome job, Jesse!

On Tuesday, July 1, 2014, Eli Levine <elilevine@gmail.com> wrote:
> +1 for only supporting Hadoop2 initially (great work, Jesse!). It's important
> that Phoenix started implementing monitoring and metrics features. An
> incremental approach is the way to go IMHO.
> 
> Thanks,
> 
> Eli
> 
> 
> On Tue, Jul 1, 2014 at 11:13 AM, James Taylor <jamestaylor@apache.org
> <javascript:_e(%7B%7D,'cvml','jamestaylor@apache.org');> > wrote:
>> Seems like an excellent feature to have in 4.1 which I'm hoping we can
>> do by end of the month. I'd rather the feature make it in and only
>> support hadoop2 than have it not make the release.
>> 
>> Any one else have thoughts on this?
>> 
>> On Tue, Jul 1, 2014 at 8:01 PM, Jesse Yates <jesse.k.yates@gmail.com
>> <javascript:_e(%7B%7D,'cvml','jesse.k.yates@gmail.com');> > wrote:
>>> > That's certainly possible, though somewhat unsatisfying :) Its not
>>> terribly
>>> > difficult to the Hadoop1 code - I've modularized a majority of the code
so
>>> > all that is really needed is replicating the reading of the Hadoop1
>>> metrics2
>>> > classes to something the phoenix writer understands (just like in the
>>> > Hadoop2 impl).
>>> >
>>> > Happy to push the code somewhere so people can take a look... or they just
>>> > wait a couple weeks :)
>>> >
>>> > -------------------
>>> > Jesse Yates
>>> > @jesse_yates
>>> > jyates.github.com <http://jyates.github.com>
>>> >
>>> >
>>> > On Tue, Jul 1, 2014 at 10:53 AM, James Taylor <jamestaylor@apache.org
>>> <javascript:_e(%7B%7D,'cvml','jamestaylor@apache.org');> >
>>> > wrote:
>>>> >>
>>>> >> How about if we make it a hadoop2 only feature?
>>>> >>
>>>> >>
>>>> >> On Tuesday, July 1, 2014, Jesse Yates <jesse.k.yates@gmail.com
>>>> <javascript:_e(%7B%7D,'cvml','jesse.k.yates@gmail.com');> > wrote:
>>>>> >>>
>>>>> >>> I was working on a patch to support using Cloudera's HTrace
>>>>> >>> (https://github.com/cloudera/htrace) library for phoenix
queries.
>>>>> There was
>>>>> >>> a preliminary patch for the 2.X series, but it never got
committed.
Two
>>>>> >>> weeks ago I almost finished porting it to the 4.X line,
but haven't
>>>>> finished
>>>>> >>> the piece that will support Hadoop1*. I can't seem to find
the JIRA
>>>>> right
>>>>> >>> now though...
>>>>> >>>
>>>>> >>> I've gotten sidetracked on some internal work, so probably
not going
to
>>>>> >>> have time to wrap it up for a couple weeks.
>>>>> >>>
>>>>> >>> -J
>>>>> >>>
>>>>> >>> * It uses the Hadoop metrics2 library to collect the traces
and then
has
>>>>> >>> a default implementation (as a metrics sink) that writes
them to a
>>>>> phoenix
>>>>> >>> table so it can be analyzed later. Because of the changes
between
>>>>> Hadoop1
>>>>> >>> and 2, there is some reflection funkiness that is necessary
to support
>>>>> both,
>>>>> >>> but just haven't finished the hadoop1 side of it.
>>>>> >>> -------------------
>>>>> >>> Jesse Yates
>>>>> >>> @jesse_yates
>>>>> >>> jyates.github.com <http://jyates.github.com>
>>>>> >>>
>>>>> >>>
>>>>> >>> On Tue, Jul 1, 2014 at 9:29 AM, Stephen Sprague <spragues@gmail.com
>>>>> <javascript:_e(%7B%7D,'cvml','spragues@gmail.com');> >
>>>>> >>> wrote:
>>>>>> >>>>
>>>>>> >>>> following up to my own message i did see reference
to something
>>>>>> called
>>>>>> >>>> "Hannibal" (https://github.com/sentric/hannibal)
in the Phoenix doc.
>>>>>> Looks
>>>>>> >>>> like this helps with identifying imbalance within
your cluster.
>>>>>> >>>>
>>>>>> >>>> I'll keep searching around though - a Phoenix log
file looks like
>>>>>> >>>> perhaps the best way to monitor phoenix queries
at this point as i
>>>>>> don't
>>>>>> >>>> think there is any self-aware queries to ask of
it.  That's a jdbc
>>>>>> driver
>>>>>> >>>> issue anyway - not phoenix.
>>>>>> >>>>
>>>>>> >>>> Cheers,
>>>>>> >>>> Stephen.
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> On Mon, Jun 30, 2014 at 10:17 PM, Stephen Sprague
>>>>>> <spragues@gmail.com <javascript:_e(%7B%7D,'cvml','spragues@gmail.com');>
>
>>>>>> >>>> wrote:
>>>>>>> >>>>>
>>>>>>> >>>>> Hi guys,
>>>>>>> >>>>>
>>>>>>> >>>>> May i ask what tools you use to monitor
(active) Phoenix queries?
>>>>>>> >>>>>
>>>>>>> >>>>> looking at the web ui for hbase master and
region servers it isn't
>>>>>>> >>>>> exactly intuitive as to what *phoenix* is
doing.  Its probably a
>>>>>>> stretch to
>>>>>>> >>>>> ask if there's something like "show processlist"
(ala mysql) out
there
>>>>>>> >>>>> somewhere.
>>>>>>> >>>>>
>>>>>>> >>>>> so is there a log file with metrics in it
that perhaps i could
>>>>>>> tail or
>>>>>>> >>>>> analyze in realtime?   perhaps even feed
it back into hbase? :)
>>>>>>> hey, wait a
>>>>>>> >>>>> sec, anything in those system tables that
is realtime-ish?
>>>>>>> >>>>>
>>>>>>> >>>>> regardless, kinda curious how people monitor
the Phoenix plant.
>>>>>>> >>>>>
>>>>>>> >>>>> thanks,
>>>>>>> >>>>> Stephen.
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>> >>>>
>>>>> >>>
>>> >
> 



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message