There is a plan to drop hadoop1( So only supporting hadoop2 is good enough.


From: James Taylor <>
Reply-To: <>
Date: Tuesday, July 1, 2014 11:47 AM
To: "" <>
Subject: Re: monitoring

+2 Awesome job, Jesse!

On Tuesday, July 1, 2014, Eli Levine <> wrote:
+1 for only supporting Hadoop2 initially (great work, Jesse!). It's important that Phoenix started implementing monitoring and metrics features. An incremental approach is the way to go IMHO.



On Tue, Jul 1, 2014 at 11:13 AM, James Taylor <> wrote:
Seems like an excellent feature to have in 4.1 which I'm hoping we can
do by end of the month. I'd rather the feature make it in and only
support hadoop2 than have it not make the release.

Any one else have thoughts on this?

On Tue, Jul 1, 2014 at 8:01 PM, Jesse Yates <> wrote:
> That's certainly possible, though somewhat unsatisfying :) Its not terribly
> difficult to the Hadoop1 code - I've modularized a majority of the code so
> all that is really needed is replicating the reading of the Hadoop1 metrics2
> classes to something the phoenix writer understands (just like in the
> Hadoop2 impl).
> Happy to push the code somewhere so people can take a look... or they just
> wait a couple weeks :)
> -------------------
> Jesse Yates
> @jesse_yates
> On Tue, Jul 1, 2014 at 10:53 AM, James Taylor <>
> wrote:
>> How about if we make it a hadoop2 only feature?
>> On Tuesday, July 1, 2014, Jesse Yates <> wrote:
>>> I was working on a patch to support using Cloudera's HTrace
>>> ( library for phoenix queries. There was
>>> a preliminary patch for the 2.X series, but it never got committed. Two
>>> weeks ago I almost finished porting it to the 4.X line, but haven't finished
>>> the piece that will support Hadoop1*. I can't seem to find the JIRA right
>>> now though...
>>> I've gotten sidetracked on some internal work, so probably not going to
>>> have time to wrap it up for a couple weeks.
>>> -J
>>> * It uses the Hadoop metrics2 library to collect the traces and then has
>>> a default implementation (as a metrics sink) that writes them to a phoenix
>>> table so it can be analyzed later. Because of the changes between Hadoop1
>>> and 2, there is some reflection funkiness that is necessary to support both,
>>> but just haven't finished the hadoop1 side of it.
>>> -------------------
>>> Jesse Yates
>>> @jesse_yates
>>> On Tue, Jul 1, 2014 at 9:29 AM, Stephen Sprague <>
>>> wrote:
>>>> following up to my own message i did see reference to something called
>>>> "Hannibal" ( in the Phoenix doc.  Looks
>>>> like this helps with identifying imbalance within your cluster.
>>>> I'll keep searching around though - a Phoenix log file looks like
>>>> perhaps the best way to monitor phoenix queries at this point as i don't
>>>> think there is any self-aware queries to ask of it.  That's a jdbc driver
>>>> issue anyway - not phoenix.
>>>> Cheers,
>>>> Stephen.
>>>> On Mon, Jun 30, 2014 at 10:17 PM, Stephen Sprague <>
>>>> wrote:
>>>>> Hi guys,
>>>>> May i ask what tools you use to monitor (active) Phoenix queries?
>>>>> looking at the web ui for hbase master and region servers it isn't
>>>>> exactly intuitive as to what *phoenix* is doing.  Its probably a stretch to
>>>>> ask if there's something like "show processlist" (ala mysql) out there
>>>>> somewhere.
>>>>> so is there a log file with metrics in it that perhaps i could tail or
>>>>> analyze in realtime?   perhaps even feed it back into hbase? :)  hey, wait a
>>>>> sec, anything in those system tables that is realtime-ish?
>>>>> regardless, kinda curious how people monitor the Phoenix plant.
>>>>> thanks,
>>>>> Stephen.

NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.