phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Di Spaltro <dan.dispal...@gmail.com>
Subject Re: ManagedTests and 4.1.0-RC1
Date Mon, 01 Sep 2014 02:29:35 GMT
Here's more detail on the exception and the related line where HBase is
adding non-integer KV's.

https://issues.apache.org/jira/browse/PHOENIX-1226

https://github.com/apache/hbase/blob/0.98.5/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RequestContext.java#L105

Thanks for the help,

-Dan


On Sun, Aug 31, 2014 at 5:27 PM, Dan Di Spaltro <dan.dispaltro@gmail.com>
wrote:

> Oh and yes Ill create a JIRA today.
>
>
> On Fri, Aug 29, 2014 at 5:09 PM, Dan Di Spaltro <dan.dispaltro@gmail.com>
> wrote:
>
>> If I disable the metrics stuff on every query everything works fine btw.
>>  Therefore, I don't think it's anything related to tests.
>>
>>
>> On Fri, Aug 29, 2014 at 3:03 PM, Samarth Jain <samarth.jain@gmail.com>
>> wrote:
>>
>>> + Jesse
>>>
>>> I think Jesse probably envisioned early use of annotations to be around
>>> numerical metrics, hence the String-Integer pair. But we should change it
>>> to be more generic to accept things like tenantId or userName. Do you mind
>>> filing an enhancement JIRA for this Dan?
>>>
>>> Thanks,
>>> Samarth
>>>
>>>
>>>
>>> On Fri, Aug 29, 2014 at 2:51 PM, Dan Di Spaltro <dan.dispaltro@gmail.com
>>> > wrote:
>>>
>>>> Okay that helps, but it still doesn't really explain that code path.
>>>>  Why are annotations being converted byte-wise to ints?  Aren't annotations
>>>> potentially opaque strings or something else?
>>>>
>>>> -Dan
>>>>
>>>>
>>>> On Fri, Aug 29, 2014 at 11:27 AM, James Taylor <jamestaylor@apache.org>
>>>> wrote:
>>>>
>>>>> Hey Dan,
>>>>> There were some changes in the test framework to make them run faster.
>>>>> Our entire test suite can run in about 10-15mins instead of 60mins
>>>>> now. One of the new requirements is adding the annotation that Samarth
>>>>> indicated. Once JUnit releases 4.12, this will no longer be necessary,
>>>>> as the annotation will automatically be inherited from the
>>>>> BaseClientManagedTimeIT class.
>>>>> Thanks,
>>>>> James
>>>>>
>>>>> On Thu, Aug 28, 2014 at 10:08 PM, Dan Di Spaltro
>>>>> <dan.dispaltro@gmail.com> wrote:
>>>>> > I basically inherit from BaseClientManagedTimeIT and write a junit
>>>>> tests
>>>>> >
>>>>> > It's been working great up until 4.1.
>>>>> >
>>>>> > This code just doesn't look right, why would an annotation
>>>>> necessarily have
>>>>> > to be an int?
>>>>> >
>>>>> >
>>>>> https://github.com/apache/phoenix/blob/29a7be42bfa468b12d16fd0756b987f5359c45c4/phoenix-hadoop2-compat/src/main/java/org/apache/phoenix/trace/TraceMetricSource.java#L122
>>>>> >
>>>>> > then calls the below function, which takes bytes and makes an int
>>>>> from the
>>>>> > bytes...
>>>>> >
>>>>> >
>>>>> https://github.com/apache/phoenix/blob/f99e5d8d609d326fb3571255cd8f47961b1c6860/phoenix-hadoop-compat/src/main/java/org/apache/phoenix/trace/TracingCompat.java#L56
>>>>> >
>>>>> >
>>>>> > On Thu, Aug 28, 2014 at 11:43 AM, Samarth Jain <
>>>>> samarth.jain@gmail.com>
>>>>> > wrote:
>>>>> >>
>>>>> >> Dan,
>>>>> >>
>>>>> >> Can you tell me how you are running your tests? Do you have
the
>>>>> test class
>>>>> >> annotated with the right category annotation -
>>>>> >> @Category(HBaseManagedTimeTest.class). Also, can you send over
your
>>>>> test
>>>>> >> class to see what might be causing problems?
>>>>> >>
>>>>> >> Thanks,
>>>>> >> Samarth
>>>>> >>
>>>>> >>
>>>>> >> On Thu, Aug 28, 2014 at 10:34 AM, Dan Di Spaltro <
>>>>> dan.dispaltro@gmail.com>
>>>>> >> wrote:
>>>>> >>>
>>>>> >>> Any idea on this, it's blocking my usage in tests and I
can't tell
>>>>> if I
>>>>> >>> am just setting something up incorrectly?  Also I am concerned
>>>>> that this can
>>>>> >>> affect production since this code path I would assume is
used
>>>>> frequently.
>>>>> >>>
>>>>> >>> -Dan
>>>>> >>>
>>>>> >>>
>>>>> >>> On Tue, Aug 26, 2014 at 10:49 PM, Dan Di Spaltro
>>>>> >>> <dan.dispaltro@gmail.com> wrote:
>>>>> >>>>
>>>>> >>>> I inherit from the BaseHBaseManagedTimeIT and implement
my own
>>>>> tests
>>>>> >>>> using the infrastructure you've put together.  It's
worked pretty
>>>>> well,
>>>>> >>>> minus the fact I use an Ivy resolver which doesn't deal
with
>>>>> jarless pom's
>>>>> >>>> well.
>>>>> >>>>
>>>>> >>>> So I've upgraded from 4.0 to 4.1 and ran into a single
issue that
>>>>> looks
>>>>> >>>> related to Tracing, and I can't really figure it out.
 When I
>>>>> start the
>>>>> >>>> cluster everything works as expected but after I am
done creating
>>>>> tables
>>>>> >>>> like clockwork I get this:
>>>>> >>>>
>>>>> >>>> 58062 [defaultRpcServer.handler=2,queue=0,port=53950]
WARN
>>>>> >>>> org.apache.hadoop.ipc.RpcServer  -
>>>>> >>>> defaultRpcServer.handler=2,queue=0,port=53950: caught:
>>>>> >>>> java.lang.IllegalArgumentException: offset (0) + length
(4)
>>>>> exceed the
>>>>> >>>> capacity of the array: 3
>>>>> >>>> at
>>>>> >>>>
>>>>> org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:600)
>>>>> >>>> at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:749)
>>>>> >>>> at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:725)
>>>>> >>>> at
>>>>> >>>>
>>>>> org.apache.phoenix.trace.TracingCompat.readAnnotation(TracingCompat.java:56)
>>>>> >>>> at
>>>>> >>>>
>>>>> org.apache.phoenix.trace.TraceMetricSource.receiveSpan(TraceMetricSource.java:121)
>>>>> >>>> at org.cloudera.htrace.Tracer.deliver(Tracer.java:81)
>>>>> >>>> at org.cloudera.htrace.impl.MilliSpan.stop(MilliSpan.java:70)
>>>>> >>>> at org.cloudera.htrace.TraceScope.close(TraceScope.java:70)
>>>>> >>>> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
>>>>> >>>> at
>>>>> >>>>
>>>>> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
>>>>> >>>> at
>>>>> org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
>>>>> >>>> at java.lang.Thread.run(Thread.java:744)
>>>>> >>>>
>>>>> >>>> And the test just stops, which I imagine is a byproduct
of this
>>>>> >>>> exception.  I inspected at this point and there are
two traces
>>>>> the one it
>>>>> >>>> throws on is the key is "user" and value is my username.
It's
>>>>> trying to
>>>>> >>>> convert it to an int
>>>>> >>>> ...
>>>>> >>>> return new Pair<String, String>(new String(key),
>>>>> >>>> Integer.toString(Bytes.toInt(value)));
>>>>> >>>> ...
>>>>> >>>>
>>>>> >>>> Any ideas?
>>>>> >>>>
>>>>> >>>> -Dan
>>>>> >>>>
>>>>> >>>> --
>>>>> >>>> Dan Di Spaltro
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> --
>>>>> >>> Dan Di Spaltro
>>>>> >>
>>>>> >>
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Dan Di Spaltro
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Dan Di Spaltro
>>>>
>>>
>>>
>>
>>
>> --
>> Dan Di Spaltro
>>
>
>
>
> --
> Dan Di Spaltro
>



-- 
Dan Di Spaltro

Mime
View raw message