phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Huo <s...@crunchyroll.com>
Subject Re: Aggregation queries on a big dataset are failing
Date Fri, 03 Oct 2014 00:17:47 GMT
Phoenix pom file has this dependency entry
<guava.version>12.0.1</guava.version>
I would imagine that this particular code path is executed when the
datasize is large. I had the same problem before.


On Thu, Oct 2, 2014 at 4:53 PM, Krishna <research800@gmail.com> wrote:

> Thanks Sean. I'll try that.
> Do you think, this is happening only with large datasets because of the
> spills to disk during "group by" and guava package is used in such
> scenarios?
>
> On Thu, Oct 2, 2014 at 3:31 PM, Sean Huo <sean@crunchyroll.com> wrote:
>
>> You have to upgrade the guava jar on the regionservers. I am using
>> guava-12.0.1.jar.
>>
>> On Thu, Oct 2, 2014 at 2:51 PM, Krishna <research800@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Aggregate queries seem to be working fine on smaller datasets but when
>>> the data needs to be aggregated over millions of rows, query fails with
>>> following error stack. I'm running Phoenix 3.1 on HBase 0.94.18. Any help?
>>>
>>> Query is something like this:
>>>
>>>> select a.customer_id, a.product_id, count(*) from customer as a join
>>>> product as b on a.product_id = b.product_id where b.category = 'retail'
>>>> group by a.customer_id, a.product_id
>>>
>>>
>>> Caused by: org.apache.hadoop.ipc.RemoteException:
>>> org.apache.hadoop.hbase.DoNotRetryIOException:
>>> CUSTOMER,\x11\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1412194938071.5602bf7e28a72ad6e3db6257b22e38f8.:
>>> com.google.common.hash.BloomFilter.put(Ljava/lang/Object;)Z
>>>         at
>>> org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:73)
>>>         at
>>> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:91)
>>>         at
>>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1333)
>>>         at
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2588)
>>>         at
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2556)
>>>         at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
>>>         at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>         at
>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:354)
>>>         at
>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)
>>> Caused by: java.lang.NoSuchMethodError:
>>> com.google.common.hash.BloomFilter.put(Ljava/lang/Object;)Z
>>>         at
>>> org.apache.phoenix.cache.aggcache.SpillMap$MappedByteBufferMap.addElement(SpillMap.java:437)
>>>         at
>>> org.apache.phoenix.cache.aggcache.SpillMap.put(SpillMap.java:294)
>>>         at
>>> org.apache.phoenix.cache.aggcache.SpillManager.spill(SpillManager.java:261)
>>>         at
>>> org.apache.phoenix.cache.aggcache.SpillableGroupByCache$1.removeEldestEntry(SpillableGroupByCache.java:190)
>>>         at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:431)
>>>         at java.util.HashMap.put(HashMap.java:505)
>>>         at
>>> org.apache.phoenix.cache.aggcache.SpillableGroupByCache.cache(SpillableGroupByCache.java:249)
>>>         at
>>> org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.scanUnordered(GroupedAggregateRegionObserver.java:384)
>>>         at
>>> org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:130)
>>>         at
>>> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:89)
>>>         ... 8 more
>>>
>>>         at
>>> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1012)
>>>         at
>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
>>>         at com.sun.proxy.$Proxy6.openScanner(Unknown Source)
>>>         at
>>> org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:224)
>>>         at
>>> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:126)
>>>         at
>>> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
>>>         at
>>> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:164)
>>>
>>>
>>
>

Mime
View raw message