phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krishna <research...@gmail.com>
Subject Re: Aggregation queries on a big dataset are failing
Date Thu, 02 Oct 2014 23:53:56 GMT
Thanks Sean. I'll try that.
Do you think, this is happening only with large datasets because of the
spills to disk during "group by" and guava package is used in such
scenarios?

On Thu, Oct 2, 2014 at 3:31 PM, Sean Huo <sean@crunchyroll.com> wrote:

> You have to upgrade the guava jar on the regionservers. I am using
> guava-12.0.1.jar.
>
> On Thu, Oct 2, 2014 at 2:51 PM, Krishna <research800@gmail.com> wrote:
>
>> Hi,
>>
>> Aggregate queries seem to be working fine on smaller datasets but when
>> the data needs to be aggregated over millions of rows, query fails with
>> following error stack. I'm running Phoenix 3.1 on HBase 0.94.18. Any help?
>>
>> Query is something like this:
>>
>>> select a.customer_id, a.product_id, count(*) from customer as a join
>>> product as b on a.product_id = b.product_id where b.category = 'retail'
>>> group by a.customer_id, a.product_id
>>
>>
>> Caused by: org.apache.hadoop.ipc.RemoteException:
>> org.apache.hadoop.hbase.DoNotRetryIOException:
>> CUSTOMER,\x11\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1412194938071.5602bf7e28a72ad6e3db6257b22e38f8.:
>> com.google.common.hash.BloomFilter.put(Ljava/lang/Object;)Z
>>         at
>> org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:73)
>>         at
>> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:91)
>>         at
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1333)
>>         at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2588)
>>         at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2556)
>>         at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at
>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:354)
>>         at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)
>> Caused by: java.lang.NoSuchMethodError:
>> com.google.common.hash.BloomFilter.put(Ljava/lang/Object;)Z
>>         at
>> org.apache.phoenix.cache.aggcache.SpillMap$MappedByteBufferMap.addElement(SpillMap.java:437)
>>         at
>> org.apache.phoenix.cache.aggcache.SpillMap.put(SpillMap.java:294)
>>         at
>> org.apache.phoenix.cache.aggcache.SpillManager.spill(SpillManager.java:261)
>>         at
>> org.apache.phoenix.cache.aggcache.SpillableGroupByCache$1.removeEldestEntry(SpillableGroupByCache.java:190)
>>         at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:431)
>>         at java.util.HashMap.put(HashMap.java:505)
>>         at
>> org.apache.phoenix.cache.aggcache.SpillableGroupByCache.cache(SpillableGroupByCache.java:249)
>>         at
>> org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.scanUnordered(GroupedAggregateRegionObserver.java:384)
>>         at
>> org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:130)
>>         at
>> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:89)
>>         ... 8 more
>>
>>         at
>> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1012)
>>         at
>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
>>         at com.sun.proxy.$Proxy6.openScanner(Unknown Source)
>>         at
>> org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:224)
>>         at
>> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:126)
>>         at
>> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
>>         at
>> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:164)
>>
>>
>

Mime
View raw message