getting the same error with the most recent phoenix v3.0 

in order to fix had to add to incubator-phoenix/bin/sqlline.py:
    ' -Djava.library.path= /var/lib/hadoop/lib/native/Linux-amd64-64'+\

snappy binaries were installed 
sudo yum install snappy snappy-devel
ln -sf /usr/lib64/libsnappy.so /var/lib/hadoop/lib/native/Linux-amd64-64/.
ln -sf /usr/lib64/libsnappy.so /var/lib/hbase/lib/native/Linux-amd64-64/.

 opened https://issues.apache.org/jira/browse/PHOENIX-877


---------- Forwarded message ----------
From: James Taylor <jamestaylor@apache.org>
Date: Tue, Jan 14, 2014 at 9:55 PM
Subject: Re: native snappy library not available when using count(distinct) grammar
To: Sean Huo <seanxhuo@gmail.com>, Mujtaba Chohan <mchohan@salesforce.com>
Cc: Phoenix HBase User <phoenix-hbase-user@googlegroups.com>


Thanks for that info, Sean.
Mujtaba - we should add Snappy to the other jar as well. We use it for compression for larger COUNT DISTINCT and other aggregates as well as to compress larger ARRAY values.

James


On Tue, Jan 14, 2014 at 6:49 PM, Sean Huo <seanxhuo@gmail.com> wrote:
Also it only happens when the data size of the query is big.


On Tuesday, January 14, 2014 4:43:23 PM UTC-8, Sean Huo wrote:
I am running on Linux, Mujtaba.

On Tuesday, January 14, 2014 4:30:05 PM UTC-8, Mujtaba wrote:
Hi Sean,

What is your client platform? Mac or Linux? Snappy is part of client jar and I'm not getting this exception on my system.

Thanks,
Mujtaba

On Tuesday, 14 January 2014 14:43:33 UTC-8, Sean Huo wrote:
So I do find a workaround. add this cmdline argument to the sqline

-Djava.library.path=/usr/lib/hadoop/lib/native

assuming that a snappy implementation is already being installed in hadoop.


On Tuesday, January 14, 2014 2:08:37 PM UTC-8, Sean Huo wrote:
Any update on this issue? I am encountering the same problem. 
I am using the trunk build, which is 3.00.snapshot.
The jar file does include the snappy library as shown:

jar tvf phoenix-3.0.0-SNAPSHOT-client.jar | grep -i snappy

     0 Wed Oct 05 10:34:56 PDT 2011 META-INF/maven/org.xerial.snappy/

     0 Wed Oct 05 10:34:56 PDT 2011 META-INF/maven/org.xerial.snappy/snappy-java/

 11560 Wed Oct 05 10:34:46 PDT 2011 META-INF/maven/org.xerial.snappy/snappy-java/LICENSE

   144 Wed Oct 05 10:34:56 PDT 2011 META-INF/maven/org.xerial.snappy/snappy-java/pom.properties

  9273 Wed Oct 05 10:34:48 PDT 2011 META-INF/maven/org.xerial.snappy/snappy-java/pom.xml

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/

  1691 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/OSInfo.class

  9444 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/Snappy.class

   660 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyBundleActivator.class

  2384 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyCodec.class

  1250 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyError.class

  1790 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyErrorCode.class

  1740 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyException.class

  5354 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyInputStream.class

 11188 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyLoader.class

  1409 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyNative.class

   789 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyNativeAPI.class

  1179 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/SnappyNativeLoader.bytecode

  3911 Wed Oct 05 10:34:52 PDT 2011 org/xerial/snappy/SnappyOutputStream.class

    17 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/VERSION

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Linux/

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Linux/amd64/

991112 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/native/Linux/amd64/libsnappyjava.so

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Linux/i386/

814468 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/native/Linux/i386/libsnappyjava.so

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Mac/

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Mac/i386/

 44036 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/native/Mac/i386/libsnappyjava.jnilib

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Mac/x86_64/

 49792 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/native/Mac/x86_64/libsnappyjava.jnilib

    71 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/native/README

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Windows/

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Windows/amd64/

645632 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/native/Windows/amd64/snappyjava.dll

     0 Wed Oct 05 10:34:56 PDT 2011 org/xerial/snappy/native/Windows/x86/

600064 Wed Oct 05 10:34:48 PDT 2011 org/xerial/snappy/native/Windows/x86/snappyjava.dll

     0 Wed Oct 03 05:05:16 PDT 2012 org/apache/hadoop/io/compress/snappy/

  3960 Wed Oct 03 05:05:16 PDT 2012 org/apache/hadoop/io/compress/SnappyCodec.class

  1483 Wed Oct 03 05:05:16 PDT 2012 org/apache/hadoop/io/compress/snappy/LoadSnappy.class

  4439 Wed Oct 03 05:05:16 PDT 2012 org/apache/hadoop/io/compress/snappy/SnappyCompressor.class

  3976 Wed Oct 03 05:05:16 PDT 2012 org/apache/hadoop/io/compress/snappy/SnappyDecompressor.class

It looks like it tries to use org.apache.hadoop.io.compress.snappy.LoadSnappy class to load the library instead of 

org.xerial.snappy.SnappyLoader. 

as shown in the stacktrace here

java.lang.RuntimeException: native snappy library not available

at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:189)

at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:125)

at org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getDecompressor(Compression.java:331)

at com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator.aggregate(DistinctValueWithCountClientAggregator.java:70)


Thanks

Sean





On Thursday, December 19, 2013 8:56:43 AM UTC-8, Mujtaba wrote:
Correct. Snappy is bundled in Phoenix 2.2.1 client jar (posted on 12/17) https://github.com/forcedotcom/phoenix/wiki/Download

//mujtaba

On Wednesday, 18 December 2013 22:29:17 UTC-8, James Taylor wrote:
The count distinct function requires Snappy when the aggregation gets too big. I believe we've added this to the client jar in newer releases. Is that correct, Mujtaba?

You can do this yourself too, by adding the Snappy jar to the classpath in the sqlline script if you don't want to upgrade.

Thanks,
James


On Tue, Dec 17, 2013 at 4:15 AM, Ming Yang <yangmin...@gmail.com> wrote:
create table summary(
id integer not null,
date date not null
device varchar not null
vv bigint
constraint pk primary key (id, date, device)
) salt_buckets = 30,versions=1,compression='snappy';

Loaded 18 million rows,query 1 is ok,query 2 throws out exception.I'm sure that snappy library is available,is this a bug of Phoenix?

query 1 : select id,device,sum(vv) from summary group by id,device;

query 2 : select count(distinct id) from summary;
+-------------------------+
| DISTINCT_COUNT(ID) |
+-------------------------+
java.lang.RuntimeException: native snappy library not available
        at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:189)
        at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:125)
        at org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getDecompressor(Compression.java:331)
        at com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator.aggregate(DistinctValueWithCountClientAggregator.java:70)
        at com.salesforce.phoenix.expression.aggregator.ClientAggregators.aggregate(ClientAggregators.java:73)
        at com.salesforce.phoenix.iterate.GroupedAggregatingResultIterator.next(GroupedAggregatingResultIterator.java:85)
        at com.salesforce.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:49)
        at com.salesforce.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:741)
        at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2429)
        at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2074)
        at sqlline.SqlLine.print(SqlLine.java:1735)
        at sqlline.SqlLine$Commands.execute(SqlLine.java:3683)
        at sqlline.SqlLine$Commands.sql(SqlLine.java:3584)
        at sqlline.SqlLine.dispatch(SqlLine.java:821)
        at sqlline.SqlLine.begin(SqlLine.java:699)
        at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441)
        at sqlline.SqlLine.main(SqlLine.java:424)

--
You received this message because you are subscribed to the Google Groups "Phoenix HBase User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phoenix-hbase-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Phoenix HBase User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phoenix-hbase-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Phoenix HBase User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phoenix-hbase-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.