phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kamil <alex.ka...@gmail.com>
Subject Re: Phoenix response time
Date Sat, 06 Sep 2014 06:41:22 GMT
do you use EBS or ephemeral storage, I found EBS performance to be somewhat
unpredictable


On Sat, Sep 6, 2014 at 2:37 AM, Vikas Agarwal <vikas@infoobjects.com> wrote:

> Hbase is 0.98.0
> Phoenix is 4.0
>
>
> On Sat, Sep 6, 2014 at 12:04 PM, Vikas Agarwal <vikas@infoobjects.com>
> wrote:
>
>> Yes, that is why it is a trouble for me. However, on contrary, HBase
>> shell is also on the same machine and same environment, so if it is an
>> issue of resource (CPU or memory) it should have affected the HBase too,
>> but HBase is able to give me results within 0.0150 seconds. :(
>>
>> No, I haven't tested it outside AWS. I guess, it should not be the case
>> due to much better performance by native HBase query on HBase shell.
>>
>>
>> On Sat, Sep 6, 2014 at 11:59 AM, James Taylor <jamestaylor@apache.org>
>> wrote:
>>
>>> Something is up in your environment. What version of Phoenix and HBase
>>> are you using and in what environment? Have you tried this locally,
>>> outside of AWS to compare?
>>>
>>> Take a look at our perf numbers, generated more-or-less daily, and
>>> which run over more data that what you're testing against:
>>>
>>> http://phoenix-bin.github.io/client/performance/phoenix-20140904095313.htm
>>>
>>> Some of these are point queries and they take in the neighborhood of
>>> 0.01 seconds.
>>>
>>> Thanks,
>>> James
>>>
>>> On Fri, Sep 5, 2014 at 10:48 PM, Vikas Agarwal <vikas@infoobjects.com>
>>> wrote:
>>> > Missed to mention that count query (posted in my last mail) is also
>>> taking
>>> > very long time to return the count.
>>> >
>>> >
>>> > On Sat, Sep 6, 2014 at 11:17 AM, Vikas Agarwal <vikas@infoobjects.com>
>>> > wrote:
>>> >>
>>> >> As I mentioned, schema is nothing but bunch of fields (some being
>>> >> integers, longs and text) along with primary key (row key) and I am
>>> making
>>> >> simple query to get result for a particular primary key, nothing more
>>> than
>>> >> that.
>>> >>
>>> >> 0: jdbc:phoenix:localhost> SELECT count(1) FROM table_name;
>>> >>
>>> >> +------------+
>>> >>
>>> >> |  COUNT(1)  |
>>> >>
>>> >> +------------+
>>> >>
>>> >> | 4667515    |
>>> >>
>>> >> +------------+
>>> >>
>>> >> 1 row selected (132.11 seconds)
>>> >>
>>> >>
>>> >>
>>> >> On Sat, Sep 6, 2014 at 11:09 AM, Puneet Kumar Ojha
>>> >> <puneet.kumar@pubmatic.com> wrote:
>>> >>>
>>> >>> If you can share the schema,data type,cardinality of each dimension
>>> and
>>> >>> usual queries, I can help to design a schema with performance of
>>> less than 1
>>> >>> sec using Phoenix.
>>> >>>
>>> >>>
>>> >>>
>>> >>> Thanks
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> ------ Original message------
>>> >>>
>>> >>> From: James Taylor
>>> >>>
>>> >>> Date: Sat, Sep 6, 2014 10:15 AM
>>> >>>
>>> >>> To: user;
>>> >>>
>>> >>> Subject:Re: Phoenix response time
>>> >>>
>>> >>>
>>> >>>
>>> >>> Vikas,
>>> >>> Please post your schema and query.
>>> >>> Thanks,
>>> >>> James
>>> >>>
>>> >>> On Fri, Sep 5, 2014 at 9:18 PM, Vikas Agarwal <vikas@infoobjects.com
>>> >
>>> >>> wrote:
>>> >>> > Ours is also a single node setup right now and as of now there
are
>>> less
>>> >>> > than
>>> >>> > 1 million rows which is expected to grow around 100m at minimum.
>>> >>> >
>>> >>> > I am aware of secondary indexes but when I am querying on
>>> primary/row
>>> >>> > key,
>>> >>> > why would it take so much time?
>>> >>> >
>>> >>> > I am directly querying using sqlline for Phoenix and hbase
shell
>>> for
>>> >>> > HBase
>>> >>> > query. I am not expecting to do any fine tuning for such small
>>> dataset.
>>> >>> > I am
>>> >>> > assumimg a minimum performance level out of the box.
>>> >>> >
>>> >>> > On Friday, September 5, 2014, yeshwanth kumar <
>>> yeshwanth43@gmail.com>
>>> >>> > wrote:
>>> >>> >>
>>> >>> >> hi vikas,
>>> >>> >>
>>> >>> >> we used phoenix on a 4 core/23Gb machine, as a single node
setup.
>>> >>> >> used HDP 2.1
>>> >>> >> our table has 50-70M rows,
>>> >>> >> select on that table took less than 2 seconds.
>>> >>> >> Aggregation queries took less than 8 seconds.
>>> >>> >> for achieving good performance we created secondary index
on the
>>> >>> >> table.
>>> >>> >>
>>> >>> >> make sure you finetuned hbase,
>>> >>> >> enabling compression on the data makes a difference in
response.
>>> >>> >> if u distribute the data and load over all regions in hbase,
>>> >>> >> look at the performance tips mentioned in phoenix blog
>>> >>> >>
>>> >>> >> -yeshwanth
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> Cheers,
>>> >>> >> Yeshwanth
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> On Fri, Sep 5, 2014 at 5:42 PM, Vikas Agarwal <
>>> vikas@infoobjects.com>
>>> >>> >> wrote:
>>> >>> >>>
>>> >>> >>> Hi,
>>> >>> >>>
>>> >>> >>> Preface: We are testing phoenix using Hortonworks distribution
>>> for
>>> >>> >>> HBase
>>> >>> >>> on Amazon EC2 instance (r3.large, 2 CPU/15 GB RAM).
>>> >>> >>>
>>> >>> >>> With contrast to performance benchmarks, I found Phoenix
to be
>>> very
>>> >>> >>> slow
>>> >>> >>> in querying even on primary key or row key. So, tried
to
>>> increase the
>>> >>> >>> RAM
>>> >>> >>> for HBase and Phoenix and increasing the CPU and RAM
by
>>> upgrading the
>>> >>> >>> EC2
>>> >>> >>> machine type to r3.xlarge (4 CPU, 30 GB RAM). Results
were like
>>> this:
>>> >>> >>>
>>> >>> >>> Time takes in returning result of query on row key:
>>> >>> >>> With Storm running and very less RAM available: 50
sec
>>> >>> >>>
>>> >>> >>> With Storm stopped and RAM available to Phoenix and
HBase: 18 sec
>>> >>> >>>
>>> >>> >>> With new machine of next higher category (4 CPU and
30 GB RAM):
>>> 8 sec
>>> >>> >>>
>>> >>> >>> Pure HBase query by row key with Storm stopped and
(2 CPU, 15 GB
>>> >>> >>> RAM):
>>> >>> >>> 0.0150 seconds. :)
>>> >>> >>>
>>> >>> >>> So, the difference seems to be many fold of what native
HBase is
>>> >>> >>> providing to us. I am not able to understand how it
can be
>>> possible?
>>> >>> >>> What I
>>> >>> >>> am missing here?
>>> >>> >>>
>>> >>> >>> --
>>> >>> >>> Regards,
>>> >>> >>> Vikas Agarwal
>>> >>> >>> 91 – 9928301411
>>> >>> >>>
>>> >>> >>> InfoObjects, Inc.
>>> >>> >>> Execution Matters
>>> >>> >>> http://www.infoobjects.com
>>> >>> >>> 2041 Mission College Boulevard, #280
>>> >>> >>> Santa Clara, CA 95054
>>> >>> >>> +1 (408) 988-2000 Work
>>> >>> >>> +1 (408) 716-2726 Fax
>>> >>> >>
>>> >>> >>
>>> >>> >
>>> >>> >
>>> >>> > --
>>> >>> > Regards,
>>> >>> > Vikas Agarwal
>>> >>> > 91 – 9928301411
>>> >>> >
>>> >>> > InfoObjects, Inc.
>>> >>> > Execution Matters
>>> >>> > http://www.infoobjects.com
>>> >>> > 2041 Mission College Boulevard, #280
>>> >>> > Santa Clara, CA 95054
>>> >>> > +1 (408) 988-2000 Work
>>> >>> > +1 (408) 716-2726 Fax
>>> >>> >
>>> >>> >
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Regards,
>>> >> Vikas Agarwal
>>> >> 91 – 9928301411
>>> >>
>>> >> InfoObjects, Inc.
>>> >> Execution Matters
>>> >> http://www.infoobjects.com
>>> >> 2041 Mission College Boulevard, #280
>>> >> Santa Clara, CA 95054
>>> >> +1 (408) 988-2000 Work
>>> >> +1 (408) 716-2726 Fax
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Regards,
>>> > Vikas Agarwal
>>> > 91 – 9928301411
>>> >
>>> > InfoObjects, Inc.
>>> > Execution Matters
>>> > http://www.infoobjects.com
>>> > 2041 Mission College Boulevard, #280
>>> > Santa Clara, CA 95054
>>> > +1 (408) 988-2000 Work
>>> > +1 (408) 716-2726 Fax
>>>
>>
>>
>>
>> --
>> Regards,
>> Vikas Agarwal
>> 91 – 9928301411
>>
>> InfoObjects, Inc.
>> Execution Matters
>> http://www.infoobjects.com
>> 2041 Mission College Boulevard, #280
>> Santa Clara, CA 95054
>> +1 (408) 988-2000 Work
>> +1 (408) 716-2726 Fax
>>
>>
>
>
> --
> Regards,
> Vikas Agarwal
> 91 – 9928301411
>
> InfoObjects, Inc.
> Execution Matters
> http://www.infoobjects.com
> 2041 Mission College Boulevard, #280
> Santa Clara, CA 95054
> +1 (408) 988-2000 Work
> +1 (408) 716-2726 Fax
>
>

Mime
View raw message