phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vikas Agarwal <vi...@infoobjects.com>
Subject Re: Phoenix response time
Date Sat, 06 Sep 2014 05:32:08 GMT
We are generating DDL using Java string concatenation and a table is
created on the topology submission. I missed to copy the exact statement,
so it would take me some time to get it again  because I have to resubmit
the topology.


On Sat, Sep 6, 2014 at 10:50 AM, James Taylor <jamestaylor@apache.org>
wrote:

> Would you mind posting the CREATE TABLE statement you used to create
> the table, as it's a little easier to read?
> Thanks,
> James
>
> On Fri, Sep 5, 2014 at 10:19 PM, Vikas Agarwal <vikas@infoobjects.com>
> wrote:
> > James,
> >
> > Schema is pretty simple, I guess. Here it is (I have renamed some actual
> > column names)
> >
> > TABLE_CAT  | TABLE_SCHEM | TABLE_NAME | COLUMN_NAME | DATA_TYPE  |
> TYPE_NAME
> > | COLUMN_SIZE | BUFFER_LENGTH | DECIMAL_DIGITS | NUM_PREC_RADIX |
> NULLABLE
> > | COLUMN_DEF | SQ |
> >
> +------------+-------------+------------+-------------+------------+------------+-------------+---------------+----------------+----------------+------------+------------+----+
> > | null       | null        | table_name | TIMESTAMP   | -5         |
> BIGINT
> > | null        | null          | null           | null           | 0
> > | null      |
> > | null       | null        | table_name | ID          | 12         |
> VARCHAR
> > | 255         | null          | null           | null           | 0
> > | null      |
> > | null       | null        | table_name | TEXT_FIELD  | 12         |
> VARCHAR
> > | 255         | null          | null           | null           | 1
> > | null      |
> > | null       | null        | table_name | USER_ID     | 12         |
> VARCHAR
> > | 255         | null          | null           | null           | 0
> > | null      |
> > | null       | null        | table_name | TEXT_FIELD        | 12
>  |
> > VARCHAR    | 25523       | null          | null           | null
>    |
> > 1          | null      |
> > | null       | null        | table_name | TYPE        | 12         |
> VARCHAR
> > | 255         | null          | null           | null           | 1
> > | null      |
> > | null       | null        | table_name | COUNT_1 | 4          | INTEGER
> > | null        | null          | null           | null           | 1
> > | null   |
> > | null       | null        | table_name | COUNT_2 | 4          | INTEGER
> > | null        | null          | null           | null           | 1
> > | null  |
> > | null       | null        | table_name | COUNT_3 | 4          | INTEGER
> > | null        | null          | null           | null           | 1
> > | null    |
> > | null       | null        | table_name | COUNT_4 | -5         | BIGINT
> > | null        | null          | null           | null           | 1
> > | null      |
> > | null       | null        | table_name | COUNT_5  | -5         | BIGINT
> > | null        | null          | null           | null           | 1
> > | null      |
> > | null       | null        | table_name | COUNT_6  | -5         | BIGINT
> > | null        | null          | null           | null           | 1
> > | null      |
> > | null       | null        | table_name | TAGS        | 2003       |
> > VARCHAR_ARRAY | null        | null          | null           | null
> > | 1          | null   |
> > | null       | null        | table_name | UPDATED     | -5         |
> BIGINT
> > | null        | null          | null           | null           | 1
> > | null      |
> > | null       | null        | table_name | SOME_FIELD     | 12         |
> > VARCHAR    | 255         | null          | null           | null
>    |
> > 1          | null      |
> > | null       | null        | table_name | LOCATIONS | 12         |
> VARCHAR
> > | 255         | null          | null           | null           | 1
> > | null    |
> >
> +------------+-------------+------------+-------------+------------+------------+-------------+---------------+----------------+----------------+------------+------------+----+
> >
> >
> > Query:
> >
> > SELECT USER_ID FROM HJK_SI_LEAD_FEED WHERE ID='507449491025170432';
> >
> >
> >
> >
> >
> > On Sat, Sep 6, 2014 at 10:15 AM, James Taylor <jamestaylor@apache.org>
> > wrote:
> >>
> >> Vikas,
> >> Please post your schema and query.
> >> Thanks,
> >> James
> >>
> >> On Fri, Sep 5, 2014 at 9:18 PM, Vikas Agarwal <vikas@infoobjects.com>
> >> wrote:
> >> > Ours is also a single node setup right now and as of now there are
> less
> >> > than
> >> > 1 million rows which is expected to grow around 100m at minimum.
> >> >
> >> > I am aware of secondary indexes but when I am querying on primary/row
> >> > key,
> >> > why would it take so much time?
> >> >
> >> > I am directly querying using sqlline for Phoenix and hbase shell for
> >> > HBase
> >> > query. I am not expecting to do any fine tuning for such small
> dataset.
> >> > I am
> >> > assumimg a minimum performance level out of the box.
> >> >
> >> > On Friday, September 5, 2014, yeshwanth kumar <yeshwanth43@gmail.com>
> >> > wrote:
> >> >>
> >> >> hi vikas,
> >> >>
> >> >> we used phoenix on a 4 core/23Gb machine, as a single node setup.
> >> >> used HDP 2.1
> >> >> our table has 50-70M rows,
> >> >> select on that table took less than 2 seconds.
> >> >> Aggregation queries took less than 8 seconds.
> >> >> for achieving good performance we created secondary index on the
> table.
> >> >>
> >> >> make sure you finetuned hbase,
> >> >> enabling compression on the data makes a difference in response.
> >> >> if u distribute the data and load over all regions in hbase,
> >> >> look at the performance tips mentioned in phoenix blog
> >> >>
> >> >> -yeshwanth
> >> >>
> >> >>
> >> >>
> >> >> Cheers,
> >> >> Yeshwanth
> >> >>
> >> >>
> >> >>
> >> >> On Fri, Sep 5, 2014 at 5:42 PM, Vikas Agarwal <vikas@infoobjects.com
> >
> >> >> wrote:
> >> >>>
> >> >>> Hi,
> >> >>>
> >> >>> Preface: We are testing phoenix using Hortonworks distribution
for
> >> >>> HBase
> >> >>> on Amazon EC2 instance (r3.large, 2 CPU/15 GB RAM).
> >> >>>
> >> >>> With contrast to performance benchmarks, I found Phoenix to be
very
> >> >>> slow
> >> >>> in querying even on primary key or row key. So, tried to increase
> the
> >> >>> RAM
> >> >>> for HBase and Phoenix and increasing the CPU and RAM by upgrading
> the
> >> >>> EC2
> >> >>> machine type to r3.xlarge (4 CPU, 30 GB RAM). Results were like
> this:
> >> >>>
> >> >>> Time takes in returning result of query on row key:
> >> >>> With Storm running and very less RAM available: 50 sec
> >> >>>
> >> >>> With Storm stopped and RAM available to Phoenix and HBase: 18 sec
> >> >>>
> >> >>> With new machine of next higher category (4 CPU and 30 GB RAM):
8
> sec
> >> >>>
> >> >>> Pure HBase query by row key with Storm stopped and (2 CPU, 15 GB
> RAM):
> >> >>> 0.0150 seconds. :)
> >> >>>
> >> >>> So, the difference seems to be many fold of what native HBase is
> >> >>> providing to us. I am not able to understand how it can be possible?
> >> >>> What I
> >> >>> am missing here?
> >> >>>
> >> >>> --
> >> >>> Regards,
> >> >>> Vikas Agarwal
> >> >>> 91 – 9928301411
> >> >>>
> >> >>> InfoObjects, Inc.
> >> >>> Execution Matters
> >> >>> http://www.infoobjects.com
> >> >>> 2041 Mission College Boulevard, #280
> >> >>> Santa Clara, CA 95054
> >> >>> +1 (408) 988-2000 Work
> >> >>> +1 (408) 716-2726 Fax
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Regards,
> >> > Vikas Agarwal
> >> > 91 – 9928301411
> >> >
> >> > InfoObjects, Inc.
> >> > Execution Matters
> >> > http://www.infoobjects.com
> >> > 2041 Mission College Boulevard, #280
> >> > Santa Clara, CA 95054
> >> > +1 (408) 988-2000 Work
> >> > +1 (408) 716-2726 Fax
> >> >
> >> >
> >
> >
> >
> >
> > --
> > Regards,
> > Vikas Agarwal
> > 91 – 9928301411
> >
> > InfoObjects, Inc.
> > Execution Matters
> > http://www.infoobjects.com
> > 2041 Mission College Boulevard, #280
> > Santa Clara, CA 95054
> > +1 (408) 988-2000 Work
> > +1 (408) 716-2726 Fax
>



-- 
Regards,
Vikas Agarwal
91 – 9928301411

InfoObjects, Inc.
Execution Matters
http://www.infoobjects.com
2041 Mission College Boulevard, #280
Santa Clara, CA 95054
+1 (408) 988-2000 Work
+1 (408) 716-2726 Fax

Mime
View raw message