phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vamsi Krishna <vamsi.attl...@gmail.com>
Subject Re: how to drop SYSTEM.SEQUENCE table to reduce the no. of salt buckets for this table
Date Wed, 04 Mar 2015 16:34:33 GMT
Thanks Nick.
I configured the 'phoenix.sequence.saltBuckets' property in hbase-site.xml
on the client and now I see the SYSTEM.SEQUENCE table recreated in HBase on
the first Phoenix client connection.
But, I still don't see the table in Phoenix.
Do, I need to manually insert data in SYSTEM.CATALOG table to see
SYSTEM.SEQUENCE table in Phoenix?

James,
We are evaluating Phoenix on a 2 node cluster.
While testing the application we noticed the regions of our application
tables (10 salt buckets/10 regions) are not evenly distributed among the 2
region servers. And then we have observed the SYSTEM.SEQUENCE table is
having 256 regions. We are not sure if having too many regions is the
reason for uneven distribution of our application table regions. But
anyways we are not planning on using the Sequence feature. So, we want to
reduce the salt-buckets for SYSTEM.SEQUENCE table.

Thanks,
Vamsi Attluri.

On Tue, Mar 3, 2015 at 1:54 PM, James Taylor <jamestaylor@apache.org> wrote:

> Hi Vamsi,
> Sorry you're having trouble with this. I'd like to understand more
> first what the overhead is that you're seeing when SYSTEM.SEQUENCE is
> split 256 ways. Would you mind helping me understand this? If this is
> causing a problem, we may want to lower the default so that new users
> don't run into an issue.
>
> FWIW, the reason the SYSTEM.SEQUENCE table is salted so highly is that
> we wanted to spread the load as much as possible for incrementing
> sequence values. Sequences are allocated on the server in batches,
> though, with the client caching them and doling out until a new batch
> is need, so this mitigates the load as well. However, I saw no
> downside to salt this table because only a single row at a time is
> ever read, so the salt byte would be resolved on the client without
> any read perf hit (i.e. we're not scanning across multiple rows).
>
> As far as overwriting this value, Nick is correct - it's a client-side
> setting. Also, don't set it to 1, but set it to 0 instead if you want
> it not to be salted at all.
>
> Thanks,
> James
>
> On Tue, Mar 3, 2015 at 1:39 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
> > The first client to connect to Phoenix and notice the absence of
> > SYSTEM.SEQUENCE will create the table. That means the configuration
> > phoenix.sequence.saltBuckets must be set on that client.
> >
> >
> > On Tue, Mar 3, 2015 at 1:17 PM, Vamsi Krishna <vamsi.attluri@gmail.com>
> > wrote:
> >>
> >> James,
> >>
> >> We tried the following steps:
> >> 1) Dropped SYSTEM.SEQUENCE table from base
> >> 1.1) disable 'SYSTEM.SEQUENCE'
> >> 1.2) drop 'SYSTEM.SEQUENCE'
> >> 2) Deleted SYSTEM.SEQUENCE meta-data from phoenix system tables
> >> (SYSTEM.CATALOG, SYSTEM.STATS)
> >> 2.1) delete from SYSTEM.CATALOG where table_name = 'SEQUENCE' and
> >> table_schem = 'SYSTEM';
> >> 2.2) delete from SYSTEM.STATS where physical_name like
> '%SYSTEM.SEQUENCE%'
> >> or region_name like '%SYSTEM.SEQUENCE%';
> >> 3) Set phoenix.sequence.saltBuckets property to 1
> >> 4) Restarted all region servers and master server
> >>
> >> Observations:
> >> 1. After Master & Region servers are restarted and the client is
> >> reconnected I see SYSTEM.SEQUENCE table is re-created in HBase, but not
> in
> >> Phoenix. Do I need to manually insert the SYSTEM.SEQUENCE table
> meta-data in
> >> SYSTEM.CATALOG table?
> >> 2. When I list HDFS folder
> >> '/apps/hbase/data/data/default/SYSTEM.SEQUENCE', I see it again created
> 256
> >> regions.
> >>
> >> Am I missing any steps above?
> >>
> >> Thanks,
> >> Vamsi Attluri.
> >>
> >> On Fri, Feb 27, 2015 at 9:51 AM, James Taylor <jamestaylor@apache.org>
> >> wrote:
> >>>
> >>> I'd recommend dropping the SYSTEM.SEQUENCE table from the HBase shell
> >>> (instead of deleting the folder in HDFS). Everything else sounded
> >>> fine, but make sure to bounce your cluster and restart your clients
> >>> after doing this.
> >>>
> >>> Thanks,
> >>> James
> >>>
> >>> On Thu, Feb 26, 2015 at 12:28 PM, Vamsi Krishna <
> vamsi.attluri@gmail.com>
> >>> wrote:
> >>> > Hi,
> >>> >
> >>> > From phoenix archives I see that we can drop SYSTEM.SEQUENCE table
> and
> >>> > set
> >>> > 'phoenix.sequence.saltBuckets' property to '1' to see the
> >>> > SYSTEM.SEQUENCE
> >>> > table recreated with 1 salt bucket on cluster restart.
> >>> > Reference:
> >>> >
> >>> >
> http://mail-archives.apache.org/mod_mbox/incubator-phoenix-user/201412.mbox/%3CCAAF1Jdi4SViGfnQy0h45pM2YhiQiNpbpHGJ3Ov69QB6dFVwahw@mail.gmail.com%3E
> >>> >
> >>> > But, we are not able to drop the SYSTEM.SEQUENCE table.
> >>> > We are seeing the following error when we try to drop the table:
> >>> > DROP TABLE SYSTEM."SEQUENCE";
> >>> > Error: ERROR 1010 (42M01): Not allowed to mutate table.
> >>> > tableName=SYSTEM.SEQUENCE (state=42M01,code=1010)
> >>> >
> >>> > How to drop SYSTEM.SEQUENCE table?
> >>> >
> >>> > Can we delete "default/SYSTEM.SEQUENCE" folder under HBase data
> >>> > directory on
> >>> > HDFS and delete the SYSTEM.SEQUENCE meta data from SYSTEM.CATALOG,
> >>> > SYSTEM.STATS tables using below belete queries?
> >>> >
> >>> > delete from SYSTEM.CATALOG where table_name = 'SEQUENCE' and
> >>> > table_schem =
> >>> > 'SYSTEM';
> >>> > delete from SYSTEM.STATS where physical_name = 'SYSTEM.SEQUENCE';
> >>> >
> >>> > Will this create any other issues?
> >>> >
> >>> > Thanks,
> >>> > Vamsi Attluri.
> >>
> >>
> >
>

Mime
View raw message