phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krishna <research...@gmail.com>
Subject Re: Recreating SYSTEM.CATALOG metadata
Date Tue, 30 Sep 2014 16:41:20 GMT
I filed this JIRA: https://issues.apache.org/jira/browse/PHOENIX-1306

Thanks

On Tue, Sep 30, 2014 at 8:53 AM, James Taylor <jamestaylor@apache.org>
wrote:

> Thanks, Krishna. Please file a JIRA for the "option in CREATE TABLE &
> CREATE INDEX clauses to indicate that underlying table is already a
> phoenix table". This would be a reasonable thing to add.
>
>     James
>
> On Mon, Sep 29, 2014 at 11:03 PM, Krishna <research800@gmail.com> wrote:
> > Hi James/Lars,
> >
> > I did another run of bulk load, backup & restore on a new test cluster
> but
> > did not encounter the issue this time. It appears to be some specific
> issue
> > with that version of backup. I will pass on the steps, as soon as I can,
> to
> > replicate it with a small dataset.
> >
> > Having said that, it is still beneficial to have some kind of an option
> in
> > CREATE TABLE & CREATE INDEX clauses to indicate that underlying table is
> > already a phoenix table.
> >
> > Thanks,
> > Krishna
> >
> >
> >
> >
> > On Monday, September 29, 2014, lars hofhansl <larsh@apache.org> wrote:
> >>
> >> Not offhand.
> >>
> >> A few guesses/questions for Krihna:
> >> - does the restore tool restore the exact timestamps? In fact how
> exactly
> >> do you backup and restore the tables?
> >> - is it possible that you updated the CATALOG (via some DDL) while the
> >> backup was running? (although I think that should be OK)
> >>
> >> - does it restore into a clean(ed) cluster? If cleaned, was the ZK state
> >> removed?
> >> - after you restore the two tables, can you scan SYSTEM.CATALOG from an
> >> HBase shell? Or does that part hang?
> >> - in the HBase UI, are there any regions in transition? (might be some
> >> outdated coprocessors, etc, although unlikely)
> >> - exact same version of Phoenix? (between the Phoenix that wrote the
> data
> >> and the one that restored it)
> >> - same for HBase. Exact same version used for the restore?
> >>
> >> As James said, we need some logs from both the client and the region
> >> server(s).
> >> Also if possible, do you have some precise steps to reproduce with a
> small
> >> table?
> >>
> >> Lots of questions :)
> >>
> >> Thanks.
> >>
> >> -- Lars
> >>
> >>
> >> ----- Original Message -----
> >> From: James Taylor <jamestaylor@apache.org>
> >> To: user <user@phoenix.apache.org>; lars hofhansl <larsh@apache.org>
> >> Cc:
> >> Sent: Monday, September 29, 2014 4:26 PM
> >> Subject: Re: Recreating SYSTEM.CATALOG metadata
> >>
> >> @Lars - any idea why Krishna may run into issues using Phoenix after a
> >> restore from an HBase backup?
> >>
> >>
> >>
> >>
> >> On Sun, Sep 28, 2014 at 9:00 PM, James Taylor <jamestaylor@apache.org>
> >> wrote:
> >> > Hi Krishna,
> >> > I think that's what we need to figure out - why is Phoenix having
> >> > trouble when you restore the SYSTEM.CATALOG table. Any client or
> >> > server-side exceptions in the logs when it hangs?
> >> > Thanks,
> >> > James
> >> >
> >> > On Sun, Sep 28, 2014 at 6:18 PM, Krishna <research800@gmail.com>
> wrote:
> >> >> Hi James,
> >> >>
> >> >> I did include SYSTEM.CATALOG table in the backup & restore process.
I
> >> >> dont
> >> >> know why sqlline is having trouble using the backup version - it just
> >> >> hangs,
> >> >> unless catalog table is dropped from hbase shell. I will see if there
> >> >> are
> >> >> any errors in logs that's causing the behavior.
> >> >>
> >> >> Regards
> >> >> Krishna
> >> >>
> >> >>
> >> >> On Sunday, September 28, 2014, James Taylor <jamestaylor@apache.org>
> >> >> wrote:
> >> >>>
> >> >>> Hi Krishna,
> >> >>> Any reason why the SYSTEM.CATALOG hbase table isn't restored as
well
> >> >>> from backup? Yes, if you try to re-create the SYSTEM.CATALOG by
> >> >>> re-issuing your DDL statement, Phoenix won't know that the tables
> were
> >> >>> Phoenix tables before, so will try to add the empty key value to
> each
> >> >>> row. It's possible that an option could be made to avoid this,
but
> >> >>> it'd be good to understand a bit more why this is needed.
> >> >>> Thanks,
> >> >>> James
> >> >>>
> >> >>> On Sun, Sep 28, 2014 at 1:56 PM, Krishna <research800@gmail.com>
> >> >>> wrote:
> >> >>> > Hi,
> >> >>> >
> >> >>> > When I restore hbase from a backup, sqlline gets stuck unless
> >> >>> > SYSTEM.CATALOG
> >> >>> > table is dropped. It is automatically re-created via sqlline.
> >> >>> > However,
> >> >>> > metadata of previously created phoenix tables is lost.
> >> >>> >
> >> >>> > So, to restore the metadata, when a 'CREATE TABLE' statement
is
> >> >>> > re-issued,
> >> >>> > Phoenix takes very, very long time to execute which I think,
is
> >> >>> > because
> >> >>> > Phoenix is upserting null value for _0 column qualifier again
for
> >> >>> > every
> >> >>> > row
> >> >>> > (billions of rows in the table).
> >> >>> >
> >> >>> > Is there a work-around for this by somehow telling Phoenix
that
> the
> >> >>> > underlying table is already a Phoenix table? Using views is
not an
> >> >>> > option
> >> >>> > here. If my understanding is wrong, any pointers as to why
it
> takes
> >> >>> > so
> >> >>> > long
> >> >>> > for executing 'CREATE TABLE' statement is appreciated.
> >> >>> >
> >> >>> > Thanks
> >> >>> >
> >> >>> >
> >>
> >
>

Mime
View raw message