phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bavineni, Bharata" <Bharata.Bavin...@epsilon.com>
Subject RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Date Sun, 01 May 2016 05:37:34 GMT
Ankit,
We will try restarting HBase cluster. Adding explicit “put” in HBase with timestamp as
9 for these two columns has any side effects?

Thank you,
Bharathi.

From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Sunday, May 01, 2016 12:35 AM
To: user@phoenix.apache.org
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP

As you are also not seeing any cell having timestamp greater than 9 (in scan 'SYSTEM.CATALOG',{RAW
=> true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"})
then I think restarting your hbase cluster will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.

On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <Bharata.Bavineni@epsilon.com<mailto:Bharata.Bavineni@epsilon.com>>
wrote:
Hi Ankit,
     We have removed all cells for these two columns and scan with raw => true does not
return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00",
STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00",
STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.

Do we need to explicitly delete from phoenix as well?

Thanks,
Bharathi.

From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<mailto:ankitsinghal59@gmail.com>]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org<mailto:user@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP

Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any
cell for SYSTEM.CATALOG row having timestamp greater than 9.

On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
Ankit,

Follwing the steps, the below exception is thrown. Can you let me know what could have gone
wrong?

ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already
exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)

Thanks,
Arun

On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <ankitsinghal59@gmail.com<mailto:ankitsinghal59@gmail.com>>
wrote:
Hi Arun,

Actually , we missed the quotes as shell will consider everything after ";" as second command.

bin/sqlline.py "localhost;currentSCN=9"

So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not
tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query
running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to
add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG

Regards,
Ankit Singhal



Regards,
Ankit Singhal

On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
James,

Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first
place? Do you have any pointers for us to look into?

Thanks,
Arun

On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <jamestaylor@apache.org<mailto:jamestaylor@apache.org>>
wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system
catalog table.
Thanks,
James


On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
Hi Ankit,

Just following with the question that when the alter statement was issued with CurrentSCN=9,
the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?

Thanks,
Arun

On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
Hi Ankit,

Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set
to 9. Will this cause an issue in the future if it has to compare the timestamps?

After the alter statement was executed and when queried from HBase, the following is the output.

hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN                               CELL
 0 ATA_TYPE                         timestamp=1461362240655, value=\x80\x00\x00\x10
 0:IS_VIEW_REFERENCED                timestamp=1461362240655, value=\x00
 0:NULLABLE                          timestamp=1461362240655, value=\x80\x00\x00\x01
 0 RDINAL_POSITION                  timestamp=1461362240655, value=\x80\x00\x004
 0 ORT_ORDER                        timestamp=1461362240655, value=\x80\x00\x00\x02


Thanks,
Arun


On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <ankitsinghal59@gmail.com<mailto:ankitsinghal59@gmail.com>>
wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember
use correct INTEGER byte representation for DATA_TYPE column).

And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'

Regards,
Ankit

On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
Ankit,

Though the ALTER in pheonix failed, a bunch of puts in HBase  did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in
phoenix supposed to do.

   put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
   put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
   put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
   put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
   put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
   put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9

Thanks,
Arun

On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
Ankit,

When tried executing the query, the following exception is thrown.

Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent
modification to table. tableName=SYSTEM.CATALOG

        at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
        at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
        at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
        at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
        at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
        at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
        at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
        at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
        at sqlline.Commands.execute(Commands.java:822)
        at sqlline.Commands.sql(Commands.java:732)
        at sqlline.SqlLine.dispatch(SqlLine.java:808)
        at sqlline.SqlLine.begin(SqlLine.java:681)
        at sqlline.SqlLine.start(SqlLine.java:398)



Thank,
Arun

On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <ankitsinghal59@gmail.com<mailto:ankitsinghal59@gmail.com>>
wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view
upgrade for version < v4.5) doesn't execute.

But anyways, let's add the required columns manually by following below step;

>  ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit

Quit the shell and start new session without CurrentSCN.
>  ./sqlline.py localhost
> !describe system.catalog

this should resolve the issue of missing column.

Regards,
Ankit Singhal


On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
Ankit,James,

Please find the results you have asked for in the attachment text document.

Thanks,
Arun

On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <ankitsinghal59@gmail.com<mailto:ankitsinghal59@gmail.com>>
wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of

echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG

So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your
phoenix setup works properly and you can also see it in !describe command.






On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)

In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,

!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP

But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.

Is this an expected behavior?

If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking
us in the production environment. Any help to resolve or workaround is highly appreciated.

Thanks,
Arun


On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <ankitsinghal59@gmail.com<mailto:ankitsinghal59@gmail.com>>
wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG',
{RAW=>true}).

On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?

I have limited the scan in HBase and here is the following data.

\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt

Thanks,
Arun


On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
James,

Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|',
coprocessor$2 => '|org.apache

.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1

|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
=> '|org.apache.phoenix.coprocessor.MetaDa

taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|',
METADATA => {'SPLIT_POLICY' => 'org.apache.

phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE
=> '0', COMPRESSION => 'NONE', VERSIONS => '1000',

MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}

1 row in 0.6060 seconds

The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true}
is too huge.

Thanks,
Arun


On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <jamestaylor@apache.org<mailto:jamestaylor@apache.org>>
wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James


On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND
TENANT_ID IS NULL ;

Thanks,
Arun

On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <ankitsinghal59@gmail.com<mailto:ankitsinghal59@gmail.com>>
wrote:
Hi Arun,

Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.


if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp
greater v4.6 timestamp , and which stopping upgrade code to add a new column.

scan 'SYSTEM.CATALOG', {RAW=>true}



Regards,
Ankit Singhal

On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP
column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP"
. Does this mean that PTableImpl is reading from cached entity of  SYSTEM.CATALOG  before
4.6 upgrade?"

We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl
class, but not for 4.6


Thanks,
Arun

On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com<mailto:arun1087@gmail.com>>
wrote:
James,

To add more information on this issue, this happens in new phoenix views associated with brand
new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific
way to reproduce this issue yet. Could you throw some ideas on what direction this problem
could be approached from this point?

Thanks,
Arun




















________________________________

This e-mail and files transmitted with it are confidential, and are intended solely for the
use of the individual or entity to whom this e-mail is addressed. If you are not the intended
recipient, or the employee or agent responsible to deliver it to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this communication
is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason
to believe that you received this message in error, please immediately notify sender by e-mail,
and destroy the original message. Thank You.


________________________________

This e-mail and files transmitted with it are confidential, and are intended solely for the
use of the individual or entity to whom this e-mail is addressed. If you are not the intended
recipient, or the employee or agent responsible to deliver it to the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this communication
is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason
to believe that you received this message in error, please immediately notify sender by e-mail,
and destroy the original message. Thank You.
Mime
View raw message