It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).

On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <arun1087@gmail.com> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?

I have limited the scan in HBase and here is the following data.

\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089

\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\

x00default

\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_PK

\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_SUB_ORG_ID\x00default

\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_BY\x00default

\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt

\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn

_0_1460354090089\x00CDS_UPDATED_DATE\x00defau

lt


Thanks,

Arun



On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com> wrote:
James,

Table SYSTEM.CATALOG is ENABLED 
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache

.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1

|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa

taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.

phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}} 
COLUMN FAMILIES DESCRIPTION 
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',

MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}

1 row in 0.6060 seconds

The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.

Thanks,
Arun


On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <jamestaylor@apache.org> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James


On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <arun1087@gmail.com> wrote:

!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.

But we do see this column from select statement:

select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;

 

Thanks,

Arun


On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <ankitsinghal59@gmail.com> wrote:
Hi Arun,

Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.


if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.

scan 'SYSTEM.CATALOG', {RAW=>true}



Regards,
Ankit Singhal

On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of  SYSTEM.CATALOG  before 4.6 upgrade?"

We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6


Thanks,
Arun

On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <arun1087@gmail.com> wrote:
James,

To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?

Thanks,
Arun