phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Kumaran Sabtharishi <arun1...@gmail.com>
Subject Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Date Fri, 22 Apr 2016 15:44:10 GMT
Ankit,

Though the ALTER in pheonix failed, a bunch of puts in HBase  did the job.
Is it okay to do the following? These puts in HBase intends to do what the
ALTER command in phoenix supposed to do.

   put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9

   put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9

   put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9

   put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9

   put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9

   put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9


Thanks,

Arun

On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:

> Ankit,
>
> When tried executing the query, the following exception is thrown.
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
>         at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>         at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>         at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>         at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>         at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>         at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>         at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>         at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>         at sqlline.Commands.execute(Commands.java:822)
>         at sqlline.Commands.sql(Commands.java:732)
>         at sqlline.SqlLine.dispatch(SqlLine.java:808)
>         at sqlline.SqlLine.begin(SqlLine.java:681)
>         at sqlline.SqlLine.start(SqlLine.java:398)
>
>
> Thank,
> Arun
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <ankitsinghal59@gmail.com>
> wrote:
>
>> Arun,
>> As per the system.catalog ,there seems no way that upgrade code(adding
>> new columns and view upgrade for version < v4.5) doesn't execute.
>>
>> But anyways, let's add the required columns manually by following below
>> step;
>>
>> >  ./sqlline.py localhost;CurrentSCN=9
>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>> IS_ROW_TIMESTAMP BOOLEAN;
>> >!quit
>>
>> Quit the shell and start new session without CurrentSCN.
>> >  ./sqlline.py localhost
>> > !describe system.catalog
>>
>> this should resolve the issue of missing column.
>>
>> Regards,
>> Ankit Singhal
>>
>>
>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> Ankit,James,
>>>
>>> Please find the results you have asked for in the attachment text
>>> document.
>>>
>>> Thanks,
>>> Arun
>>>
>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <ankitsinghal59@gmail.com
>>> > wrote:
>>>
>>>> Arun,
>>>> the observations you had is understandable.
>>>> So, that's why we again requested the output of
>>>>
>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>>>>
>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>> system.catalog table so that your phoenix setup works properly and you can
>>>> also see it in !describe command.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>
>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>>> client,
>>>>>
>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>
>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>>>
>>>>> Is this an expected behavior?
>>>>>
>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I
>>>>> do? This issue is blocking us in the production environment. Any help
to
>>>>> resolve or workaround is highly appreciated.
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>>
>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>
>>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> One more question to add,
>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>
>>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x05
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>> timestamp=1460354090655, value=default
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>>>>>> value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>>>>>>> value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>>>>>> value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>>>>> value=u
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> value=\x80\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Arun
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> James,
>>>>>>>>
>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|',
coprocessor$2 =>
>>>>>>>> '|org.apache
>>>>>>>>
>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>> coprocessor$3 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>
>>>>>>>> |', coprocessor$4 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>
>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|',
METADATA =>
>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>
>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' =>
'true'}}
>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER
=>
>>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE',
VERSIONS => '1000',
>>>>>>>>
>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS
=>
>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false',
BLOCKCACHE => 'true'}
>>>>>>>>
>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>
>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for
scan
>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Arun
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Arun,
>>>>>>>>> Please run the command Ankit mentioned in an HBase shell
and post
>>>>>>>>> the output back here.
>>>>>>>>> Thanks,
>>>>>>>>> James
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi
<
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>> column.
>>>>>>>>>>
>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>
>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’
AND
>>>>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>
>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG,
by
>>>>>>>>>>> doing !describe on system.catalog.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> if not,
>>>>>>>>>>> can you share the output of below command. As
it seems
>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater
v4.6 timestamp , and
>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>
>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran
Sabtharishi <
>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> After further investigation, we found that
Phoenix Upsert query
>>>>>>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column,
but PTableImpl.getColumn() is
>>>>>>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP"
. Does
>>>>>>>>>>>> this mean that PTableImpl is reading from
cached entity of  SYSTEM.CATALOG
>>>>>>>>>>>>  before 4.6 upgrade?"
>>>>>>>>>>>>
>>>>>>>>>>>> We do see that clearCache() is being called
for 4.7, and 4.7
>>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl
class, but not for 4.6
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran
Sabtharishi <
>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> James,
>>>>>>>>>>>>>
>>>>>>>>>>>>> To add more information on this issue,
this happens in new
>>>>>>>>>>>>> phoenix views associated with brand new
tables as well. So, this cannot be
>>>>>>>>>>>>> an upgrade/migration issue. Not figured
out a specific way to reproduce
>>>>>>>>>>>>> this issue yet. Could you throw some
ideas on what direction this problem
>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message