Also, 

   ALTER INDEX "idx_name" ON "tblname" REBUILD

   Doesn't honor the quoted idx_name. Looks for a table IDX_NAME.


Regards,
Abhilash L L
Capillary Technologies
M:919886208262
abhilash@capillarytech.com | www.capillarytech.com

Email from people at capillarytech.com may not represent official policy of  Capillary Technologies unless explicitly stated. Please see our Corporate-Email-Policy for details. Contents of this email are confidential. Please contact the Sender if you have received this email in error.



On Fri, Jul 4, 2014 at 1:26 AM, Abhilash L L <abhilash@capillarytech.com> wrote:
We are loading data via hive. We see that the phoenix hive serde is still under conceptualization / not released.
So we made some code changes in the serde to support composite row keys and also use phoenix serialization.

Just saw that when the put is being sent from phoenix jdbc, there is extra attributes being passed which helps in getting index maintainers.(IdxUUID and IdxMD)

This would be empty when we are trying to do puts via shell or a hbase client.
Without the attributes, the codec would always say its disabled.

So, only way is to use phoenix sql ?



Regards,
Abhilash L L
Capillary Technologies
M:919886208262

Email from people at capillarytech.com may not represent official policy of  Capillary Technologies unless explicitly stated. Please see our Corporate-Email-Policy for details. Contents of this email are confidential. Please contact the Sender if you have received this email in error.



On Thu, Jul 3, 2014 at 6:00 PM, James Taylor <jamestaylor@apache.org> wrote:
It's best to go through the JDBC APIs - any particular reason you're
bypassing them?

You'll need to insert the empty key value if you want Phoenix to work
correctly. We do this for performance reasons as well as correctness
(otherwise a row would disappear if you set all of its values to
null). We add this emtpy key value for each row when an UPSERT
statement is executed.

Secondary indexes on mutable tables do not require you to go through
our APIs in order for them to be maintained, but this hasn't been
tested that thoroughly.

Thanks,
James

On Thu, Jul 3, 2014 at 1:15 PM, Abhilash L L <abhilash@capillarytech.com> wrote:
> Hello,
>
>    In phoenix 3, we are creating the table / index via JDBC. And we
> inserting using hbase apis. Would the index be updated?
>
>    We see that when the insert is done via phoenix, its adding an extra
> column value to 0:_0 to the main data table and also index refers to it.
>
> baby table scan:
> babyname1                              column=0:_0, timestamp=1404315599040,
> value=
> babyname1                              column=0:occurrences,
> timestamp=1404315599040, value=\x80\x00\x00\x00\x00\x00\x00\x14
>
>
> baby names idx table scan:
> \xC1\x15\x00babyname1                  column=0:_0, timestamp=1404315599040,
> value=
>
>
>     But when we insert via the hbase shell, we do not issue this extra put.
>
> Doubts:
>
> 1) For mutable tables, do the puts have to be via phoenix/jdbc?
>
> 2) When we have a table which has column families says cf1, cf2. Where is _0
> populated ?
>
> Regards,
> Abhilash L L
> Capillary Technologies
> M:919886208262
> abhilash@capillarytech.com | www.capillarytech.com
>
> Email from people at capillarytech.com may not represent official policy of
> Capillary Technologies unless explicitly stated. Please see our
> Corporate-Email-Policy for details. Contents of this email are confidential.
> Please contact the Sender if you have received this email in error.
>
>
>
> On Thu, Jul 3, 2014 at 9:47 AM, Abhilash L L <abhilash@capillarytech.com>
> wrote:
>>
>> Thanks James, its picking up the indexes as expected with phoenix 3.
>>
>> We have integrated phoenix 3 in our code paths.
>>
>> Also, I checked out the 3.0.0 tag. Saw that for hadoop 2, its compiled
>> against hadoop 2.0.4 alpha.
>> We are using hadoop 2.4 and hbase 94.18. Would it be fine if we compiled
>> against 2.4?
>>
>> Regards,
>> Abhilash L L
>> Capillary Technologies
>> M:919886208262
>> abhilash@capillarytech.com | www.capillarytech.com
>>
>> Email from people at capillarytech.com may not represent official policy
>> of  Capillary Technologies unless explicitly stated. Please see our
>> Corporate-Email-Policy for details. Contents of this email are confidential.
>> Please contact the Sender if you have received this email in error.
>>
>>
>>
>> On Wed, Jul 2, 2014 at 3:49 PM, Abhilash L L <abhilash@capillarytech.com>
>> wrote:
>>>
>>> We thought phoenix 3.x is for hbase 96+ which isnt supported on amazon
>>> emr.
>>>
>>> Just now saw that it works on 94.x onwards..  we will be testing that,
>>> and if that works will be switching to phoenix 3.
>>>
>>> Will update the thread once we are done.
>>>
>>> Thanks!
>>>
>>>
>>> Regards,
>>> Abhilash L L
>>> Capillary Technologies
>>> M:919886208262
>>> abhilash@capillarytech.com | www.capillarytech.com
>>>
>>> Email from people at capillarytech.com may not represent official policy
>>> of  Capillary Technologies unless explicitly stated. Please see our
>>> Corporate-Email-Policy for details. Contents of this email are confidential.
>>> Please contact the Sender if you have received this email in error.
>>>
>>>
>>>
>>> On Wed, Jul 2, 2014 at 2:48 PM, James Taylor <jamestaylor@apache.org>
>>> wrote:
>>>>
>>>> I'd recommend upgrading to 3.0 as these issues (and many more) have
>>>> already been fixed. It's an easy transition from 2.2.3 to 3.0. We have
>>>> an upgrade process in place documented here:
>>>> http://phoenix.apache.org/upgrade_from_2_2.html
>>>> Thanks,
>>>> James
>>>>
>>>> On Mon, Jun 30, 2014 at 8:58 PM, Abhilash L L
>>>> <abhilash@capillarytech.com> wrote:
>>>> > Also the index name doesnt honor the quoted name passed. It creates
>>>> > the
>>>> > index table with upper case name
>>>> >
>>>> >
>>>> > Regards,
>>>> > Abhilash L L
>>>> > Capillary Technologies
>>>> > M:919886208262
>>>> > abhilash@capillarytech.com | www.capillarytech.com
>>>> >
>>>> > Email from people at capillarytech.com may not represent official
>>>> > policy of
>>>> > Capillary Technologies unless explicitly stated. Please see our
>>>> > Corporate-Email-Policy for details. Contents of this email are
>>>> > confidential.
>>>> > Please contact the Sender if you have received this email in error.
>>>> >
>>>> >
>>>> >
>>>> > On Tue, Jul 1, 2014 at 12:20 AM, Abhilash L L
>>>> > <abhilash@capillarytech.com>
>>>> > wrote:
>>>> >>
>>>> >> Thanks James.
>>>> >>
>>>> >> What version of Phoenix are you currently using?
>>>> >> --> 2.2.3 incubating
>>>> >>
>>>> >> Would it be possible to use uppercase table and column qualifier
>>>> >> names
>>>> >> as a work around?
>>>> >> --> It will be quite some changes at this point in time in our
>>>> >> project.
>>>> >>
>>>> >> The index selection is done in QueryOptimizer, but I doubt the bug
>>>> >> would be there. Might be in IndexStatementRewriter which is the code
>>>> >> that re-writes a SQL statement to use the index table rather than the
>>>> >> data table.
>>>> >> --> Thanks. WIll take a look.
>>>> >>
>>>> >>
>>>> >>
>>>> >> Regards,
>>>> >> Abhilash L L
>>>> >> Capillary Technologies
>>>> >> M:919886208262
>>>> >> abhilash@capillarytech.com | www.capillarytech.com
>>>> >>
>>>> >> Email from people at capillarytech.com may not represent official
>>>> >> policy
>>>> >> of  Capillary Technologies unless explicitly stated. Please see our
>>>> >> Corporate-Email-Policy for details. Contents of this email are
>>>> >> confidential.
>>>> >> Please contact the Sender if you have received this email in error.
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Mon, Jun 30, 2014 at 8:28 PM, James Taylor
>>>> >> <jamestaylor@apache.org>
>>>> >> wrote:
>>>> >>>
>>>> >>> Hi,
>>>> >>> What version of Phoenix are you currently using? I remember a bug
>>>> >>> along these lines that was fixed, but I thought it made it into
>>>> >>> 3.0/4.0. Does the problem occur on the latest on the 3.0/4.0 branch?
>>>> >>>
>>>> >>> The index selection is done in QueryOptimizer, but I doubt the bug
>>>> >>> would be there. Might be in IndexStatementRewriter which is the code
>>>> >>> that re-writes a SQL statement to use the index table rather than
>>>> >>> the
>>>> >>> data table.
>>>> >>>
>>>> >>> Would it be possible to use uppercase table and column qualifier
>>>> >>> names
>>>> >>> as a work around?
>>>> >>>
>>>> >>> Thanks,
>>>> >>> James
>>>> >>>
>>>> >>> On Mon, Jun 30, 2014 at 4:25 PM, Abhilash L L
>>>> >>> <abhilash@capillarytech.com> wrote:
>>>> >>> > Hello,
>>>> >>> >
>>>> >>> >    Seems like there is an issue in index selection when the
>>>> >>> > columns
>>>> >>> > involved
>>>> >>> > are case sensitive. Here is an example from the one of the slides.
>>>> >>> >
>>>> >>> >
>>>> >>> > ========================================================
>>>> >>> > CREATE TABLE baby_names (
>>>> >>> >     name VARCHAR PRIMARY KEY,
>>>> >>> >     occurrences BIGINT);
>>>> >>> >
>>>> >>> > CREATE INDEX baby_names_idx ON baby_names(occurrences);
>>>> >>> >
>>>> >>> > explain SELECT name, occurrences FROM baby_names WHERE occurrences
>>>> >>> > >
>>>> >>> > 100;
>>>> >>> > -- CLIENT PARALLEL 1-WAY RANGE SCAN OVER BABY_NAMES_IDX [1E+2] -
>>>> >>> > [*]
>>>> >>> > ========================================================
>>>> >>> >
>>>> >>> >
>>>> >>> > ========================================================
>>>> >>> > CREATE TABLE baby_names (
>>>> >>> >     "name" VARCHAR PRIMARY KEY,
>>>> >>> >     "occurrences" BIGINT);
>>>> >>> >
>>>> >>> > CREATE INDEX baby_names_idx ON baby_names("occurrences");
>>>> >>> >
>>>> >>> > explain SELECT "name", "occurrences" FROM baby_names WHERE
>>>> >>> > "occurrences" >
>>>> >>> > 100;
>>>> >>> > -- CLIENT PARALLEL 1-WAY FULL SCAN OVER BABY_NAMES SERVER FILTER
>>>> >>> > BY
>>>> >>> > occurrences > 100
>>>> >>> > ========================================================
>>>> >>> >
>>>> >>> > We would like to run a patched version of phoenix for now, since
>>>> >>> > we are
>>>> >>> > nearing our release date. Would help us a great deal if someone
>>>> >>> > can
>>>> >>> > point us
>>>> >>> > to the class where the index selection is being done.
>>>> >>> >
>>>> >>> > Regards,
>>>> >>> > Abhilash L L
>>>> >>> > Capillary Technologies
>>>> >>> > M:919886208262
>>>> >>> > abhilash@capillarytech.com | www.capillarytech.com
>>>> >>> >
>>>> >>> > Email from people at capillarytech.com may not represent official
>>>> >>> > policy of
>>>> >>> > Capillary Technologies unless explicitly stated. Please see our
>>>> >>> > Corporate-Email-Policy for details. Contents of this email are
>>>> >>> > confidential.
>>>> >>> > Please contact the Sender if you have received this email in
>>>> >>> > error.
>>>> >>> >
>>>> >>> >
>>>> >>> > Email from people at capillarytech.com may not represent official
>>>> >>> > policy of
>>>> >>> > Capillary Technologies unless explicitly stated. Please see our
>>>> >>> > Corporate-Email-Policy for details.Contents of this email are
>>>> >>> > confidential.
>>>> >>> > Please contact the Sender if you have received this email in
>>>> >>> > error.
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> > Email from people at capillarytech.com may not represent official
>>>> > policy of
>>>> > Capillary Technologies unless explicitly stated. Please see our
>>>> > Corporate-Email-Policy for details.Contents of this email are
>>>> > confidential.
>>>> > Please contact the Sender if you have received this email in error.
>>>
>>>
>>
>
>
> Email from people at capillarytech.com may not represent official policy of
> Capillary Technologies unless explicitly stated. Please see our
> Corporate-Email-Policy for details.Contents of this email are confidential.
> Please contact the Sender if you have received this email in error.



Email from people at capillarytech.com may not represent official policy of Capillary Technologies unless explicitly stated. Please see our Corporate-Email-Policy for details.Contents of this email are confidential. Please contact the Sender if you have received this email in error.