phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhilash L L <abhil...@capillarytech.com>
Subject Re: Issue with index selection
Date Thu, 03 Jul 2014 20:51:59 GMT
Also,

   ALTER INDEX "idx_name" ON "tblname" REBUILD

   Doesn't honor the quoted idx_name. Looks for a table IDX_NAME.


Regards,
Abhilash L L
Capillary Technologies
M:919886208262
abhilash@capillarytech.com | www.capillarytech.com

Email from people at capillarytech.com may not represent official policy of
 Capillary Technologies unless explicitly stated. Please see our
Corporate-Email-Policy
<http://support.capillary.co.in/policy-public/Corporate-Email-Policy.pdf>
for details. Contents of this email are confidential. Please contact the
Sender if you have received this email in error.



On Fri, Jul 4, 2014 at 1:26 AM, Abhilash L L <abhilash@capillarytech.com>
wrote:

> We are loading data via hive. We see that the phoenix hive serde is still
> under conceptualization / not released.
> So we made some code changes in the serde to support composite row keys
> and also use phoenix serialization.
>
> Just saw that when the put is being sent from phoenix jdbc, there is extra
> attributes being passed which helps in getting index maintainers.(IdxUUID
> and IdxMD)
>
> This would be empty when we are trying to do puts via shell or a hbase
> client.
> Without the attributes, the codec would always say its disabled.
>
> So, only way is to use phoenix sql ?
>
>
>
> Regards,
> Abhilash L L
> Capillary Technologies
> M:919886208262
> abhilash@capillarytech.com | www.capillarytech.com
>
> Email from people at capillarytech.com may not represent official policy
> of  Capillary Technologies unless explicitly stated. Please see our
> Corporate-Email-Policy
> <http://support.capillary.co.in/policy-public/Corporate-Email-Policy.pdf>
> for details. Contents of this email are confidential. Please contact the
> Sender if you have received this email in error.
>
>
>
> On Thu, Jul 3, 2014 at 6:00 PM, James Taylor <jamestaylor@apache.org>
> wrote:
>
>> It's best to go through the JDBC APIs - any particular reason you're
>> bypassing them?
>>
>> You'll need to insert the empty key value if you want Phoenix to work
>> correctly. We do this for performance reasons as well as correctness
>> (otherwise a row would disappear if you set all of its values to
>> null). We add this emtpy key value for each row when an UPSERT
>> statement is executed.
>>
>> Secondary indexes on mutable tables do not require you to go through
>> our APIs in order for them to be maintained, but this hasn't been
>> tested that thoroughly.
>>
>> Thanks,
>> James
>>
>> On Thu, Jul 3, 2014 at 1:15 PM, Abhilash L L <abhilash@capillarytech.com>
>> wrote:
>> > Hello,
>> >
>> >    In phoenix 3, we are creating the table / index via JDBC. And we
>> > inserting using hbase apis. Would the index be updated?
>> >
>> >    We see that when the insert is done via phoenix, its adding an extra
>> > column value to 0:_0 to the main data table and also index refers to it.
>> >
>> > baby table scan:
>> > babyname1                              column=0:_0,
>> timestamp=1404315599040,
>> > value=
>> > babyname1                              column=0:occurrences,
>> > timestamp=1404315599040, value=\x80\x00\x00\x00\x00\x00\x00\x14
>> >
>> >
>> > baby names idx table scan:
>> > \xC1\x15\x00babyname1                  column=0:_0,
>> timestamp=1404315599040,
>> > value=
>> >
>> >
>> >     But when we insert via the hbase shell, we do not issue this extra
>> put.
>> >
>> > Doubts:
>> >
>> > 1) For mutable tables, do the puts have to be via phoenix/jdbc?
>> >
>> > 2) When we have a table which has column families says cf1, cf2. Where
>> is _0
>> > populated ?
>> >
>> > Regards,
>> > Abhilash L L
>> > Capillary Technologies
>> > M:919886208262
>> > abhilash@capillarytech.com | www.capillarytech.com
>> >
>> > Email from people at capillarytech.com may not represent official
>> policy of
>> > Capillary Technologies unless explicitly stated. Please see our
>> > Corporate-Email-Policy for details. Contents of this email are
>> confidential.
>> > Please contact the Sender if you have received this email in error.
>> >
>> >
>> >
>> > On Thu, Jul 3, 2014 at 9:47 AM, Abhilash L L <
>> abhilash@capillarytech.com>
>> > wrote:
>> >>
>> >> Thanks James, its picking up the indexes as expected with phoenix 3.
>> >>
>> >> We have integrated phoenix 3 in our code paths.
>> >>
>> >> Also, I checked out the 3.0.0 tag. Saw that for hadoop 2, its compiled
>> >> against hadoop 2.0.4 alpha.
>> >> We are using hadoop 2.4 and hbase 94.18. Would it be fine if we
>> compiled
>> >> against 2.4?
>> >>
>> >> Regards,
>> >> Abhilash L L
>> >> Capillary Technologies
>> >> M:919886208262
>> >> abhilash@capillarytech.com | www.capillarytech.com
>> >>
>> >> Email from people at capillarytech.com may not represent official
>> policy
>> >> of  Capillary Technologies unless explicitly stated. Please see our
>> >> Corporate-Email-Policy for details. Contents of this email are
>> confidential.
>> >> Please contact the Sender if you have received this email in error.
>> >>
>> >>
>> >>
>> >> On Wed, Jul 2, 2014 at 3:49 PM, Abhilash L L <
>> abhilash@capillarytech.com>
>> >> wrote:
>> >>>
>> >>> We thought phoenix 3.x is for hbase 96+ which isnt supported on amazon
>> >>> emr.
>> >>>
>> >>> Just now saw that it works on 94.x onwards..  we will be testing that,
>> >>> and if that works will be switching to phoenix 3.
>> >>>
>> >>> Will update the thread once we are done.
>> >>>
>> >>> Thanks!
>> >>>
>> >>>
>> >>> Regards,
>> >>> Abhilash L L
>> >>> Capillary Technologies
>> >>> M:919886208262
>> >>> abhilash@capillarytech.com | www.capillarytech.com
>> >>>
>> >>> Email from people at capillarytech.com may not represent official
>> policy
>> >>> of  Capillary Technologies unless explicitly stated. Please see our
>> >>> Corporate-Email-Policy for details. Contents of this email are
>> confidential.
>> >>> Please contact the Sender if you have received this email in error.
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Jul 2, 2014 at 2:48 PM, James Taylor <jamestaylor@apache.org>
>> >>> wrote:
>> >>>>
>> >>>> I'd recommend upgrading to 3.0 as these issues (and many more) have
>> >>>> already been fixed. It's an easy transition from 2.2.3 to 3.0. We
>> have
>> >>>> an upgrade process in place documented here:
>> >>>> http://phoenix.apache.org/upgrade_from_2_2.html
>> >>>> Thanks,
>> >>>> James
>> >>>>
>> >>>> On Mon, Jun 30, 2014 at 8:58 PM, Abhilash L L
>> >>>> <abhilash@capillarytech.com> wrote:
>> >>>> > Also the index name doesnt honor the quoted name passed. It
creates
>> >>>> > the
>> >>>> > index table with upper case name
>> >>>> >
>> >>>> >
>> >>>> > Regards,
>> >>>> > Abhilash L L
>> >>>> > Capillary Technologies
>> >>>> > M:919886208262
>> >>>> > abhilash@capillarytech.com | www.capillarytech.com
>> >>>> >
>> >>>> > Email from people at capillarytech.com may not represent official
>> >>>> > policy of
>> >>>> > Capillary Technologies unless explicitly stated. Please see
our
>> >>>> > Corporate-Email-Policy for details. Contents of this email
are
>> >>>> > confidential.
>> >>>> > Please contact the Sender if you have received this email in
error.
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > On Tue, Jul 1, 2014 at 12:20 AM, Abhilash L L
>> >>>> > <abhilash@capillarytech.com>
>> >>>> > wrote:
>> >>>> >>
>> >>>> >> Thanks James.
>> >>>> >>
>> >>>> >> What version of Phoenix are you currently using?
>> >>>> >> --> 2.2.3 incubating
>> >>>> >>
>> >>>> >> Would it be possible to use uppercase table and column
qualifier
>> >>>> >> names
>> >>>> >> as a work around?
>> >>>> >> --> It will be quite some changes at this point in time
in our
>> >>>> >> project.
>> >>>> >>
>> >>>> >> The index selection is done in QueryOptimizer, but I doubt
the bug
>> >>>> >> would be there. Might be in IndexStatementRewriter which
is the
>> code
>> >>>> >> that re-writes a SQL statement to use the index table rather
than
>> the
>> >>>> >> data table.
>> >>>> >> --> Thanks. WIll take a look.
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> Regards,
>> >>>> >> Abhilash L L
>> >>>> >> Capillary Technologies
>> >>>> >> M:919886208262
>> >>>> >> abhilash@capillarytech.com | www.capillarytech.com
>> >>>> >>
>> >>>> >> Email from people at capillarytech.com may not represent
official
>> >>>> >> policy
>> >>>> >> of  Capillary Technologies unless explicitly stated. Please
see
>> our
>> >>>> >> Corporate-Email-Policy for details. Contents of this email
are
>> >>>> >> confidential.
>> >>>> >> Please contact the Sender if you have received this email
in
>> error.
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> On Mon, Jun 30, 2014 at 8:28 PM, James Taylor
>> >>>> >> <jamestaylor@apache.org>
>> >>>> >> wrote:
>> >>>> >>>
>> >>>> >>> Hi,
>> >>>> >>> What version of Phoenix are you currently using? I
remember a bug
>> >>>> >>> along these lines that was fixed, but I thought it
made it into
>> >>>> >>> 3.0/4.0. Does the problem occur on the latest on the
3.0/4.0
>> branch?
>> >>>> >>>
>> >>>> >>> The index selection is done in QueryOptimizer, but
I doubt the
>> bug
>> >>>> >>> would be there. Might be in IndexStatementRewriter
which is the
>> code
>> >>>> >>> that re-writes a SQL statement to use the index table
rather than
>> >>>> >>> the
>> >>>> >>> data table.
>> >>>> >>>
>> >>>> >>> Would it be possible to use uppercase table and column
qualifier
>> >>>> >>> names
>> >>>> >>> as a work around?
>> >>>> >>>
>> >>>> >>> Thanks,
>> >>>> >>> James
>> >>>> >>>
>> >>>> >>> On Mon, Jun 30, 2014 at 4:25 PM, Abhilash L L
>> >>>> >>> <abhilash@capillarytech.com> wrote:
>> >>>> >>> > Hello,
>> >>>> >>> >
>> >>>> >>> >    Seems like there is an issue in index selection
when the
>> >>>> >>> > columns
>> >>>> >>> > involved
>> >>>> >>> > are case sensitive. Here is an example from the
one of the
>> slides.
>> >>>> >>> >
>> >>>> >>> >
>> >>>> >>> > ========================================================
>> >>>> >>> > CREATE TABLE baby_names (
>> >>>> >>> >     name VARCHAR PRIMARY KEY,
>> >>>> >>> >     occurrences BIGINT);
>> >>>> >>> >
>> >>>> >>> > CREATE INDEX baby_names_idx ON baby_names(occurrences);
>> >>>> >>> >
>> >>>> >>> > explain SELECT name, occurrences FROM baby_names
WHERE
>> occurrences
>> >>>> >>> > >
>> >>>> >>> > 100;
>> >>>> >>> > -- CLIENT PARALLEL 1-WAY RANGE SCAN OVER BABY_NAMES_IDX
[1E+2]
>> -
>> >>>> >>> > [*]
>> >>>> >>> > ========================================================
>> >>>> >>> >
>> >>>> >>> >
>> >>>> >>> > ========================================================
>> >>>> >>> > CREATE TABLE baby_names (
>> >>>> >>> >     "name" VARCHAR PRIMARY KEY,
>> >>>> >>> >     "occurrences" BIGINT);
>> >>>> >>> >
>> >>>> >>> > CREATE INDEX baby_names_idx ON baby_names("occurrences");
>> >>>> >>> >
>> >>>> >>> > explain SELECT "name", "occurrences" FROM baby_names
WHERE
>> >>>> >>> > "occurrences" >
>> >>>> >>> > 100;
>> >>>> >>> > -- CLIENT PARALLEL 1-WAY FULL SCAN OVER BABY_NAMES
SERVER
>> FILTER
>> >>>> >>> > BY
>> >>>> >>> > occurrences > 100
>> >>>> >>> > ========================================================
>> >>>> >>> >
>> >>>> >>> > We would like to run a patched version of phoenix
for now,
>> since
>> >>>> >>> > we are
>> >>>> >>> > nearing our release date. Would help us a great
deal if someone
>> >>>> >>> > can
>> >>>> >>> > point us
>> >>>> >>> > to the class where the index selection is being
done.
>> >>>> >>> >
>> >>>> >>> > Regards,
>> >>>> >>> > Abhilash L L
>> >>>> >>> > Capillary Technologies
>> >>>> >>> > M:919886208262
>> >>>> >>> > abhilash@capillarytech.com | www.capillarytech.com
>> >>>> >>> >
>> >>>> >>> > Email from people at capillarytech.com may not
represent
>> official
>> >>>> >>> > policy of
>> >>>> >>> > Capillary Technologies unless explicitly stated.
Please see our
>> >>>> >>> > Corporate-Email-Policy for details. Contents of
this email are
>> >>>> >>> > confidential.
>> >>>> >>> > Please contact the Sender if you have received
this email in
>> >>>> >>> > error.
>> >>>> >>> >
>> >>>> >>> >
>> >>>> >>> > Email from people at capillarytech.com may not
represent
>> official
>> >>>> >>> > policy of
>> >>>> >>> > Capillary Technologies unless explicitly stated.
Please see our
>> >>>> >>> > Corporate-Email-Policy for details.Contents of
this email are
>> >>>> >>> > confidential.
>> >>>> >>> > Please contact the Sender if you have received
this email in
>> >>>> >>> > error.
>> >>>> >>
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> > Email from people at capillarytech.com may not represent official
>> >>>> > policy of
>> >>>> > Capillary Technologies unless explicitly stated. Please see
our
>> >>>> > Corporate-Email-Policy for details.Contents of this email are
>> >>>> > confidential.
>> >>>> > Please contact the Sender if you have received this email in
error.
>> >>>
>> >>>
>> >>
>> >
>> >
>> > Email from people at capillarytech.com may not represent official
>> policy of
>> > Capillary Technologies unless explicitly stated. Please see our
>> > Corporate-Email-Policy for details.Contents of this email are
>> confidential.
>> > Please contact the Sender if you have received this email in error.
>>
>
>

-- 
Email from people at capillarytech.com may not represent official policy of 
Capillary Technologies unless explicitly stated. Please see our 
Corporate-Email-Policy for details.Contents of this email are confidential. 
Please contact the Sender if you have received this email in error.

Mime
View raw message