phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kamil <alex.ka...@gmail.com>
Subject Re: speed up ALTER
Date Wed, 17 Sep 2014 10:44:49 GMT
for reference, after replacing ALTER with CREATE it brought down total
bootstrap time from 5hr to 5min, about 1-1.5 sec per CREATE stmt (with 2000
columns per table)
 it looks like  "ALTER TABLE  mytable ADD IF NOT EXISTS  col1 varchar, col2
.." generates individual calls for each column (based on zookeeper trace),
while CREATE does it in one shot. probably expected behavior.

On Wed, Sep 17, 2014 at 1:03 AM, Alex Kamil <alex.kamil@gmail.com> wrote:

> oh, nevermind, there is CSV bulk loader
> <http://phoenix.apache.org/bulk_dataload.html>, somehow I missed it
>
> On Wed, Sep 17, 2014 at 12:53 AM, Alex Kamil <alex.kamil@gmail.com> wrote:
>
>> ..looks like there is a way to create/update Hbase table via MapReduce
>> job and TableOutputFormat, can it be applied to phoenix metadata imports
>>
>> On Wed, Sep 17, 2014 at 12:28 AM, Alex Kamil <alex.kamil@gmail.com>
>> wrote:
>>
>>> Is there a faster alternative to ALTER statement?
>>>
>>> I'm trying to add ~800 columns to a table (3 regionservers,
>>> salt_bucket=3) and it takes >2 min. As I generate about 300 of these
>>> statements during bootstrap it takes hours to finish
>>>
>>> May be via direct update to meta tables via HBase API or even HDFS
>>> files, anything goes
>>>
>>
>>
>

Mime
View raw message