phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amey hegde <arhegde0...@gmail.com>
Subject Possible Optimization Tips for Hbase via Phoenix
Date Mon, 12 Dec 2016 22:26:01 GMT
I am new to HBASE and Phoenix world. 

I have designed and executed a map reduce job which writes around 2.4 billion cells (rows*columns)
in HBASE via Phoenix in about 80min. I have reduced the "mapreduce.input.fileinputformat.split.maxsize"
to 8MB to increase the number of mapper which helped me to increase throughput and reduce
duration from 140min to 80min. I have wide rows i.e each row has about ~27-30k columns and
most of these ( 95%) are dynamic columns which are created during UPSERT operations.

I wanted guidance with if I can reduce the time duration of my job any further by tweaking
any parameters which are default right now from either HBASE side  or Phoenix side.

Thanks.
Mime
View raw message