phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cox, Jonathan A" <ja...@sandia.gov>
Subject Java Out of Memory Errors with CsvBulkLoadTool
Date Fri, 18 Dec 2015 00:46:41 GMT
I am trying to ingest a 575MB CSV file with 192,444 lines using the CsvBulkLoadTool MapReduce
job. When running this job, I find that I have to boost the max Java heap space to 48GB (24GB
fails with Java out of memory errors).

I'm concerned about scaling issues. It seems like it shouldn't require between 24-48GB of
memory to ingest a 575MB file. However, I am pretty new to Hadoop/HBase/Phoenix, so maybe
I am off base here.

Can anybody comment on this observation?

Thanks,
Jonathan

Mime
View raw message