phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sriram Nookala <sri...@firstfuel.com>
Subject Re: Phoenix CSV Bulk Load fails to load a large file
Date Wed, 06 Sep 2017 21:01:26 GMT
Phoenix 4.11.0, HBase 1.3.1

This is what I get from jstack

"main" #1 prio=5 os_prio=0 tid=0x00007fb3d0017000 nid=0x5de7 waiting on
condition [0x00007fb3d75f7000]

   java.lang.Thread.State: WAITING (parking)

at sun.misc.Unsafe.park(Native Method)

- parking to wait for  <0x00000000f2222588> (a
java.util.concurrent.FutureTask)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)

at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)

at java.util.concurrent.FutureTask.get(FutureTask.java:191)

at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:604)

at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:463)

at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:373)

at
org.apache.phoenix.mapreduce.AbstractBulkLoadTool.completebulkload(AbstractBulkLoadTool.java:355)

at
org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:332)

at
org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:270)

at
org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:183)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at
org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:101)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)




On Wed, Sep 6, 2017 at 4:16 PM, Sergey Soldatov <sergeysoldatov@gmail.com>
wrote:

> Do you have more details on the version of Phoenix/HBase you are using as
> well as how it hangs (Exceptions/messages that may help to understand the
> problem)?
>
> Thanks,
> Sergey
>
> On Wed, Sep 6, 2017 at 1:13 PM, Sriram Nookala <sriram@firstfuel.com>
> wrote:
>
>> I'm trying to load a 3.5G file with 60 million rows using
>> CsvBulkLoadTool. It hangs while loading HFiles. This runs successfully if I
>> split this into 2 files, but I'd like to avoid doing that. This is on
>> Amazon EMR, is this an issue due to disk space or memory. I have a single
>> master and 2 region server configuration with 16 GB memory on each node.
>>
>
>

Mime
View raw message