phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: [IndexTool NOT working] mapreduce.LoadIncrementalHFiles: Split occured while grouping HFiles
Date Tue, 30 May 2017 03:04:44 GMT
Hey Anil,

That's a relatively old version of HDP. Do you have options to update to 
something more recent?

This reads like a bug to me (a JIRA ID escapes me though). However, if 
you would manage to get around this, there are more to come.

On 5/25/17 7:38 PM, anil gupta wrote:
> Hi,
> 
> We are using HDP2.3.2(Phoenix 4.4 and HBase 1.1), we created a secondary 
> index on an already existing table. We paused all writes to Primary 
> table. Then we invoked IndexTool to populate secondary index table. We 
> have tried same steps many times but we keep on getting following 
> error(we have also tried drop the index and adding it again):
> 
> 2017-05-24 18:00:10,281 WARN  [LoadIncrementalHFiles-2] 
> util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit 
> is deprecated by hbase.regionserver.global.memstore.size
> 2017-05-24 18:00:10,340 WARN  [LoadIncrementalHFiles-12] 
> util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit 
> is deprecated by hbase.regionserver.global.memstore.size
> 2017-05-24 18:00:10,342 INFO  [LoadIncrementalHFiles-11] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/c79ae6d27824424f99523dad586e86b1

> first=JF2GPADC8GH331037\x00\x80\x00\x1A0\x80\x00\x01Wj\x03r1defc4d301e4ec172b49be4a7ea33c2f7

> last=JTHBK1GG4E2122477\x00\x80\x00$\xE4\x80\x00\x01[\xAD`{\x17901d036d588292854ac5b1d4c29d8e1e
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-14] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/f0e97b218aed4abf9949cf49a57e559b

> first=5NPEB4AC3DH620091\x00\x80\x00\xE0\x16\x80\x00\x01X\xE5g\xD6\x0B81d210ac753ed281e8627e5edb7eb59f

> last=JF2GPADC8GH331037\x00\x80\x00\x1A0\x80\x00\x01W]&\xE54f37d636104f6cd916b2b07bf3aa94d3f
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-2] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/27c0a905ee174c9898d324acf1554bf9

> first=WMWZP3C58FT708786\x00\x80\x00\xE0\x16\x80\x00\x01Y\xB8\x95U\xA0d21d32aed18af976dd53735705c728cd

> last=`1GCRCPE05BZ430377\x00\x80\x00}\x05\x80\x00\x01[\xDEE\x91L383768c6ac5f306fa99f68964b4f18aa
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-12] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/d0a6c4b727bb416f840ed254658f3982

> first=1N4BZ0CP4GC308715\x00\x80\x01T\xFC\x80\x00\x01U\xE3\x7FL\x9A37b77d47941e99e430fcb0e0657f5558

> last=2GKALMEK7H6220949\x00\x80\x00!\x1A\x80\x00\x01Y\x18\xE6\xB3\xB42e72036f7e7e03078f41fc82712c5de7
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-0] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/11627a2861e3446e9d6f684ab534563e

> first=3C7WRNAJ6GG313342\x00\x80\x00NB\x80\x00\x01V}\xFD\xE4+65bbebdd06dedd8466a31ebd33841a51

> last=3N1CE2CP2FL407481\x00\x80\x00\xE0\x16\x80\x00\x01W\x1B\x0A\x02\xC1fc95d4114d5e91197a5e41bf37c9e8c7
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-1] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/23df78bafd304ff887385a2b6becf06d

> first=1C4RJFLT6HC742023\x00\x80\x00x\xFF\x80\x00\x01[J8\x0Ac8b65a80fe1662fb25d80798a66cc83dc

> last=1FMCU9J90EUB68140\x00\x80\x01X\xA4\x80\x00\x01[\x1B\xDD\xB2\x1C577502512ec987844b0108738a9ec6ba
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-3] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/39dc73882bec49a0bdd5d787b06ac032

> first=1G1JD5SB5H4136951\x00\x80\x00!\x8A\x80\x00\x01Z%\xF6\x7Ffef0b8faeeeb4a10103e1a67ea5ebdbec

> last=1GNKVHKD7HJ275239\x00\x80\x00$\x87\x80\x00\x01Z%\xF6s\xDC0961566a370af3b7da440e9705bc4c8c
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-8] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/a37a2a56ff5c48399cf1abd92f99662f

> first=###########239824\x00\x80\x01(\xFE\x80\x00\x01Z\xAE\xD6\xE0Xe5a45a2beab337228bdba90c06f34a12

> last=1C4RJFLT6HC742023\x00\x80\x00x\xFF\x80\x00\x01[H\xF9w\x8D60edb518c27ef80f8a751701926d9174
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-10] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/b5843a62a6bd47fbbfc29303bee158e3

> first=5FNRL5H4XFB033259\x00\x80\x00\x1EZ\x80\x00\x01\x5C"\x87s\xF5ce24ec7e2a3698836386bccabc1265af

> last=5NPEB4AC3DH620091\x00\x80\x00\xE0\x16\x80\x00\x01X\xE4\x1B\x9Dq95568f371c1ebd06c497df7129f248a2
> 2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-5] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/595e89b2fae8494d8a878bc6ba306e2f

> first=JTHBK1GG4E2122477\x00\x80\x00$\xE4\x80\x00\x01[\xAF\xF0]%d306ddb81ea3bc093c40efe9f198f03a

> last=KNDPMCAC1H7201793\x00\x80\x00\x1E\xC7\x80\x00\x01X\xD5\xF5\xEA\xD183ff28e7e1aeeff315b167d3d869b797
> 2017-05-24 18:00:10,348 INFO  [LoadIncrementalHFiles-9] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/ac4edbfdf7394e88aaf84f5489511b7e

> first=1GNKVHKD7HJ275239\x00\x80\x00$\x87\x80\x00\x01Z+\x1C\x87\xB65ab1c776dde09cf3257821e420288075

> last=1N4BZ0CP4GC308715\x00\x80\x01T\xFC\x80\x00\x01U\xE2\xA1 
> \xB743fe1c3b744fae3c2f9e6a896003869f
> 2017-05-24 18:00:10,348 INFO  [LoadIncrementalHFiles-4] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/5580694921e846949f49c1426162da6e

> first=2GKALMEK7H6220949\x00\x80\x00!\x1A\x80\x00\x01Y!\xE5\xCF\xC110e1f23860ff6a21fe848bb85e2e0969

> last=3C7WRNAJ6GG313342\x00\x80\x00NB\x80\x00\x01V}\x1D\xF5\x10620faf6227fb4181a7ee99fa07a52f08
> 2017-05-24 18:00:10,349 INFO  [LoadIncrementalHFiles-7] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/7bc03c5b15304ec083e18c2376789562

> first=1FMCU9J90EUB68140\x00\x80\x01X\xA4\x80\x00\x01[\x1D&\x9D\x12f80b9cac0af584642ecbd6e1b1bbd617

> last=1G1JD5SB5H4136951\x00\x80\x00!\x8A\x80\x00\x01Z$\xB0\xAC\x1625dc1ae357cb3e9332d823210102de85
> 2017-05-24 18:00:10,349 INFO  [LoadIncrementalHFiles-13] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/d28c4d2668cc4169b201b349b2522f9a

> first=3N1CE2CP2FL407531\x00\x80\x00\x0A\xC9\x80\x00\x01X\xD1\xBC\xD195d73fa7cd13082d2e6f590ca1578b453

> last=5FNRL5H4XFB033259\x00\x80\x00\x1EZ\x80\x00\x01\x5C!8\xA7.dd776deeea36c2c5b132fafed7013b16
> 2017-05-24 18:00:10,350 INFO  [LoadIncrementalHFiles-6] 
> mapreduce.LoadIncrementalHFiles: Trying to load 
> hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/5d90de564af645f084319a03f64b0a3d

> first=KNDPMCAC1H7201793\x00\x80\x00\x1E\xC7\x80\x00\x01X\xD7F#\xEBcd2872ac4f258a091a79a55d4dbfef9e

> last=WMWZP3C58FT708786\x00\x80\x00\xE0\x16\x80\x00\x01Y\xB7?\xCA\xADb28da871b5581e409561c813ed9c0805
> 2017-05-24 18:00:11,780 WARN  [LoadIncrementalHFiles-15] 
> mapreduce.LoadIncrementalHFiles: Attempt to bulk load region containing  
> into table BI.VIN_IDX with files [family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/d28c4d2668cc4169b201b349b2522f9a,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/23df78bafd304ff887385a2b6becf06d,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/39dc73882bec49a0bdd5d787b06ac032,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/a37a2a56ff5c48399cf1abd92f99662f,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/11627a2861e3446e9d6f684ab534563e,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/ac4edbfdf7394e88aaf84f5489511b7e,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/f0e97b218aed4abf9949cf49a57e559b,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/7bc03c5b15304ec083e18c2376789562,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/5d90de564af645f084319a03f64b0a3d,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/5580694921e846949f49c1426162da6e,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/595e89b2fae8494d8a878bc6ba306e2f,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/27c0a905ee174c9898d324acf1554bf9,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/c79ae6d27824424f99523dad586e86b1,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/b5843a62a6bd47fbbfc29303bee158e3,

> family:I 
> path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/d0a6c4b727bb416f840ed254658f3982]

> failed.  This is recoverable and they will be retried.
> 2017-05-24 18:00:11,793 INFO  [main] mapreduce.LoadIncrementalHFiles: 
> Split occured while grouping HFiles, retry attempt 1 with 15 files 
> remaining to group or split
> 
> 
> -- 
> Thanks & Regards,
> Anil Gupta

Mime
View raw message