flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhishan Li <zhishan...@gmail.com>
Subject Data loss by HDFS SINK metrics
Date Wed, 09 Dec 2015 03:40:42 GMT
All,

Dose the HDFS Sink loss data?

Here is metrics from http monitor feature.
"SINK.k1": {
  "BatchCompleteCount": "824",
  "ConnectionFailedCount": "0",
  "EventDrainAttemptCount": "4124988",
  "ConnectionCreatedCount": "28",
  "Type": "SINK",
  "BatchEmptyCount": "0",
  "ConnectionClosedCount": "20",
  "EventDrainSuccessCount": "4120000",
  "StopTime": "0",
  "StartTime": "1449490280602",
  "BatchUnderflowCount": "0"
},

EventDrainAttemptCount: 4124988 is what HDFS sink received events.
But EventDrainSuccessCount: 4120000 is those events flushed to hdfs. Right?

Where is other events: 4988 ( =4124988 - 4120000) ?

Thanks


Mime
View raw message