flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Grimes <frankgrime...@yahoo.com>
Subject agentBESink and logical nodes
Date Fri, 14 Oct 2011 15:17:43 GMT

Is there a way to have an agentBESink map to a logical node? 
It seems to only expect machine/port despite what's documented in section 8.2.3 of the user

I tried expanding it myself to:
 lazyOpen stubbornAppend logicalSink("logicalNodeName") 

However, it doesn't seem to back off correctly when the collector node goes down. It dumps
append errors to the console, chewing up a bunch of CPU.

In digging into AgentSink.java, I found code which expands BEST_EFFORT to:
  "< " + batchGzDeco + " insistentOpen stubbornAppend rpcSink(\"%s\", %d) ? null>"

So I then tried writing my agent node config as:
  < insistentOpen stubbornAppend logicalSink("logicalNodeName") ? null>

Still, it doesn't back off when the collector node becomes unavailable.
If I specify it with agentBESink("machine", port) it backs off correctly.

I've tried this with 0.9.4 and the latest 0.9.5.SNAPSHOT.

What am I missing here?


Frank Grimes
View raw message