I'm currently plan to migrate from Flume 0.9 to Flume-ng 1.6, but I'm having troubles trying to find a reliable setup for this one.
My sink is a 3 nodes Kafka cluster. I must avoid to lose events in case the main sink is down, broken or unreachable for a while.
In Flume 0.9, I use a memory channel with the store on failure feature, which starts writing events on the local disk in case the target sink is not available.
In Flume-ng 1.6 the same behaviour would be accomplished by setting up a Spillable memory channel, but the problem with this solution is written in the end of the channel's description: "This channel is currently experimental and not recommended for use in production."
In Flume-ng 1.6, it's possible to setup a pool of Failover sinks. So, I was thinking to hypothetically configure a File Roll as Secondary sink in case the Primary is down. However, once the Primary sink would be back online, the data placed on the Secondary sink (local disk) won't be automatically pushed on the Primary one.
Another option would be setting up a file channel: write each event on the disk and then sink. Without mentioning that I don't love the idea to write/delete each single event continuously on a SSD, this setup is taking 90% of CPU. The same exactly configuration but using a memory channel takes 3%.
Other solutions to evaluate ?