From user-return-6176-apmail-flume-user-archive=flume.apache.org@flume.apache.org Fri Oct 24 07:02:38 2014 Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9EFC6109BB for ; Fri, 24 Oct 2014 07:02:38 +0000 (UTC) Received: (qmail 32233 invoked by uid 500); 24 Oct 2014 07:02:38 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 32177 invoked by uid 500); 24 Oct 2014 07:02:38 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 32167 invoked by uid 99); 24 Oct 2014 07:02:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Oct 2014 07:02:38 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of gshapira@cloudera.com designates 209.85.213.53 as permitted sender) Received: from [209.85.213.53] (HELO mail-yh0-f53.google.com) (209.85.213.53) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Oct 2014 07:02:13 +0000 Received: by mail-yh0-f53.google.com with SMTP id z6so3138085yhz.40 for ; Fri, 24 Oct 2014 00:00:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=zZaG3ifm9z9M6dOcIVkeBCsTD03Fm1EGzs8/CfxVkLU=; b=JrHEjl12tMfVZrQhepE4EZg5jIThtFvxObsPZCGNexG4kCwFswdC8CBT4CGARDRD08 9kZJZsxvWe6nIgc9xFL+wyQ3XjPbYpIPntJjSgsPiz4Ray6sHfDj6f52inH0IjKL20oQ 5rf9cJVczEU/M2Cc7WQZCrppZm4xW7VyaaiBkt6KXZzIkOg/cPEe+1Ic+K05SNUE59+e ZuNm2XyXPcwpgyA9ZKLmGxpTMhTY4bMZMNqDs9nuiejVm/1RlCmsUT5ui5Qpzl4emvlk K8YqIeLiFRxVjSpIafs9AufChE1yXylFOcLPzbVhLSLJjsCRlPdcTWSXY4vEq6ydD5pt +bAg== X-Gm-Message-State: ALoCoQmzMLXtdlUxgtvaOsWS9/o4wfA/wVe/zb3myAtLh/sBj6n7MitKQWMyci8bxPiUp+WfcFwG MIME-Version: 1.0 X-Received: by 10.170.79.65 with SMTP id v62mr3774860ykv.18.1414134041891; Fri, 24 Oct 2014 00:00:41 -0700 (PDT) Received: by 10.170.212.212 with HTTP; Fri, 24 Oct 2014 00:00:41 -0700 (PDT) In-Reply-To: <4D0460317630EE44880FCCD3194AAA5061E05CEB@wdc1exchmbxp04.hq.corp.viasat.com> References: <4D0460317630EE44880FCCD3194AAA5061E05CEB@wdc1exchmbxp04.hq.corp.viasat.com> Date: Fri, 24 Oct 2014 00:00:41 -0700 Message-ID: Subject: Re: Kafka version From: Gwen Shapira To: user@flume.apache.org Content-Type: text/plain; charset=UTF-8 X-Virus-Checked: Checked by ClamAV on apache.org Actually, new and improved Kafka source and sink were committed to Flume trunk (but no release yet). The source is slightly better than the github one - we support buffering by number of messages and by latency. I tested both with 0.8.1.1 and with Kafka trunk. I blogged about them a bit here: http://ingest.tips/2014/09/26/trying-to-decide-between-flume-and-kafka-try-both/ If you happen to use Flume from CDH5.2, you have the Kafka source and sink in your distribution. Gwen On Thu, Oct 23, 2014 at 11:51 PM, Mangtani, Kushal wrote: > Yes, Flume has support for Kafka 0.8.1. Refer the below link for Flume Kafka > sink,source. I have been using this in my prod and it is relatively stable > https://github.com/thilinamb/flume-ng-kafka-sink > > -Kushal Mangtani > ________________________________ > From: Mongeol Heo [mongeol.heo@gmail.com] > Sent: Thursday, October 23, 2014 11:38 PM > To: user@flume.apache.org > Subject: Kafka version > > Hi, > > My question is that will flume sink and source for apache kafka support > latest version of it? > As I know, even sink/source for kafka 0.7.2 is not released yet. > It will be really great if it does. > Thanks. > > Best regards, > > - Mungeol