flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mingjie Lai <mjla...@gmail.com>
Subject Re: collectorSink is always using default from flume-conf.xml
Date Tue, 27 Sep 2011 19:00:50 GMT
Thai.

I checked out the 0.9.4-branch from git://github.com/cloudera/flume.git, 
but didn't see the problem.

In the source code, you should override build() instead of create() here.

-mingjie

On 09/27/2011 03:59 AM, Bao Thai Ngo wrote:
> any thought?
>
> On Wed, Sep 21, 2011 at 4:19 PM, Bao Thai Ngo <baothaingo@gmail.com
> <mailto:baothaingo@gmail.com>> wrote:
>
>     I'd also like to keep you informed that when compiling FLume
>     Helloworld (modified according to some changes in version 0.9.4):
>     ----------------------------------
>       public static SinkDecoBuilder builder() {
>          return new SinkDecoBuilder() {
>            // construct a new parameterized decorator
>            @Override
>            public EventSinkDecorator<EventSink> create(Context
>     context,Object... argv) {
>              Preconditions.checkArgument(argv.length == 0,"usage:
>     helloWorldDecorator");
>
>              return new HelloWorldDecorator<EventSink>(null);
>            }
>
>          };
>        }
>
>     -----------------------
>
>     I get the following error:
>
>     [javac]
>     /usr/lib/flume/plugins/helloworld/src/HelloWorldDecorator.java:34:
>     <anonymous helloworld.HelloWorldDecorator$1> is not abstract and
>     does not override abstract method
>     build(com.cloudera.flume.conf.Context,java.lang.String...) in
>     com.cloudera.flume.conf.SinkFactory.SinkDecoBuilder
>          [javac]     return new SinkDecoBuilder() {
>
>     Was I missing something?
>
>     ~Thai
>
>
>     On Wed, Sep 21, 2011 at 3:41 PM, Bao Thai Ngo <baothaingo@gmail.com
>     <mailto:baothaingo@gmail.com>> wrote:
>
>         Hello list,
>
>         After updating to Flume v0.9.4-cdh3u1, I do meet this error:
>         WARN com.cloudera.flume.conf.FlumeBuilder: Deprecated syntax:
>         Expected a format spec but instead had a (String) raw
>         As a result, collector does not save events on storage backend
>
>         Before the updating, my plug-ins worked completely well. Below
>         is my typical config:
>         exec config node1 test
>         'tail("/path/to/test.txt",startFromEnd="true")'
>         '{deco_modiy_event_body()=>autoDFOChain}'
>
>         Any thoughts?
>
>         Thanks,
>         ~Thai
>
>         On Sat, Sep 10, 2011 at 4:14 AM, Jeff Hansen <dscheffy@gmail.com
>         <mailto:dscheffy@gmail.com>> wrote:
>
>             Sorry, I should have also mentioned that the output itself
>             is coming
>             out in the raw format (consistent with the xml config rather
>             than the
>             runtime config)
>
>             On Fri, Sep 9, 2011 at 4:10 PM, Jeff Hansen
>             <dscheffy@gmail.com <mailto:dscheffy@gmail.com>> wrote:
>              > When I configure a collectorSink to use a format
>             different from the
>              > default specified in the flume-conf.xml file, it
>             continues to use the
>              > default.  I get the following error
>              >
>              > 2011-09-09 16:02:29,059 [Roll-TriggerThread-0] WARN
>             conf.FlumeBuilder:
>              > Deprecated syntax: Expected a format spec but instead had
>             a (String)
>              > raw
>              >
>              > even though I'm using the following combination of
>             configurations.  Is
>              > the xml file configuration supposed to supercede that of
>             the runtime
>              > configuration?
>              >
>              > I'm running version "Flume 0.9.4-cdh3u1"
>              >
>              > Here's the complete startup script I'm using:
>              >
>              >    #!/bin/sh
>              >    gnome-terminal -e "flume master"
>              >    sleep 10
>              >    flume shell -c localhost -e "exec config agent
>              > 'tail(\"/var/log/apache2/access.log\")' '[console,
>              >
>             collectorSink(\"hdfs://localhost/flume/avro/\",\"log\",60000,avrojson)]'"
>              >    gnome-terminal -e "flume node -n agent"
>              >
>              > Here's the appropriate entry from my flume-conf.xml file:
>              > <property>
>              > <name>flume.collector.output.format</name>
>              > <value>raw</value>
>              > <description>The output format for the data written by a
>             Flume
>              >    collector node.  There are several formats available:
>              >      syslog - outputs events in a syslog-like format
>              >      log4j - outputs events in a pattern similar to
>             Hadoop's log4j pattern
>              >      raw - Event body only.  This is most similar to
>             copying a file but
>              >        does not preserve any uniqifying metadata like
>             host/timestamp/nanos.
>              >      avro - Avro Native file format.  Default currently
>             is uncompressed.
>              >      avrojson - this outputs data as json encoded by avro
>              >      avrodata - this outputs data as a avro binary
>             encoded data
>              >      debug - used only for debugging
>              > </description>
>              > </property>
>              >
>
>
>
>

Mime
View raw message