flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Stricks <...@wapolabs.com>
Subject Re: Re: Re: encounter "Address already in use" WARN during starting a node
Date Tue, 06 Mar 2012 03:56:12 GMT
Nah, that would be like having to install Firefox multiple times to have
multiple windows open, you know what I mean?

The user guide is pretty straightforward on how to set up a single node and
master. The simplest case is to have them both on the same server, but that
obviously doesn't need to be the case.

You start a Flume node by executing {$FLUME_HOME}/bin/flume node -n
{node_name}.  Then you start a master with {$FLUME_HOME}/bin/flume master.
You submit the configs to the master (via Flume shell or via the master's
web interface; examples are in the User Guide) to indicate source and sink
for the agent. Then, just start up a new Flume agent (on the same server or
not as either of the first two) with a new name.  Submit its configs to the
master.  You may have to change a couple of the default fields in
the{$FLUME_HOME}/conf/flume-conf./xml file depending on whether the master
is on a different server or not;  save the changes in
{$FLUME_HOME}/conf/flume-site.xml.

Just go through the user guide again, maybe even just the first few
sections to see it makes sense now.

2012/3/5 lulynn_2008 <lulynn_2008@163.com>

> Thanks for explanation.
> "Each of those has to have its own flume process running and its own
> configurations"
> Is that mean we need to install multiple flume in a host?
>
>
> At 2012-03-06 11:02:03,"Jay Stricks" <jay@wapolabs.com> wrote:
>
> You can have multiple source-sink pairs on the same physical machine.
> Each of those has to have its own flume process running and its own
> configurations, though. So you give each a unique name, and then execute
> the {$FLUME_HOME}/bin/flume node -n {node_name} for each, or adjust the
> initialization script you're using.
>
> 2012/3/5 lulynn_2008 <lulynn_2008@163.com>
>
>> Yes, there is already a node before we start. Do you mean we just can
>> start 1 agent node in a host?
>>
>>
>>
>> At 2012-03-05 20:02:53,"alo alt" <wget.null@googlemail.com> wrote:
>> >Hi,
>> >
>> >1. as the message says - address in use. Check with ps if more as one process
is running at one time. Stop the node, do ps and kill all processes with flume inside (ps
waux|grep flume, if the process are runs as flume you can try killall -9 -u flume)
>> >2. as in the mail before, 0.94 does not work with hadoop-1.0.0
>> >3. the flume master isn't available, or the port is blocked or network issue
or something like that
>> >
>> >- Alex
>> >
>> >
>> >--
>> >Alexander Lorenz
>> >http://mapredit.blogspot.com
>>
>> >
>> >On Mar 5, 2012, at 12:40 PM, lulynn_2008 wrote:
>> >
>> >>  Hi,
>> >> -  Every time I start a agent node, the following WARN appears. Is there
any way to avoid this?
>> >> -  Besides, I found this "Flume is using Hadoop core 1.0.0 which does not
support Security / Authentication: null". Is this mean flume-0.9.4 can not work with hadoop-1.0.0?
>> >> -  Is there any problem with this "No active master RPC connection"?
>> >>
>> >> 2012-03-05 03:02:04,483 [main] WARN util.InternalHttpServer: Caught exception
during HTTP server start.
>> >> java.net.BindException: Address already in use
>> >>         at sun.nio.ch.Net.bind(Native Method)
>> >>         at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:156)
>> >>         at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:71)
>> >>         at org.m ortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
>> >>         at org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:315)
>> >>         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>> >>         at org.mortbay.jetty.Server.doStart(Server.java:235)
>> >>         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>> >>         at com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:188)
>> >>         at com.cloudera.flume.agent.FlumeNode.start(FlumeNode.java:275)
>> >>         at com.cloudera.flume.agent.FlumeNode.setup(FlumeNode.java:504)
>> >>         at com.cloudera.flume.agent.FlumeNode.main(FlumeNode.j ava:665)
>> >> 2012-03-05 03:02:04,485 [main] ERROR agent.FlumeNode: Unexpected exception/error
thrown! Unable to start HTTP server
>> >> com.cloudera.util.InternalHttpServer$InternalHttpServerException: Unable
to start HTTP server
>> >>         at com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:194)
>> >>         at com.cloudera.flume.agent.FlumeNode.start(FlumeNode.java:275)
>> >>         at com.cloudera.flume.agent.FlumeNode.setup(FlumeNode.java:504)
>> >>         at com.cloudera.flume.agent.FlumeNode.main(FlumeNode.java:665)
>> >> Caused by: java.net.BindException: Address already in use
>> >>         at sun.nio.ch.Net.bind(Native Method)
>> >>         at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:156)
>> >>         at sun. nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:71)
>> >>         at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
>> >>         at org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:315)
>> >>         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>> >>         at org.mortbay.jetty.Server.doStart(Server.java:235)
>> >>         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>> >>         at com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:188)
>> >>         ... 3 more
>> >> 2012-03-05 03:02:04,527 [Heartbeat] INFO agent.MultiMasterRPC: No active
master RPC connection
>> >> 2012-03-05 03:02:04,568 [Heartbeat] INFO agent. ThriftMasterRPC: Connected
to master at svltest150.svl.ibm.com:35872
>> >> 2012-03-05 03:02:04,711 [main] INFO agent.LogicalNodeManager: creating new
logical node agent
>> >> 2012-03-05 03:02:04,879 [main] WARN agent.FlumeNode: Flume is using Hadoop
core 1.0.0 which does not support Security / Authentication: null
>> >>
>> >>
>> >
>>
>>
>>
>>
>
>
>

Mime
View raw message