flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: HTTPSource is not listening on the port
Date Mon, 31 Dec 2012 18:41:53 GMT
Looks like your config file contains the configuration for an agent named
"agent" and you are trying to run flume with the agent name "agent1". Run
the flume agent as agent or change the name of your agent in the config
file to "agent1" (simply replace all instances of "agent" with "agent1"

Hari

On Monday, December 31, 2012, Felix.徐 wrote:

> Hi all,
>
> I set the flume agent with a HTTPSource , but after I start the agent ,
> the HTTPSource seems not works properly.
> Here is my configuration file:
>
> # The configuration file needs to define the sources,
> # the channels and the sinks.
> # Sources, channels and sinks are defined per agent,
> # in this case called 'agent'
>
> agent.sources = httpSrc
> agent.channels = memoryChannel
> agent.sinks = loggerSink
>
> # For each one of the sources, the type is defined
> agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
> agent.sources.httpSrc.port = 51400
> agent.sources.httpSrc.handler = org.apache.flume.http.JSONHandler
>
> # The channel can be defined as follows.
> agent.sources.httpSrc.channels = memoryChannel
>
> # Each sink's type must be defined
> agent.sinks.loggerSink.type = logger
>
> #Specify the channel the sink should use
> agent.sinks.loggerSink.channel = memoryChannel
>
> # Each channel's type is defined.
> agent.channels.memoryChannel.type = memory
>
> # Other config values specific to each type of channel(sink or source)
> # can be defined as well
> # In this case, it specifies the capacity of the memory channel
> agent.channels.memoryChannel.capacity = 100
>
>
>
> Here is the log after starting:
>
> Info: Sourcing environment configuration script
> /home/extend/flume/apache-flume-1.3.0-bin/conf/flume-env.sh
> + exec /home/jdk1.6.0_37/bin/java -Xmx20m
> -Dflume.root.logger=DEBUG,console -cp
> '/home/extend/flume/apache-flume-1.3.0-bin/conf:/home/extend/flume/apache-flume-1.3.0-bin/lib/*'
> -Djava.library.path= org.apache.flume.node.Application -f
> conf/flume-conf.properties -n agent1
> 2012-12-31 19:43:29,514 (main) [INFO -
> org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
> Starting lifecycle supervisor 1
> 2012-12-31 19:43:29,520 (main) [INFO -
> org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node
> starting - agent1
> 2012-12-31 19:43:29,525 (lifecycleSupervisor-1-0) [INFO -
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)]
> Node manager starting
> 2012-12-31 19:43:29,529 (lifecycleSupervisor-1-1) [INFO -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)]
> Configuration provider starting
> 2012-12-31 19:43:29,529 (lifecycleSupervisor-1-0) [INFO -
> org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
> Starting lifecycle supervisor 9
> 2012-12-31 19:43:29,531 (lifecycleSupervisor-1-0) [DEBUG -
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)]
> Node manager started
> 2012-12-31 19:43:29,532 (lifecycleSupervisor-1-1) [DEBUG -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)]
> Configuration provider started
> 2012-12-31 19:43:29,533 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
> Checking file:conf/flume-conf.properties for changes
> 2012-12-31 19:43:29,534 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)]
> Reloading configuration file:conf/flume-conf.properties
> 2012-12-31 19:43:29,542 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:912)]
> Added sinks: loggerSink Agent: agent
> 2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
> Processing:loggerSink
> 2012-12-31 19:43:29,543 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1002)]
> Created context for loggerSink: type
> 2012-12-31 19:43:29,543 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:998)]
> Processing:loggerSink
> 2012-12-31 19:43:29,544 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
> Starting validation of configuration for agent: agent,
> initial-configuration: AgentConfiguration[agent]
> SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
> type=org.apache.flume.source.http.HTTPSource,
> handler=org.apache.flume.http.JSONHandler} }}
> CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
> SINKS: {loggerSink={ parameters:{type=logger, channel=memoryChannel} }}
>
> 2012-12-31 19:43:29,551 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
> Created channel memoryChannel
> 2012-12-31 19:43:29,573 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:655)]
> Creating sink: loggerSink using LOGGER
> 2012-12-31 19:43:29,587 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
> Post validation configuration for agent
> AgentConfiguration created without Configuration stubs for which only
> basic syntactical validation was performed[agent]
> SOURCES: {httpSrc={ parameters:{port=51400, channels=memoryChannel,
> type=org.apache.flume.source.http.HTTPSource,
> handler=org.apache.flume.http.JSONHandler} }}
> CHANNELS: {memoryChannel={ parameters:{capacity=100, type=memory} }}
> AgentConfiguration created with Configuration stubs for which full
> validation was performed[agent]
> SINKS: {loggerSink=ComponentConfiguration[loggerSink]
>   CONFIG:
>     CHANNEL:memoryChannel
> }
>
> 2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
> Channels:memoryChannel
>
> 2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)]
> Sinks loggerSink
>
> 2012-12-31 19:43:29,588 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)]
> Sources httpSrc
>
> 2012-12-31 19:43:29,589 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:122)]
> Post-validation flume configuration contains configuration  for agents:
> [agent]
> 2012-12-31 19:43:29,589 (conf-file-poller-0) [WARN -
> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:227)]
> No configuration found for this host:agent1
> 2012-12-31 19:43:59,589 (conf-file-poller-0) [DEBUG -
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
> Checking file:conf/flume-conf.properties for changes
>

Mime
View raw message