flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <shouvanik.hal...@accenture.com>
Subject RE: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1
Date Tue, 11 Dec 2012 08:11:55 GMT
Hi,

This error is removed. I copied flume-node as flume-master, replace all "node" with "master"
and thus it worked! Thanks

Now, I am facing a new weird error. I had mentioned source and sink. Now, Flume node status
is set to "OPENING" and nothing else is happening even I do a refresh.

In the flume-master console, I executed the below mentioned commands at the "Configure multiple
nodes" textarea!

Please help me. Which logs should I check!

Commands at "Configure multiple nodes" textarea
---------------------------------------------------
ec2-75-101-165-36.compute-1.amazonaws.com:tail("/var/log/flume/test1.log")|agentBESink("ec2-75-101-165-36.compute-1.amazonaws.com",35853);
ec2-75-101-165-36.compute-1.amazonaws.com:collectorSource(35853)|EHCacheSink("Y","/var/log/flume");

When I tail the log at "/var/log/flume/flume-flume-node-ip-10-40-222-77.log", the logs don't
move after

"2012-12-11 02:59:48,598 INFO com.cloudera.flume.conf.SinkFactoryImpl: Found sink decorator
EHCacheDecorator in com.EHCacheDecorator
2012-12-11 02:59:48,644 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully
set to FlumeConfigData: {srcVer:'Wed Dec 31 19:00:00 EST 1969' snkVer:'Wed Dec 31 19:00:00
EST 1969'  ts='Wed Dec 31 19:00:00 EST 1969' flowId:'null' source:'null' sink:'null' }
2012-12-11 02:59:48,771 INFO com.cloudera.flume.agent.FlumeNode: Hadoop Security enabled:
false
2012-12-11 02:59:53,484 INFO com.EHCacheSink: EHCacheSink: setting EHCache Debug=Y debugfilePath=/var/log/flume
2012-12-11 02:59:53,486 INFO com.cloudera.flume.collector.CollectorSource: opened
2012-12-11 02:59:53,486 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully
set to FlumeConfigData: {srcVer:'Tue Dec 11 02:55:27 EST 2012' snkVer:'Tue Dec 11 02:55:27
EST 2012'  ts='Tue Dec 11 02:55:27 EST 2012' flowId:'default-flow' source:'collectorSource(
35853 )' sink:'EHCacheSink( "Y", "/var/log/flume" )' }
2012-12-11 02:59:53,498 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting
blocking thread pool server on port 35853..."


When I do a "netstat -nlp| grep 35853"
I get
"[root@ip-10-40-222-77 bin]# netstat -nlp| grep 35853
tcp        0      0 :::35853                    :::*                        LISTEN      21096/java

"

Please help!

Thanks and Regards,
Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 

-----Original Message-----
From: Alexander Alten-Lorenz [mailto:wget.null@gmail.com] 
Sent: Tuesday, December 11, 2012 1:03 PM
To: user@flume.apache.org
Cc: nitinpawar432@gmail.com
Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog:
Subprocess exited with value 1

Hi,

yum remove --force <paketname>

Or simply create a empty file called /etc/init.d/flume-master:

echo "foo" > /etc/init.d/flume-master

best
- Alex

On Dec 11, 2012, at 8:27 AM, shouvanik.haldar@accenture.com wrote:

> Hi,
> 
> Can you please help me on one thing! Mistakenly deleted /etc/init.d/flume-master script.
Now flume-master is not getting uninstalled or removed! Please help!
> 
> Thanks and Regards,
> Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568
> 
> From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
> Sent: Tuesday, December 11, 2012 12:55 PM
> To: Haldar, Shouvanik
> Cc: user@flume.apache.org
> Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog:
Subprocess exited with value 1
> 
> there is no master/slave architecture now
> 
> its a simple conf file like below <@EOM> and then just start the agent. it will
do needful
> you need to start flume like the command below
> 
> 
> 
> 
> bin/flume-ng agent --conf-file example.conf --name agent1
> 
> 
> agent1.sources = tail
> agent1.channels = MemoryChannel-2
> agent1.sinks = HDFS
> agent1.sources.tail.type = exec
> agent1.sources.tail.command = tail -F <file>
> agent1.sources.tail.channels = MemoryChannel-2
> agent1.sources.tail.interceptors = hostint
> agent1.sources.tail.interceptors.hostint.type = org.apache.flume.interceptor.HostInterceptor$Builder
> agent1.sources.tail.interceptors.hostint.preserveExisting = true
> agent1.sources.tail.interceptors.hostint.useIP = true
> agent1.sinks.HDFS.channel = MemoryChannel-2
> agent1.sinks.HDFS.type = hdfs
> agent1.sinks.HDFS.hdfs.path = hdfs://url:8020/user/flume/%{host}
> agent1.sinks.HDFS.hdfs.file.Type = SequenceFile
> agent1.sinks.HDFS.hdfs.writeFormat = Text
> agent1.sinks.HDFS.hdfs.rollInterval = 30
> agent1.sinks.HDFS.hdfs.rollCount = 0
> agent1.sinks.HDFS.hdfs.rollSize = 5000000000
> agent1.channels.MemoryChannel-2.type = memory
> 
> more examples at http://flume.apache.org/FlumeUserGuide.html
> 
> 
> On Tue, Dec 11, 2012 at 12:44 PM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
> Hi,
> I installed flume 1.x higher version today! But is this very much different from the
earlier. ? Is there no master / node concept here? How do I configure the source and sink?
> 
> Think I got screwed here!
> 
> Thanks and Regards,
> Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568>
> 
> From: Nitin Pawar [mailto:nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>]
> Sent: Monday, December 10, 2012 3:34 PM
> 
> To: Haldar, Shouvanik
> Cc: user@flume.apache.org<mailto:user@flume.apache.org>
> Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog:
Subprocess exited with value 1
> 
> just a random question, does the user running flume agent have permission to read the
 event details out of terracotta ?
> 
> from the code here http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera/flume-core/0.9.4-cdh4.1.0/com/cloudera/flume/watchdog/Watchdog.java
> 
> line 209 onwards, it looks like the error condition is achieved when you have channel
set as stdin (I am not sure about this so you may need to look at code which part of code
you are hitting depending on the flume configs you have)
> 
> 
> 
> On Mon, Dec 10, 2012 at 3:01 PM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
> Hello,
> 
> I am sure, the problem is due to Terracotta. Can you please help me?
> 
> Thanks and Regards,
> Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568>
> 
> From: Nitin Pawar [mailto:nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>]
> Sent: Monday, December 10, 2012 2:43 PM
> To: Haldar, Shouvanik
> Cc: user@flume.apache.org<mailto:user@flume.apache.org>
> 
> Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog:
Subprocess exited with value 1
> 
> I just hope you are not hitting this https://issues.apache.org/jira/browse/FLUME-796
> I would also like to recommend you to upgraded to flume 1.x asap
> 
> if possible can you try changing your flume source to some random file with tail -f and
try sinking it to some file. That will narrow the problem is on flume side or terracotta side
> 
> Thanks,
> Nitin
> 
> On Mon, Dec 10, 2012 at 1:47 PM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
> Hi,
> 
> I am using Flume -> Flume 0.9.4-cdh3u5.
> 
> Initially, the logs are ok. But when I start terracotta (./start-tc-server.sh) , then
only I experience the above phenomena! Is it something to do with terracotta.!
> 
> 
> 2012-12-10 03:14:08,806 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting
blocking thread pool server on port 35853...
> 2012-12-10 03:14:08,894 INFO net.sf.ehcache.terracotta.TerracottaClient: Thread [logicalNode
ec2-75-101-165-36.compute-1.amazonaws.com-19] [cacheManager: 'RTLogCacheManager']: Creating
new ClusteredInstanceFactory
> 
> Before starting terracotta, Flume Node status is "OPENING".
> But when I start terracotta, I got the error "INFO com.cloudera.flume.watchdog.Watchdog:
Subprocess exited with value 1"
> 
> Please help !
> 
> 
> Thanks and Regards,
> Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568>
> 
> From: Nitin Pawar [mailto:nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>]
> Sent: Monday, December 10, 2012 12:52 PM
> To: user@flume.apache.org<mailto:user@flume.apache.org>
> Cc: Haldar, Shouvanik
> Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog:
Subprocess exited with value 1
> 
> what version of flume are you using? If it is < 1.x then recommend you to upgrade
to latest 1.0.3
> If you are on 1.x then can you tell us whats the hostname for your server?
> 
> On Mon, Dec 10, 2012 at 11:40 AM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
> Before the below mentioned error, I get a success message like
> 
> 2012-12-10 01:09:14,403 INFO net.sf.ehcache.terracotta.TerracottaClient: Thread [logicalNode
ec2-75-101-165-36.compute-1.amazonaws.com-19] [cacheManager: 'RTLogCacheManager']: Creating
new ClusteredInstanceFactory
> 
> Thanks and Regards,
> Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568>
> 
> From: Haldar, Shouvanik
> Sent: Monday, December 10, 2012 11:34 AM
> To: 'user@flume.apache.org<mailto:user@flume.apache.org>'
> Subject: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog:
Subprocess exited with value 1
> 
> I had configured flume-master as well as flume-node on a single machine.
> Now, as I run terracotta -> L1 cache, I get the INFO below in /
> 
> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited
with value 1
> 
> Can you please help? I am struggling with this for the past 1 day without success!
> 
> Thanks and Regards,
> Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568<tel:%2B91-9830017568>
> 
> 
> ________________________________
> This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise private information. If you have received it in error, please notify the sender
immediately and delete the original. Any other use of the e-mail by you is prohibited.
> 
> Where allowed by local law, electronic communications with Accenture and its affiliates,
including e-mail and instant messaging (including content), may be scanned by our systems
for the purposes of information security and assessment of internal compliance with Accenture
policy.
> 
> ______________________________________________________________________________________
> 
> www.accenture.com<http://www.accenture.com>
> 
> 
> 
> --
> Nitin Pawar
> 
> 
> 
> --
> Nitin Pawar
> 
> 
> 
> --
> Nitin Pawar
> 
> 
> 
> --
> Nitin Pawar

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF




Mime
View raw message