flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1
Date Tue, 11 Dec 2012 07:24:56 GMT
there is no master/slave architecture now

its a simple conf file like below <@EOM> and then just start the agent. it
will do needful
you need to start flume like the command below

bin/flume-ng agent --conf-file example.conf --name agent1



agent1.sources = tail
agent1.channels = MemoryChannel-2
agent1.sinks = HDFS
agent1.sources.tail.type = exec
agent1.sources.tail.command = tail -F <file>
agent1.sources.tail.channels = MemoryChannel-2
agent1.sources.tail.interceptors = hostint
agent1.sources.tail.interceptors.hostint.type =
org.apache.flume.interceptor.HostInterceptor$Builder
agent1.sources.tail.interceptors.hostint.preserveExisting = true
agent1.sources.tail.interceptors.hostint.useIP = true
agent1.sinks.HDFS.channel = MemoryChannel-2
agent1.sinks.HDFS.type = hdfs
agent1.sinks.HDFS.hdfs.path = hdfs://url:8020/user/flume/%{host}
agent1.sinks.HDFS.hdfs.file.Type = SequenceFile
agent1.sinks.HDFS.hdfs.writeFormat = Text
agent1.sinks.HDFS.hdfs.rollInterval = 30
agent1.sinks.HDFS.hdfs.rollCount = 0
agent1.sinks.HDFS.hdfs.rollSize = 5000000000
agent1.channels.MemoryChannel-2.type = memory

more examples at http://flume.apache.org/FlumeUserGuide.html



On Tue, Dec 11, 2012 at 12:44 PM, <shouvanik.haldar@accenture.com> wrote:

>  Hi,****
>
> I installed flume 1.x higher version today! But is this very much
> different from the earlier. ? Is there no master / node concept here? How
> do I configure the source and sink?****
>
> ** **
>
> Think I got screwed here!****
>
> ** **
>
> *Thanks and Regards,*
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *
>
> ** **
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Monday, December 10, 2012 3:34 PM
>
> *To:* Haldar, Shouvanik
> *Cc:* user@flume.apache.org
> *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ** **
>
> just a random question, does the user running flume agent have permission
> to read the  event details out of terracotta ? ****
>
> ** **
>
> from the code here
> http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera/flume-core/0.9.4-cdh4.1.0/com/cloudera/flume/watchdog/Watchdog.java
> ****
>
> ** **
>
> line 209 onwards, it looks like the error condition is achieved when you
> have channel set as stdin (I am not sure about this so you may need to look
> at code which part of code you are hitting depending on the flume configs
> you have) ****
>
> ** **
>
> ** **
>
> ** **
>
> On Mon, Dec 10, 2012 at 3:01 PM, <shouvanik.haldar@accenture.com> wrote:**
> **
>
> Hello,****
>
>  ****
>
> I am sure, the problem is due to Terracotta. Can you please help me? ****
>
>  ****
>
> *Thanks and Regards,*****
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *****
>
>  ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Monday, December 10, 2012 2:43 PM
> *To:* Haldar, Shouvanik
> *Cc:* user@flume.apache.org****
>
>
> *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ****
>
> I just hope you are not hitting this
> https://issues.apache.org/jira/browse/FLUME-796****
>
> I would also like to recommend you to upgraded to flume 1.x asap ****
>
>  ****
>
> if possible can you try changing your flume source to some random file
> with tail -f and try sinking it to some file. That will narrow the problem
> is on flume side or terracotta side ****
>
>  ****
>
> Thanks,****
>
> Nitin ****
>
>  ****
>
> On Mon, Dec 10, 2012 at 1:47 PM, <shouvanik.haldar@accenture.com> wrote:**
> **
>
> Hi,****
>
>  ****
>
> I am using Flume -> Flume 0.9.4-cdh3u5. ****
>
>  ****
>
> Initially, the logs are ok. But when I start terracotta
> (./start-tc-server.sh) , then only I experience the above phenomena! Is it
> something to do with terracotta.!****
>
>  ****
>
>  ****
>
> 2012-12-10 03:14:08,806 INFO
> com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking
> thread pool server on port 35853...****
>
> 2012-12-10 03:14:08,894 INFO net.sf.ehcache.terracotta.TerracottaClient:
> Thread [logicalNode ec2-75-101-165-36.compute-1.amazonaws.com-19]
> [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanceFactory
> ****
>
>  ****
>
> Before starting terracotta, Flume Node status is “OPENING”.****
>
> But when I start terracotta, I got the error “INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1”****
>
>  ****
>
> Please help !****
>
>  ****
>
>  ****
>
> *Thanks and Regards,*****
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *****
>
>  ****
>
> *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com]
> *Sent:* Monday, December 10, 2012 12:52 PM
> *To:* user@flume.apache.org
> *Cc:* Haldar, Shouvanik
> *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ****
>
> what version of flume are you using? If it is < 1.x then recommend you to
> upgrade to latest 1.0.3 ****
>
> If you are on 1.x then can you tell us whats the hostname for your server?
> ****
>
>  ****
>
> On Mon, Dec 10, 2012 at 11:40 AM, <shouvanik.haldar@accenture.com> wrote:*
> ***
>
> Before the below mentioned error, I get a success message like****
>
>  ****
>
> 2012-12-10 01:09:14,403 INFO net.sf.ehcache.terracotta.TerracottaClient:
> Thread [logicalNode ec2-75-101-165-36.compute-1.amazonaws.com-19]
> [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanceFactory
> ****
>
>  ****
>
> *Thanks and Regards,*****
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *****
>
>  ****
>
> *From:* Haldar, Shouvanik
> *Sent:* Monday, December 10, 2012 11:34 AM
> *To:* 'user@flume.apache.org'
> *Subject:* Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ****
>
> I had configured flume-master as well as flume-node on a single machine. *
> ***
>
> Now, as I run terracotta -> L1 cache, I get the INFO below in /****
>
>  ****
>
> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog:
> Subprocess exited with value 1****
>
>  ****
>
> Can you please help? I am struggling with this for the past 1 day without
> success!****
>
>  ****
>
> *Thanks and Regards,*****
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *****
>
>  ****
>
>  ****
>  ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise private information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and
> assessment of internal compliance with Accenture policy.
>
>
> ______________________________________________________________________________________
>
> www.accenture.com****
>
>
>
> ****
>
>  ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
>  ****
>
> --
> Nitin Pawar****
>
>
>
> ****
>
> ** **
>
> --
> Nitin Pawar****
>



-- 
Nitin Pawar

Mime
View raw message