flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Junxian Yan <junxian....@gmail.com>
Subject Re: can not get config after launch master process
Date Fri, 29 Jul 2011 01:54:31 GMT
Seems not like this case. Here is part of my configure file

  connect MASTER:35873
  exec map MASTER collector1
  exec map MASTER agent1

  exec config collector1 logicalSource
'collectorSink("s3n://%{tailSrcFile}/%Y-%m-%d/","%{tailSrcFile}-%Y%m%d.")'
  exec config agent1
'tailDir("/var/apps/log/",fileregex="*.log$",startFromEnd=true)'
'logicalSink("collector1")'



   exec map MASTER collector2
   exec map MACHINE1 agent2

   exec config collector2 logicalSource
'collectorSink("s3n://%{tailSrcFile}/%Y-%m-%d/","%{tailSrcFile}-%Y%m%d.")'
   exec config agent2
'tailDir("/var/apps/log/",fileregex="*.log$",startFromEnd=true)'
'logicalSink("collector2")'


R

On Fri, Jul 29, 2011 at 9:05 AM, Dan Everton <dan@iocaine.org> wrote:

> Are your configuration scripts calling "exec refreshAll"? We've found
> that using that command can cause a deadlock in the master process. See
> this message for an example stack trace:
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-flume-user/201107.mbox/%3C1311734821.16079.2156289349@webmail.messagingengine.com%3E
>
> On Thu, 28 Jul 2011 16:49 -0700, "Junxian Yan" <junxian.yan@gmail.com>
> wrote:
> > Hi Guys
> >
> > I encountered a weird problem. When I start a master process and a couple
> > of
> > node process, run the configure script on each node and master machine, I
> > tried  to access the MASTER:35871. I found I can not get the
> > configuration.
> >  Seems the system hang. I also tried it in flume shell. The same problem.
> > After I send the getconfigs command, no return and the command didn't
> > complete, no error message. When I restart master process, it's ok.
> >
> > I wanna know if anyone else encountered the same issue with me. Is there
> > any
> > investigation for this issue. I'm sorry I can not provide flume log, I
> > can
> > not reproduce it in short period.
> >
> > My flume environment is:   there're 10 collectors and master in the same
> > machine. And there are another 5 machines running flume node. Every time,
> > the flume node will connect to master, submit the configuration.
> >
> > R
> >
>

Mime
View raw message