flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Ravi <bharathra...@gmail.com>
Subject Re: Running multiple physical nodes on the same machine?
Date Tue, 20 Mar 2012 18:40:22 GMT
Thanks a lot Kowshik!

I just modified the daemon script to allow multiple physical nodes.
For anyone who looks at this in future, I replaced line 85 with the
following:

args=`echo $@ | sed -e "s/[ \t\n]\+//"`
pid=$FLUME_PID_DIR/flume-$FLUME_IDENT_STRING-$command-$args.pid

Thanks!
--
Bharath Ravi



On 20 March 2012 00:02, Kowshik Prakasam <kowshik@gmail.com> wrote:

> Hi Bharath,
>
> One solution is to use the flume binary directly, instead of the daemon
> script which doesn't seem
> to allow multiple physical nodes to start up. Else you could try patching
> the daemon script to
> enalbe handling of multiple flume daemons.
>
> Following are the commands to start-up multiple physical nodes using the
> flume binary:
>
> $ flume node_nowatch -n a
> $ flume node_nowatch -n b
>
>
> Thanks!
> -Kowshik
>
>
> On Mon, Mar 19, 2012 at 8:53 PM, Bharath Ravi <bharathravi1@gmail.com>wrote:
>
>> Hi all,
>>
>> The FAQ here<https://cwiki.apache.org/FLUME/user-faq.html#UserFAQ-CanIruntwoinstancesoftheflumenodeonthesameunixmachine%253F>
says
>> you can start multiple physical nodes on the same machine by simply
>> specifying different names with the "-n" flag.
>>
>> This doesn't work for me.
>>
>> Here's a sample command I tried:
>>
>> # ./bin/flume-daemon.sh start node -n a
>> starting node, logging to ...
>> # ./bin/flume-daemon.sh start node -n b
>> node running as process 24948. Stop it first.
>>
>> Am I missing something? What is the right way to do this?
>>
>> Thanks in advance,
>> --
>> Bharath Ravi
>>
>>
>>
>> On 19 March 2012 23:50, Bharath Ravi <bharathravi1@gmail.com> wrote:
>>
>>> On 19 March 2012 18:04, Jay Stricks <jay@wapolabs.com> wrote:
>>>
>>>> I couldn't get this to work either, so I started writing commands to a
>>>> file and running: flume shell -c {master} -s {file}.
>>>>
>>>>
>>> That's the workaround I've resorted too as well, for now. Is there a
>>> better solution anyone knows of?
>>>
>>>
>>>>
>>>>
>>>> On Mon, Mar 19, 2012 at 4:31 PM, Bharath Ravi <bharathravi1@gmail.com>wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> The user guide<http://archive.cloudera.com/cdh/3/flume/UserGuide/#_using_the_flume_command_shell>
says
>>>>> it is possible to pipe stdin into flume shell.
>>>>> This does not work for me. Manually logging into the shell and running
>>>>> the same commands works perfectly fine.
>>>>> Any pointers as to what might be going wrong? Here's a command I
>>>>> tried with its output:
>>>>>
>>>>>
>>>>> # echo "connect fm-vm-010.cercs.int\nexec unconfig fm-vm-011.cercs.int"
>>>>> | ./bin/flume shell
>>>>> 2012-03-19 16:31:05,768 [main] INFO conf.FlumeConfiguration: Loading
>>>>> configurations from conf
>>>>> ==================================================
>>>>> FlumeShell v0.9.4
>>>>> Copyright (c) Cloudera 2010, All Rights Reserved
>>>>> ==================================================
>>>>> Type a command to execute (hint: many commands
>>>>> only work when you are connected to a master node)
>>>>>
>>>>> You may connect to a master node by typing:
>>>>>     connect host[:adminport=35873[:reportport=45678]]
>>>>>
>>>>> [flume (disconnected)] connect fm-vm-010.cercs.int\nexec unconfig
>>>>> fm-vm-011.cercs.int
>>>>> Failed to run command 'connect fm-vm-010.cercs.int\nexec unconfig
>>>>> fm-vm-011.cercs.int' due to null
>>>>> 2012-03-19 16:31:06,149 [main] ERROR util.FlumeShell: Failed to run
>>>>> command 'connect fm-vm-010.cercs.int\nexec unconfig
>>>>> fm-vm-011.cercs.int'
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message