flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Naik <ros...@hortonworks.com>
Subject Re: Flume on windows
Date Thu, 04 Dec 2014 21:04:01 GMT
This topic keeps coming up every now and then.  One way to get it working
on windows is just try to cross your fingers and run the plain vanilla
apache distro ...  your mileage will vary significantly depending on what
components and features you use.

for eg. spool dir with the default deserializer wont work correctly due to
improper handling of new lines, dual checkpointing needs to be disabled on
windows, etc.

There are actually several windows patches currently waiting to be
committed for windows support. If you search for all Jiras with "windows"
in the title you should be able to find them. You can either apply the ones
relevant to your needs and create a custom build or apply them all.

One of the patches includes a startup script for windows. IMO that should
be useful as it can alleviate inconveniences related to establishing the
right environment for flume.

If you prefer to not build it yourself, you could consider using some
pre-packaged distros like HDP that have Windows support.




On Thu, Dec 4, 2014 at 5:47 AM, <j.guilmard@accenture.com> wrote:

>  Hi,
>
>
>
> I do not use the File options with Flume on Windows: only memory channel
> and external sources. Nevertheless, might be worth trying a tail
> implementation for Windows (like
> http://gnuwin32.sourceforge.net/packages/coreutils.htm ), and use the
> Exec Source ?
>
>
>
> *Jeff*
>
>
>
> *From:* Mahadevappa, Shobha [mailto:Shobha.Mahadevappa@nttdata.com]
> *Sent:* jeudi 4 décembre 2014 11:40
> *To:* user@flume.apache.org
> *Subject:* RE: Flume on windows
>
>
>
> Thanks all for the inputs.
>
> I tried the latest flume version on windows and was successful in starting
> up the collector and agents.
>
> However, one of the agent source tails a file which fails on windows.
>
> Any inputs how this can be achieved?
>
>
>
> Regards,
>
> Shobha
>
>
>
> *From:* Hari Shreedharan [mailto:hshreedharan@cloudera.com
> <hshreedharan@cloudera.com>]
> *Sent:* Thursday, December 04, 2014 5:49 AM
> *To:* user@flume.apache.org
> *Cc:* user@flume.apache.org
> *Subject:* Re: Flume on windows
>
>
>
> Theoretically, it might work - but there are several flume components that
> use fsync etc  which won’t work on Windows. So any guarantees provided by
> things like the File Channel are voided. That said, I have never tested it
> on Windows - so I can’t promise anything will work.
>
>
> Thanks,
> Hari
>
>
>
> On Wed, Dec 3, 2014 at 2:40 AM, Mahadevappa, Shobha <
> Shobha.Mahadevappa@nttdata.com> wrote:
>
>  Hi,
>
> Can anyone please let me know if flume ng is supported on windows?
>
> If yes, please provide pointers to the same.
>
>
>
>
>
> Regards,
>
> *Shobha M *
>
>
>
>
> ______________________________________________________________________
> Disclaimer: This email and any attachments are sent in strictest confidence
> for the sole use of the addressee and may contain legally privileged,
> confidential, and proprietary data. If you are not the intended recipient,
> please advise the sender by replying promptly to this email and then delete
> and destroy this email and any attachments without any further use, copying
> or forwarding.
>
>
>
>
> ______________________________________________________________________
> Disclaimer: This email and any attachments are sent in strictest confidence
> for the sole use of the addressee and may contain legally privileged,
> confidential, and proprietary data. If you are not the intended recipient,
> please advise the sender by replying promptly to this email and then delete
> and destroy this email and any attachments without any further use, copying
> or forwarding.
>
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
>
> ______________________________________________________________________________________
>
> www.accenture.com
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message