flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alten-Lorenz <wget.n...@gmail.com>
Subject Re: Flume to stream logs live
Date Thu, 20 Dec 2012 07:02:38 GMT
Hey AK47,

good to know, can you describe for the community what exactly you've done and what you've
archived? Would be great,

cheers,
 Alex 

On Dec 19, 2012, at 5:20 PM, "Kartashov, Andy" <Andy.Kartashov@mpac.ca> wrote:

> Thank you for your suggestions: Mr. Chavez , Mr. Lorenz, Mr. Dechoux and Mr. Connolly,
> 
> I just managed to successfully configure flume to push logs from windows box, running
behind Proxy,  to the Avro-agent, running on Linux box, and without Cygwin.. by just running
JVM process. So happy.
> 
> Cheers,
> AK47
> 
> 
> -----Original Message-----
> From: Alexander Lorenz [mailto:wget.null@gmail.com]
> Sent: Monday, December 17, 2012 2:58 AM
> To: user@flume.apache.org
> Subject: Re: Flume to stream logs live
> 
> Hey,
> 
> fluentd is a good tip, they have a custom sink:
> https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Plugins
> 
> This could work.
> 
> Best,
> Alex
> 
> On Dec 17, 2012, at 2:33 AM, Juhani Connolly <juhani_connolly@cyberagent.co.jp>
wrote:
> 
>> If you can't get flume running on windows your alternatives look like syslog, scribed
and fluentd. We used scribed before moving to flume, but tbh, it can be a pain to compile
and the hdfs support on it is not the best, we had data loss if there was an hdfs restart
etc. I can't say much about the other two
>> 
>> On 12/14/2012 11:47 PM, Kartashov, Andy wrote:
>>> 
>>> Flummers,
>>> 
>>> Loved working with Flume 1.2 -- very easy and simple configuration, it was a
pleasure to work with. Managed to "tail --F" logs from unix server and into a hdfs cluster.
The problem started when I also needed to push logs from a Windows application server.  Spent
three days researching on how to install flume on Windows and run  a deamon/agent that will
push the logs to the Avro source I successfully configured and ran on Unix. No luck. So I
am looking t alternative. Is there other framework available out there to help me with my
issue. What about scribe?
>>> 
> 
>>> 
> 
>>> 
>>> NOTICE: This e-mail message and any attachments are confidential, subject to
copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited.
If you are not the intended recipient, please delete and contact the sender immediately. Please
consider the environment before printing this e-mail. AVIS : le présent courriel et toute
pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent
être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée
est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et
contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer
le présent courriel
>> 
> 
> NOTICE: This e-mail message and any attachments are confidential, subject to copyright
and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are
not the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe
qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite.
Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel


Mime
View raw message