Sorry, I found in the flume0.9.4 tray icon menu I could edit the parameter, and edit registry is also ok.
I add two parameter:
-XX:MaxDirectMemorySize=500m
-XX:MaxNewSize=500m
and change JvmMs to 500. It did not report out of memory again.

Thank you, Jeong-shik!
Andy

2013/1/28 Jeong-shik Jang <jsjang@gruter.com>
How about editing registry using regedit?

1. locate memory option parameters in registry:
HKEY_LOCAL_MACHINE > SOFTWARE > Apache Software Foundation > Procrun 2.0 > FlumeNode > Parameters > Java

2. Increase values in JvmMs and JvmMx

3. Stop/Start FlumeNode Service via Flume tray icon menu.

-JS


On 1/28/13 1:05 PM, 周梦想 wrote:
Thank you, Jeong-shik!
I also consider that the OPENNING status related to ack rolling. Because it happened on Windows, not on Linux although I closed the firewall of windows. But I don't know why, maybe there are some anti-virus software do some thing to stop ack.

The flume node tray icon can't modify the parameter of java VM. It's read only.

Best regards!
Andy

2013/1/25 Jeong-shik Jang <jsjang@gruter.com>
Hi Andy,

Seeing your log, out of memory error had occurred during opening stage of the logical node, and so was the last status of the logical node reported to master; no more status was reported to master due to the error since then, so it stays as OPENNING in master.
Your error seems to be related to ack rolling because agentSink is agentE2ESink; agentDFO is not related to ack but still WAL for disk failover, though.

I remember you can change java memory option in flume procrun tray (flumenodew.exe) icon menu; one looks just like Apache web server tray icon.
You need to restart(stop and start) service through the tray menu once you make the change.

-JS


On 1/25/13 5:34 PM, 周梦想 wrote:
I found that if I change windows agent configure from
tail("c:\\test.txt") agentSink("hadoop48",35853) ;
to
tail("c:\\test.txt") agentDFOSink("hadoop48",35853) ;

then the agent changes status from OPENING to ACTIVE.
my windows agent is :192.168.20.81, while linux collector is : 192.168.10.48
another linux agent is 192.168.10.46. 10.46 have no problem of agentSink.

Thanks!
Andy.

2013/1/25 周梦想 <ablozhou@gmail.com>
when i start the flume node service,the agent is OPENING.
Hello,

I need to gather logs from windows 7, but the flume master and collector is running on linux. I download 0.9.4 exe file from flume official web.

When I start the node from windows, the status is OPENING.

zhouhh.TKOffice.local zhouhh.TKOffice.local zhouhh.TKOffice.local OPENING Fri Jan 25 11:45:01 CST 2013 4

And the log reports OutOfMemoryError.

================
D:\Program Files\Cloudera\Flume 0.9.4\log\flumenode-stdout.2013-01-25.log

2013-01-25 15:31:03 Commons Daemon procrun stdout initializedListening for transport dt_socket at address: 8888
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid8100.hprof ...
Heap dump file created [13270531 bytes in 0.115 secs]

===============
D:\Program Files\Cloudera\Flume 0.9.4\log\flumenode-stderr.2013-01-25.log

2013-01-25 15:31:03 Commons Daemon procrun stderr initialized
13/01/25 15:31:03 INFO agent.FlumeNode: Flume 0.9.4
13/01/25 15:31:03 INFO agent.FlumeNode:  rev 8d0aa4be4ee50bb664cb0d1624e5634f46d7b62a
13/01/25 15:31:03 INFO agent.FlumeNode: Compiled  on  Mon Jun 20 14:46:53 PDT 2011
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.runtime.name=Java(TM) SE Runtime Environment
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.boot.library.path=C:\PROGRA~1\Java\jre7\bin
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vm.version=23.3-b01
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vm.vendor=Oracle Corporation
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vendor.url=http://java.oracle.com/
13/01/25 15:31:03 INFO agent.FlumeNode: System property path.separator=;
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vm.name=Java HotSpot(TM) Client VM
13/01/25 15:31:03 INFO agent.FlumeNode: System property file.encoding.pkg=sun.io
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.country=CN
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.script=
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.os.patch.level=Service Pack 1
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vm.specification.name=Java Virtual Machine Specification
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.dir=C:\Windows\system32
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.runtime.version=1.7.0_07-b11
13/01/25 15:31:03 INFO agent.FlumeNode: System property flume=
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.endorsed.dirs=C:\PROGRA~1\Java\jre7\lib\endorsed
13/01/25 15:31:03 INFO agent.FlumeNode: System property os.arch=x86
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.io.tmpdir=C:\Windows\TEMP\
13/01/25 15:31:03 INFO agent.FlumeNode: System property line.separator=

13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vm.specification.vendor=Oracle Corporation
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.variant=
13/01/25 15:31:03 INFO agent.FlumeNode: System property os.name=Windows 7
13/01/25 15:31:03 INFO agent.FlumeNode: System property flume.home=d:\PROGRA~1\Cloudera\FLUME0~1.4\bin\..
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.jnu.encoding=GBK
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.library.path=d:\PROGRA~1\Cloudera\FLUME0~1.4\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Program Files\Common Files\NetSarang;C:\Python27\;C:\Python27\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files\Common Files\Thunder Network\KanKan\Codecs;d:\php;d:\php\ext;D:\Program Files\MySQL\MySQL Server 5.5\bin;D:\mysql\bin;D:\wxWidgets-2.9.1\include;D:\wxWidgets-2.9.1\include\msvc;W:\Server\include;W:\Base\include;W:\Client\include;d:\Program Files\Lua\5.1;d:\Program Files\Lua\5.1\clibs;D:\Program Files\TortoiseSVN\bin;d:\Program Files\Git\cmd;d:\Program Files\Git\bin;D:\Program Files\TortoiseGit\bin;D:\docbook\bin;;.
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.specification.name=Java Platform API Specification
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.class.version=51.0
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.management.compiler=HotSpot Client Compiler
13/01/25 15:31:03 INFO agent.FlumeNode: System property os.version=6.1
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.home=C:\
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.timezone=Asia/Shanghai
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.awt.printerjob=sun.awt.windows.WPrinterJob
13/01/25 15:31:03 INFO agent.FlumeNode: System property file.encoding=GBK
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.specification.version=1.7
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.class.path=d:\PROGRA~1\Cloudera\FLUME0~1.4\bin\..\build\classes;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\activation-1.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\ahocorasick-2.x.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\ant-1.6.5.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\antlr-2.7.7.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\antlr-3.2.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\antlr-runtime-3.2.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\asm-3.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\automaton-1.0.0.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\avro-1.5.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\avro-compiler-1.5.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\avro-ipc-1.5.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\commons-cli-1.2.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\commons-codec-1.3.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\commons-collections-3.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\commons-el-1.0.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\commons-httpclient-3.1.jar;d:\PROGRA~1\Cl o udera\FLUME0~1.4\lib\common s -lang-2.5.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\commons-logging-1.0.4.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\commons-net-1.4.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\core-3.1.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\flume-core-0.9.4.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\guava-r07.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\hadoop-core-0.20.2-CDH3B4.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\hsqldb-1.8.0.7.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\irclib-1.0.0.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jackson-core-asl-1.5.5.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jackson-jaxrs-1.5.5.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jackson-mapper-asl-1.5.5.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jackson-xc-1.5.5.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jasper-compiler-5.5.23.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jasper-runtime-5.5.23.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jaxb-api-2.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jaxb-impl-2.1.12.jar;d:\PROGRA~1\Cloudera\FLU M E0~1.4\lib\jersey-core-1.4. j ar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jersey-json-1.4.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jersey-server-1.4.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jets3t-0.6.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jettison-1.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jetty-6.1.26.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jetty-util-6.1.26.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jline-0.9.94.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jms-1.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jsp-api-2.0.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\jsp-api-2.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\libthrift-0.6.0-cdh.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\log4j-1.2.15.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\mail-1.4.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\netty-3.2.4.Final.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\oro-2.0.8.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\paranamer-2.3.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\servlet-api-2.5-20081211.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\servle t -api-2.5.jar;d:\PROGRA~1\Cl o udera\FLUME0~1.4\lib\slf4j-api-1.5.11.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\slf4j-log4j12-1.5.11.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\snappy-java-1.0.1-rc3.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\stax-api-1.0-2.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\stax-api-1.0.1.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\stringtemplate-3.2.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\velocity-1.6.4.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\xmlenc-0.52.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\lib\zookeeper-3.3.1-10.jar;d:\PROGRA~1\Cloudera\FLUME0~1.4\bin\..\conf
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.name=ZHOUHH$
13/01/25 15:31:03 INFO agent.FlumeNode: System property flume.root.logger=INFO,console
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vm.specification.version=1.7
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.home=C:\PROGRA~1\Java\jre7
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.arch.data.model=32
13/01/25 15:31:03 INFO agent.FlumeNode: System property user.language=zh
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.specification.vendor=Oracle Corporation
13/01/25 15:31:03 INFO agent.FlumeNode: System property awt.toolkit=sun.awt.windows.WToolkit
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vm.info=mixed mode
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.version=1.7.0_07
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.ext.dirs=C:\PROGRA~1\Java\jre7\lib\ext;C:\Windows\Sun\Java\lib\ext
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.boot.class.path=C:\PROGRA~1\Java\jre7\lib\alt-rt.jar;C:\PROGRA~1\Java\jre7\lib\resources.jar;C:\PROGRA~1\Java\jre7\lib\rt.jar;C:\PROGRA~1\Java\jre7\lib\sunrsasign.jar;C:\PROGRA~1\Java\jre7\lib\jsse.jar;C:\PROGRA~1\Java\jre7\lib\jce.jar;C:\PROGRA~1\Java\jre7\lib\charsets.jar;C:\PROGRA~1\Java\jre7\lib\jfr.jar;C:\PROGRA~1\Java\jre7\classes
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vendor=Oracle Corporation
13/01/25 15:31:03 INFO agent.FlumeNode: System property file.separator=\
13/01/25 15:31:03 INFO agent.FlumeNode: System property java.vendor.url.bug=http://bugreport.sun.com/bugreport/
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.io.unicode.encoding=UnicodeLittle
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.cpu.endian=little
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.desktop=windows
13/01/25 15:31:03 INFO agent.FlumeNode: System property sun.cpu.isalist=pentium_pro+mmx pentium_pro pentium+mmx pentium i486 i386 i86
13/01/25 15:31:03 INFO agent.FlumeNode: Starting flume agent on: zhouhh.TKOffice.local
13/01/25 15:31:03 INFO agent.FlumeNode:  Working directory is: C:\Windows\system32\.
13/01/25 15:31:03 INFO conf.FlumeConfiguration: Loading configurations from d:/PROGRA~1/Cloudera/FLUME0~1.4/conf
13/01/25 15:31:03 WARN text.FormatFactory: Unable to load output format plugin class  - Class not found
13/01/25 15:31:03 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/01/25 15:31:03 INFO util.InternalHttpServer: Starting internal HTTP server
13/01/25 15:31:03 INFO mortbay.log: jetty-6.1.26
13/01/25 15:31:03 INFO mortbay.log: Extract d:\PROGRA~1\Cloudera\FLUME0~1.4\bin\..\webapps\flumeagent.war to C:\Windows\TEMP\Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3\webapp
13/01/25 15:31:03 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:35862
13/01/25 15:31:03 INFO util.InternalHttpServer: Server started
13/01/25 15:31:03 INFO agent.MultiMasterRPC: No active master RPC connection
13/01/25 15:31:04 INFO agent.ThriftMasterRPC: Connected to master at hadoop48:35872
13/01/25 15:31:04 INFO agent.LogicalNodeManager: creating new logical node zhouhh.TKOffice.local
13/01/25 15:31:04 INFO agent.FlumeNode: Hadoop Security enabled: false
13/01/25 15:31:04 INFO util.InternalHttpServer: Starting internal HTTP server
13/01/25 15:31:04 INFO mortbay.log: jetty-6.1.26
13/01/25 15:31:04 INFO mortbay.log: Extract d:\PROGRA~1\Cloudera\FLUME0~1.4\bin\..\webapps\flumeagent.war to C:\Windows\TEMP\Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3\webapp
13/01/25 15:31:04 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:35862
13/01/25 15:31:04 INFO util.InternalHttpServer: Server started
13/01/25 15:31:09 INFO agent.LogicalNode: Node config successfully set to com.cloudera.flume.conf.FlumeConfigData@231214
13/01/25 15:31:09 INFO durability.NaiveFileWALManager: NaiveFileWALManager is now open
13/01/25 15:31:09 INFO durability.NaiveFileWALManager: Attempting to recover c:\tmp\flume-ZHOUHH$\agent\zhouhh.TKOffice.local\sending / 20130124-103034656+0800.2144885518902.00000028
13/01/25 15:31:09 INFO debug.MemorySinkSource: Opening mem source sink that has with 0 elements
13/01/25 15:31:09 INFO endtoend.AckChecksumChecker: Starting checksum group called zhouhh.TKOffice.local2151099224995
13/01/25 15:31:09 INFO endtoend.AckChecksumChecker: initial checksum is 13c6a648fe8
13/01/25 15:31:09 INFO endtoend.AckChecksumChecker: Finishing checksum group called 'zhouhh.TKOffice.local2151099224995'
13/01/25 15:31:09 INFO endtoend.AckChecksumChecker: Checksum succeeded 13c6a648fe8
13/01/25 15:31:09 INFO endtoend.AckChecksumChecker: moved from partial to complete zhouhh.TKOffice.local2151099224995
Exception in thread "logicalNode zhouhh.TKOffice.local-25" java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:59)
at org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1945)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1845)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1891)
at com.cloudera.flume.handlers.hdfs.SeqfileEventSource.next(SeqfileEventSource.java:62)
at com.cloudera.flume.core.EventUtil.dumpAll(EventUtil.java:47)
at com.cloudera.flume.agent.durability.NaiveFileWALManager.checkAndStripAckFraming(NaiveFileWALManager.java:378)
at com.cloudera.flume.agent.durability.NaiveFileWALManager.recoverLog(NaiveFileWALManager.java:288)
at com.cloudera.flume.agent.durability.NaiveFileWALManager.recover(NaiveFileWALManager.java:411)
at com.cloudera.flume.agent.durability.NaiveFileWALDeco.open(NaiveFileWALDeco.java:240)
at com.cloudera.flume.agent.AgentSink.open(AgentSink.java:150)
at com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:88)


I have 4GB memory, and the source file to tail only 1KB, What can I do to remove this error?

Thanks!
Andy




-- 
Jeong-shik Jang / jsjang@gruter.com
Gruter, Inc., R&D Team Leader
www.gruter.com
Enjoy Connecting



-- 
Jeong-shik Jang / jsjang@gruter.com
Gruter, Inc., R&D Team Leader
www.gruter.com
Enjoy Connecting