ode-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdulkadir yaman <abdulkadirya...@gmail.com>
Subject Re: org/apache/ode/scheduler/simple/SimpleScheduler.java_processedSinceLastLoadTask concurrent map keeps growing under load test
Date Mon, 10 Oct 2011 18:16:38 GMT
Hi Tammo,

I worked around this issue by wrapping log4j dependency as a bundle in
servicemix. More elegant solution would be to add a bundle entry in
jbi-karaf/src/main/resources/features.xml<http://repo1.maven.org/maven2/org/apache/ode/ode-jbi-karaf/1.3.5/ode-jbi-karaf-1.3.5-features.xml>
as
follows;

<features>
<feature name="ode-core-deps" version="1.3.5">
<feature>transaction</feature>
*<bundle>wrap:mvn:log4j/log4j/1.2.13</bundle>*
<bundle>mvn:commons-collections/commons-collections/3.2.1</bundle>
<bundle>mvn:commons-beanutils/commons-beanutils/1.8.2</bundle>
<bundle>mvn:commons-io/commons-io/1.4</bundle>
<bundle>wrap:mvn:jaxen/jaxen/1.1.1</bundle>
<bundle>
mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.saxon/9.1.0.8_1
</bundle>
<bundle>
mvn:org.apache.geronimo.components/geronimo-connector/2.1.3
</bundle>
</feature>

Currently servicemix acquires
http://repo1.maven.org/maven2/org/apache/ode/ode-jbi-karaf/1.3.5/ode-jbi-karaf-1.3.5-features.xml
 to install necessary dependencies and core components(correct me if i am
wrong). This also has to be changed if we want servicemix 4.3.0 version to
work correctly. However i am not sure if  changing existing/already uploaded
files in repo1.maven is elegant or not.

Thanks,
Abdulkadir

On Mon, Oct 10, 2011 at 8:35 PM, Tammo van Lessen <tvanlessen@gmail.com>wrote:

> Wow, good catch! Thanks for reporting the issue!
>
> If you know how this can easily be fixed (e.g. which exact changes
> should be done to the manifest), such instructions would be highly
> appreciated. This would help us a lot as we don't work with ServiceMix
> day by day.
>
> Thanks,
>  Tammo
>
> On 10.10.2011 19:32, abdulkadir yaman wrote:
> > jira ticket created : https://issues.apache.org/jira/browse/ODE-943
> >
> >
> >
> > On Mon, Oct 10, 2011 at 10:58 AM, abdulkadir yaman <
> > abdulkadiryaman@gmail.com> wrote:
> >
> >> Hi again,
> >>
> >> In a more detailed investigation, I attached jdb to servicemix and see
> the
> >> path executed line by line. i put breakpoints to each line in boolean
> >> doLoadImmediate() block. i saw that code works without exception till
> >> the debug("LOAD IMMEDIATE complete"); line:733 in finally block. Strange
> it
> >> is that everything works great but breakpoint at
> >> _processedSinceLastLoadTask.clear(); line 727 (
> >>
> http://svn.apache.org/viewvc/ode/trunk/scheduler-simple/src/main/java/org/apache/ode/scheduler/simple/SimpleScheduler.java?revision=947456&view=markup
> )
> >> is not reached although last line hit :707 which is
> AbsoluteTimeDateFormat f = new AbsoluteTimeDateFormat();
> >>
> >> I`ve experimented with  single requests and also with JMeter (~300tps).
> The
> >> code does not enter the the catch block. And the size of
> >> concurrentHashMap _processedSinceLastLoadTask grows higher and higher
> since
> >> clear method is never executed.
> >>
> >> I do appreciate any help.
> >>
> >>
> >>
> >>
> >> Hi,
> >>
> >> I deployed ode 1.3.5 in servicemix 4.3.0 on Ubuntu 11.04 64bit with
> maxHeap
> >>
> >> size = 1024m . I have set necessary coreThreadPool sizes to 16,
> >>
> >> maxThreadPool sizes to unlimited. I generated a 300-400 tps load on a
> flow
> >>
> >> which is in-memory enabled. After a while, short gc cycles could not
> achieve
> >>
> >> to collect necessary objects and fullgc cycles try to collect with
> pausing
> >>
> >> system 2,5 - 4.0 seconds which causes system to freeze and not handle
> >>
> >> further requests. At the same time i took a heap snapshot and tried to
> >>
> >> analyse it. As a result, i saw the biggest object was SimpleScheduler
> >>
> >> instance containing ConcurrentHashMap<String,Long>
> >>
> >> _processedSinceLastLoadTask instance (almost 1gb) which contains
> >>
> >> jobId(String) type as keys and scheduling dates(Long) as values.
> >>
> >> I have researched a bit on SimpleScheduler trunk version but seen that
> 1.3.5
> >>
> >> version installed in Servicemix 4.3.0 contains an older version compared
> to
> >>
> >> trunk. I decompiled the class in servicemix version and investigated
> that
> >>
> >> either _processedSinceLastLoadTask.clear() does not work or execution
> does
> >>
> >> not endure till the line  _processedSinceLastLoadTask.clear() inside
> >>
> >>  boolean doLoadImmediate() method block.
> >>
> >> I have tried several configuration parameters such as `process-events
> >>
> >> generate="none"` `cleanup on="always"` in deploy.xml
> >>
> >> or ode-jbi.threads.pool.size=100 in ode config file. None of them have
> >>
> >> worked out.
> >>
> >> Any idea about what may cause _processedSinceLastLoadTask map to grow?
> >>
> >> Thanks.
> >>
> >>
> >
>
> --
> Tammo van Lessen - http://www.taval.de
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message