ode-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lavanya Ramakrishnan <laram...@cs.indiana.edu>
Subject Re: Problem with multiple concurrent clients
Date Sat, 01 Dec 2007 19:48:17 GMT
This is the error I see in the logs:


ERROR - GeronimoLog.error(104) | Timeout or execution error when waiting
for response to MEX {MyRoleMex#hqejbhcnphr2sems2kt6p1 [Client
hqejbhcnphr2sems2kt6p0] calling {http://ode/bpel/unit-test.w\
sdl}HelloService.hello(...)} java.util.concurrent.TimeoutException:
Message exchange
org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture@1ec697e
timed out when waiting for a response!



On Fri, 30 Nov 2007, Matthieu Riou wrote:

> On Nov 30, 2007 5:10 PM, Lavanya Ramakrishnan <laramakr@cs.indiana.edu>
> wrote:
>
> > Thanks! This is looking really good now!
> >
> > When I have a large number of connections (>75) I am beginning to see some
> > TimeoutExceptions from the reply response. Is there a parameter I can
> > tweak for the timeout value?
> >
>
> Do you know exactly what times out or have a stack trace? Could just be your
> client connection.
>
> Cheers,
> Matthieu
>
>
> >
> > thanks again!
> > Lavanya
> >
> > On Fri, 30 Nov 2007, Matthieu Riou wrote:
> >
> > > Hi,
> > >
> > > Sorry for taking so long but I've been side-tracked a couple of times
> > and
> > > this wasn't trivial to debug either. So I've found a couple of problems:
> > >
> > > 1. A potential deadlock (that was happening once in a while) in our JPA
> > > Process DAO. Basically each new instance creation resulted in an update
> > on
> > > the very same process record to increase some counter, which was pretty
> > bad.
> > >
> > > 2. A much more tricky bug with OpenJPA committing the transaction once
> > in a
> > > while without us asking for it. This appeared to be somewhat linked to
> > the
> > > Geronimo transaction manager as I've only seen that happening with it.
> > So
> > > once a while, when OpenJPA was getting a new set of sequence numbers to
> > > assign to primary keys, it forced a commit in the middle of our
> > transaction.
> > > Hence errors about not having a transaction anymore afterward. I've
> > upgraded
> > > to OpenJPA 1.0.1 and it has been fixed, so it seems to work properly for
> > me
> > > know.
> > >
> > > I've done some tests on Tomcat/MySQL with 30 threads and it worked
> > properly.
> > > I think there's a lot we could do to improve performances by optimizing
> > the
> > > queries but I don't have much time for that right now. But at still it
> > works
> > > and it's not ridiculously slow with enough threads. So update your
> > version
> > > of the 1.1 branch and you should see an improvement.
> > >
> > > Cheers,
> > > Matthieu
> > >
> > > On Nov 28, 2007 5:28 PM, Matthieu Riou <matthieu@offthelip.org> wrote:
> > >
> > > > Okay I can reproduce the problem, it's on Derby but I think in that
> > case
> > > > it doesn't make much difference. Seems to be a big database
> > deadlock... I'll
> > > > keep you posted.
> > > >
> > > > Cheers,
> > > > Matthieu
> > > >
> > > >
> > > > On Nov 28, 2007 3:51 PM, Lavanya Ramakrishnan <laramakr@cs.indiana.edu
> > >
> > > > wrote:
> > > >
> > > > > Just to clarify, I am only trying to run 5 parallel clients but they
> > do
> > > > > make repeated helloWorld calls.
> > > > >
> > > > > I added these properties. But I am still getting the NPE. but in
> > > > > addition I am seeing another exception which might be a clue?
> > > > >
> > > > > The entire log is at
> > http://www.cs.indiana.edu/~laramakr/catalina.out.1<http://www.cs.indiana.edu/%7Elaramakr/catalina.out.1>
> > <http://www.cs.indiana.edu/%7Elaramakr/catalina.out.1>
> > > > >
> > > > >
> > > > > 15:37:13,297 DEBUG [ExecutionQueueImpl] >> dequeueReaction()
> > > > > ERROR - GeronimoLog.error(108) | Method "run" in class
> > > > > "org.apache.ode.bpel.runtime.REPLY" threw an unexpected exception.
> > > > > org.apache.ode.bpel.iapi.ContextException: Unable to register
> > > > > synchronizer.
> > > > >        at
> > > > > org.apache.ode.scheduler.simple.SimpleScheduler.registerSynchronizer
> > (
> > > > > SimpleScheduler.java:206)
> > > > >        at
> > > > >
> > org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl.responseReceived(
> > > > > MyRoleMessageExchangeImpl.java :215)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.MessageExchangeImpl.setResponse(
> > > > > MessageExchangeImpl.java:173)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.BpelRuntimeContextImpl.reply(
> > > > > BpelRuntimeContextImpl.java:544)
> > > > >        at org.apache.ode.bpel.runtime.REPLY.run(REPLY.java:65)
> > > > >        at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown
> > Source)
> > > > >        at
> > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > > > > DelegatingMethodAccessorImpl.java :25)
> > > > >        at java.lang.reflect.Method.invoke(Method.java:585)
> > > > >        at
> > > > > org.apache.ode.jacob.vpu.JacobVPU$JacobThreadImpl.run(JacobVPU.java
> > :451)
> > > > >        at org.apache.ode.jacob.vpu.JacobVPU.execute(JacobVPU.java:139)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.BpelRuntimeContextImpl.execute(
> > > > > BpelRuntimeContextImpl.java:838)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.PartnerLinkMyRoleImpl.invokeNewInstance(
> > > > > PartnerLinkMyRoleImpl.java :197)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.BpelProcess.invokeProcess(
> > BpelProcess.java
> > > > > :167)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.BpelProcess.handleWorkEvent(
> > BpelProcess.java
> > > > > :335)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.BpelEngineImpl.onScheduledJob (
> > > > > BpelEngineImpl.java:328)
> > > > >        at
> > > > > org.apache.ode.bpel.engine.BpelServerImpl.onScheduledJob(
> > > > > BpelServerImpl.java:365)
> > > > >        at
> > > > > org.apache.ode.scheduler.simple.SimpleScheduler$4$1.call(
> > > > > SimpleScheduler.java:341)
> > > > >        at
> > > > > org.apache.ode.scheduler.simple.SimpleScheduler$4$1.call(
> > > > > SimpleScheduler.java:340)
> > > > >        at
> > > > > org.apache.ode.scheduler.simple.SimpleScheduler.execTransaction(
> > > > > SimpleScheduler.java:179)
> > > > >        at
> > > > > org.apache.ode.scheduler.simple.SimpleScheduler$4.call(
> > > > > SimpleScheduler.java:339)
> > > > >        at
> > > > > org.apache.ode.scheduler.simple.SimpleScheduler$4.call(
> > > > > SimpleScheduler.java:336)
> > > > >        at
> > > > > java.util.concurrent.FutureTask$Sync.innerRun (FutureTask.java:269)
> > > > >        at java.util.concurrent.FutureTask.run(FutureTask.java:123)
> > > > >        at
> > > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(
> > > > > ThreadPoolExecutor.java:650)
> > > > >        at
> > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run (
> > > > > ThreadPoolExecutor.java:675)
> > > > >        at java.lang.Thread.run(Thread.java:595)
> > > > > Caused by: java.lang.NullPointerException
> > > > >        at
> > > > > org.apache.ode.scheduler.simple.SimpleScheduler.registerSynchronizer(
> > > > > SimpleScheduler.java:194)
> > > > >        ... 25 more
> > > > >
> > > > >
> > > > >
> > > > > On Wed, 28 Nov 2007, Matthieu Riou wrote:
> > > > >
> > > > > > In internal mode you should adjust the pool size with something
> > like:
> > > > > >
> > > > > > ode-axis2.db.pool.max=100
> > > > > > ode-axis2.db.pool.min=40
> > > > > >
> > > > > > With 100 threads and the default pool size (10) you're *very*
> > probably
> > > > > > exhausting the number of connections.
> > > > > >
> > > > > > Cheers,
> > > > > > Matthieu
> > > > > >
> > > > > > On Nov 28, 2007 12:04 PM, Lavanya Ramakrishnan <
> > > > > laramakr@cs.indiana.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > > Lavanya, are you sure you configured a large enough
connection
> > > > > pool to
> > > > > > > > handle all your requests? If you start a large number
of
> > parrallel
> > > > > > > requests
> > > > > > > > they'll all try to get a DB connection and if there's
not
> > enough
> > > > > of them
> > > > > > > > available, things run bad. Let me know.
> > > > > > >
> > > > > > > I also suspected a pool exhaustion. So based on earlier
posts I
> > am
> > > > > using
> > > > > > > db.mode to be INTERNAL. How do I increase the db connection
pool
> > in
> > > > > this
> > > > > > > mode? I looked in the FAQ etc but didn't find a mention.
> > > > > > >
> > > > > > > Here is what I have so far in my ode-axis2.properites.
> > > > > > >
> > > > > > > ode-axis2.db.mode=INTERNAL
> > > > > > > ode-axis2.db.int.jdbcurl=***
> > > > > > > ode-axis2.db.int.driver=com.mysql.jdbc.Driver
> > > > > > > ode-axis2.threads.pool.size=200
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
>

Mime
View raw message