ode-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Falko Menge <m...@falko-menge.de>
Subject ODE crashes when requests arrive simultaneously
Date Tue, 29 Jan 2008 16:03:57 GMT
Hello everyone,

I'm currently working on a research project trying to improve scalability of
workflow engines by distributing processes across multiple systems. For the
empirical validation of the approach I wanted to use ODE deployed in ServiceMix.
Unfortunately, during the first stress tests with ODE it turned out, that
requests arriving nearly simultaneously seem to cause deadlocks of the database
transactions.

soupUI's load testing feature was used to produce the requests. In ODE 1.0 and
1.1 it was enough to use only two threads each sending a single request to
immediately produce errors like this:

ERROR 40XL1: A lock could not be obtained within the time requested
   at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
   ...
After that, the database seems to be in an inconsistent state, which leads to a
complete denial of service.

I made tests with all examples in the JBI release as well as own processes,
using different ServiceMix and ODE Versions under Windows and Linux.
Furthermore I tried MySQL, which also produced similar problems:

connectionErrorOccurred called with null
java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2985)
	...

In ODE 1.1.1 it seems to have improved a bit. But if the number of threads is
increased, the problem occurs again. Currently, I in-memory processes as a
workaround, but that is of course not a realistic scenario for production systems.

Has anyone experienced similar problems? I would appreciate every help to make
my experiments with persistent processes. I attached two logfiles containing
complete stack traces and I could also provide more data from my extensive
series of tests.

Greetings from Berlin
Falko

Mime
View raw message