ode-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kharche, Jitendra" <Jitendra.Khar...@geometricglobal.com>
Subject RE: SocketTimeoutException in ODE
Date Tue, 25 Mar 2014 06:54:37 GMT
Please help. I am getting this error everyday.

Regards,
Jitendra

-----Original Message-----
From: Kharche, Jitendra [mailto:Jitendra.Kharche@geometricglobal.com]
Sent: Wednesday, March 19, 2014 3:31 PM
To: user@ode.apache.org
Subject: RE: SocketTimeoutException in ODE

Hi Sathwik,

Thanks.
But I cannot find parameter *ode-axi2.threads.pool.size* in ode-axis2.properties file (attached).
Do we need to explicitly configure it?
This file is default except the database (External - PostgreSql) and transaction factory as
GeronimoFactory.

>>Probable causes could be your database is slow on committing
>>transactions,
The database is dedicated for ODE. I do not think this could be a problem.

>> or ODE is being loaded with requests and ODE scheduler is taking time
>> to process them
The load on ODE is very minimal. Only one process runs at a time. All the 11 processes are
scheduled at an interval of 5 min, 4 times a day. And each process takes around 2-3 mins for
completion. So from processing perspective there is no load on ODE.

>> You might have to play around this parameter by increasing it to higher values if
jobs are piling up on the ODE scheduler.
With this much of minimal load, I don't think the parameter *ode-axi2.threads.pool.size*setting
will make any difference.

Regards,
Jitendra

-----Original Message-----
From: Sathwik B P [mailto:sathwik.bp@gmail.com]
Sent: Wednesday, March 19, 2014 3:11 PM
To: user@ode.apache.org
Subject: Re: SocketTimeoutException in ODE

Hi Jitendra,

Your facade process is taking more than 2mins to respond. For the log snippet it's not clear
where is it taking more time.

I can only speculate,
Probable causes could be your database is slow on committing transactions, or ODE is being
loaded with requests and ODE scheduler is taking time to process them.

What is the value of the parameter *ode-axi2.threads.pool.size* in your ode-axis2.properties
file?
You might have to play around this parameter by increasing it to higher values if jobs are
piling up on the ODE scheduler.

regards,
sathwik


On Tue, Mar 18, 2014 at 4:15 PM, Kharche, Jitendra < Jitendra.Kharche@geometricglobal.com>
wrote:

> Hi Sathwik,
>
> Sorry if am not clearly able to explain. To simplify, I have 2
> processes - 1. A Façade Process, 2. My Actual/Target Process.
>
> Actual Process has following broad steps:
> 1. Receive start request
> 2. Reply to caller to indicate process started successfully 3. Invoke
> an application adapter - async call 4. Receive feedback from
> application adapter 5. further steps
>
> Façade Process has following steps:
> 1. Receive start request
> 2. Invoke Actual Process
> 3. Reply to caller
>
>
> I am pasting the exception I see in ode logs when I see the
> SocketTimoutException on client.
>
> 2014-03-12 01:37:00,711 ERROR [ODEService] Timeout or execution error
> when waiting for response to MEX {MyRoleMex#hqejbhcnphr942iow8mfqj
> [Client hqejbhcnphr942iow8mfqi] calling
> {http://a/b/processes/ProcessesFacade}ProcessesFacadeService.exportFee
> dback(...)}
> java.util.concurrent.TimeoutException: Message exchange
> org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture@75f385cetimed out(120000
ms) when waiting for a response!
> java.util.concurrent.TimeoutException: Message exchange
> org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture@75f385cetimed out(120000
ms) when waiting for a response!
>         at
> org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture.get(MyRoleMessageExchangeImpl.java:245)
>         at
> org.apache.ode.axis2.ODEService.onAxisMessageExchange(ODEService.java:162)
>         at
> org.apache.ode.axis2.hooks.ODEMessageReceiver.invokeBusinessLogic(ODEMessageReceiver.java:69)
>         at
> org.apache.ode.axis2.hooks.ODEMessageReceiver.invokeBusinessLogic(ODEMessageReceiver.java:52)
>         at
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:114)
>         at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:173)
>         at
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:173)
>         at
> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:144)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>         at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>         at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:611)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
>         at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>         at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
>         at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
>         at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
>         at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:724)
> 2014-03-12 01:37:00,789 ERROR [INVOKE] Failure during invoke: No
> response received for invoke (mexId=hqejbhcnphr942iow8mfqn), forcing
> it into a failed state.
> 2014-03-12 01:37:00,789 INFO  [BpelRuntimeContextImpl] ActivityRecovery:
> Registering activity 9, failure reason: No response received for
> invoke (mexId=hqejbhcnphr942iow8mfqn), forcing it into a failed state.
> on channel
> 18
>
> Regards,
> Jitendra
>
> -----Original Message-----
> From: Sathwik B P [mailto:sathwik.bp@gmail.com]
> Sent: Tuesday, March 18, 2014 3:40 PM
> To: user@ode.apache.org
> Subject: Re: SocketTimeoutException in ODE
>
> Can you attach the ODE logs.
>
> I don't think the process model was clear to me after your latest mail.
> You said that there 2 recevie (bpel activites) in each process and
> then in the latest mail you specify that there is receive-reply (bpel activities).
>
>
> On Tue, Mar 18, 2014 at 2:14 AM, Kharche, Jitendra <
> Jitendra.Kharche@geometricglobal.com> wrote:
>
> > Hi Sathwik,
> >
> > Thanks for quick reply.
> >
> > The façade process is request-response. When it receives a request,
> > it invokes the target process. Each target process replies as soon
> > as it receives a request (before starting its work) - so there is no
> > blocking in the target process for the façade process. As soon as
> > the façade receives the reply from the target process it replies
> > back to the client - no extra processing in the façade process.
> >
> > >> client side raises the exception since it has not recived the
> > >> response
> > within the stiputaled time.
> > This is obvious. However I am looking for the reasons for this. As I
> > mentioned this happens rarely - not always and also randomly for any
> > target process - not for any specific target process.
> >
> > Hope I am clear.
> >
> > Regards,
> > Jitendra
> >
> > -----Original Message-----
> > From: Sathwik B P [mailto:sathwik.bp@gmail.com]
> > Sent: Tuesday, March 18, 2014 2:28 PM
> > To: user@ode.apache.org
> > Subject: Re: SocketTimeoutException in ODE
> >
> > Hi Jitendra,
> >
> > Is your facade process exposed as request-response or one-way mep?
> >
> > In case of request-response mep and if the facade process is a time
> > consuming process, then you are bound to see read time out exceptions.
> >
> > Process execution within ODE is asyncronous, and hence the process
> > continues to completion. But your transport channel on the client
> > side raises the exception since it has not recived the response
> > within the stiputaled time.
> >
> > regards,
> > sathwik
> >
> >
> > On Tue, Mar 18, 2014 at 1:27 AM, Kharche, Jitendra <
> > Jitendra.Kharche@geometricglobal.com> wrote:
> >
> > > Hi,
> > >
> > > I am using ODE 1.3.6 on Tomcat 7.0.47 with 11 processes deployed
> > > into
> it.
> > > On top of these I have a process that acts as a façade so that
> > > clients have a single interface to call. The interface of the
> > > façade process is a consolidation of interfaces provided by all processes.
> > > This façade process then invokes the actual process.
> > >
> > > Each process is a long running using explicit correlation.
> > > Each process has 2 receive activities - first receive starts the
> > > process and second receive after receiving a completion reply from
> > > the application adapter.
> > > There is a scheduler to execute processes in scheduled manner. The
> > > scheduler invokes the façade process which then invokes the target
> > process.
> > >
> > > Randomly (once in 2-3 days for any random process), I get
> > > SocketTimeoutException in client (the scheduler) as follows during
> > > invocation of the process.
> > > Sometimes it is also observed that, even though the client got the
> > > exception the process started and completed successfully.
> > >
> > > What could be the cause?
> > >
> > > Thanks in advance.
> > >
> > > Exception in client:
> > >
> > >
> > > javax.xml.ws.WebServiceException: Could not send Message.
> > >
> > >                 at
> > > org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java
> > > :1
> > > 45
> > > )
> > >
> > >                 at com.sun.proxy.$Proxy104.startEmpSync3D(Unknown
> > > Source)
> > >
> > >                 at
> > > com.geo.ml.scheduler.jobs.EmpSync3DJob.executeProcess(EmpSync3DJob
> > > .j
> > > av
> > > a:115)
> > >
> > >                 at
> > > com.geo.ml.scheduler.jobs.EmpSync3DJob.execute(EmpSync3DJob.java:4
> > > 9)
> > >
> > >                 at
> > > org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> > >
> > >                 at
> > > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.
> > > ja
> > > va:557)
> > >
> > > Caused by: java.net.SocketTimeoutException: SocketTimeoutException
> > > invoking [Façade Process URL]: Read timed out
> > >
> > >                 at
> > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > Method)
> > >
> > >                 at
> > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstr
> > > uc
> > > to
> > > rAccessorImpl.java:57)
> > >
> > >                 at
> > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Delegati
> > > ng
> > > Co
> > > nstructorAccessorImpl.java:45)
> > >
> > >                 at
> > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > >
> > >                 at
> > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapE
> > > xc
> > > ep
> > > tion(HTTPConduit.java:1467)
> > >
> > >                 at
> > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.clos
> > > e(
> > > HT
> > > TPConduit.java:1452)
> > >
> > >                 at
> > > org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:
> > > 56
> > > )
> > >
> > >                 at
> > > org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:6
> > > 59
> > > )
> > >
> > >                 at
> > > org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderE
> > > nd
> > > in
> > > gInterceptor.handleMessage(MessageSenderInterceptor.java:62)
> > >
> > >                 at
> > > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterc
> > > ep
> > > to
> > > rChain.java:262)
> > >
> > >                 at
> > > org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
> > >
> > >                 at
> > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:463)
> > >
> > >                 at
> > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:366)
> > >
> > >                 at
> > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:319)
> > >
> > >                 at
> > > org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96
> > > )
> > >
> > >                 at
> > > org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java
> > > :1
> > > 34
> > > )
> > >
> > >                 ... 5 more
> > >
> > > Caused by: java.net.SocketTimeoutException: Read timed out
> > >
> > >                 at java.net.SocketInputStream.socketRead0(Native
> > > Method)
> > >
> > >                 at
> > > java.net.SocketInputStream.read(SocketInputStream.java:152)
> > >
> > >                 at
> > > java.net.SocketInputStream.read(SocketInputStream.java:122)
> > >
> > >                 at
> > > sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
> > >
> > >                 at
> > > sun.security.ssl.InputRecord.read(InputRecord.java:480)
> > >
> > >                 at
> > > sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
> > >
> > >                 at
> > > sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:8
> > > 84
> > > )
> > >
> > >                 at
> > > sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
> > >
> > >                 at
> > > java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> > >
> > >                 at
> > > java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> > >
> > >                 at
> > > java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> > >
> > >                 at
> > > sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
> > >
> > >                 at
> > > sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
> > >
> > >                 at
> > > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURL
> > > Co
> > > nn
> > > ection.java:1323)
> > >
> > >                 at
> > > java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:
> > > 46
> > > 8)
> > >
> > >                 at
> > > sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(
> > > Ht
> > > tp
> > > sURLConnectionImpl.java:338)
> > >
> > >                 at
> > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.hand
> > > le
> > > Re
> > > sponseInternal(HTTPConduit.java:1604)
> > >
> > >                 at
> > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.hand
> > > le
> > > Re
> > > sponse(HTTPConduit.java:1530)
> > >
> > >                 at
> > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.clos
> > > e(
> > > HT
> > > TPConduit.java:1438)
> > >
> > >                 ... 15 more
> > >
> > >
> > > Regards,
> > > Jitendra
> > >
> > >
> >
>

Mime
View raw message