ode-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sathwik B P <sathwik...@gmail.com>
Subject Re: SocketTimeoutException in ODE
Date Tue, 25 Mar 2014 08:36:48 GMT
Jitendra,

It's hard to identify why is your process taking such a longtime to
respond. Your logs are not indicating the real problem.
It will be helpful if you can share the bpel of your process and if you
can't, make a simulated replica of your process.

Can you share the server.xml from tomcat and ode-axis2.properties file.

You may be interested in looking into my response on this thread in
configuring ode and tomcat.
http://markmail.org/message/sfbb6otrbp5prcce?q=Error+when+connecting+ODE+with+bitronix

regards,
sathwik


On Tue, Mar 25, 2014 at 12:24 PM, Kharche, Jitendra <
Jitendra.Kharche@geometricglobal.com> wrote:

> Please help. I am getting this error everyday.
>
> Regards,
> Jitendra
>
> -----Original Message-----
> From: Kharche, Jitendra [mailto:Jitendra.Kharche@geometricglobal.com]
> Sent: Wednesday, March 19, 2014 3:31 PM
> To: user@ode.apache.org
> Subject: RE: SocketTimeoutException in ODE
>
> Hi Sathwik,
>
> Thanks.
> But I cannot find parameter *ode-axi2.threads.pool.size* in
> ode-axis2.properties file (attached). Do we need to explicitly configure it?
> This file is default except the database (External - PostgreSql) and
> transaction factory as GeronimoFactory.
>
> >>Probable causes could be your database is slow on committing
> >>transactions,
> The database is dedicated for ODE. I do not think this could be a problem.
>
> >> or ODE is being loaded with requests and ODE scheduler is taking time
> >> to process them
> The load on ODE is very minimal. Only one process runs at a time. All the
> 11 processes are scheduled at an interval of 5 min, 4 times a day. And each
> process takes around 2-3 mins for completion. So from processing
> perspective there is no load on ODE.
>
> >> You might have to play around this parameter by increasing it to higher
> values if jobs are piling up on the ODE scheduler.
> With this much of minimal load, I don't think the parameter
> *ode-axi2.threads.pool.size*setting will make any difference.
>
> Regards,
> Jitendra
>
> -----Original Message-----
> From: Sathwik B P [mailto:sathwik.bp@gmail.com]
> Sent: Wednesday, March 19, 2014 3:11 PM
> To: user@ode.apache.org
> Subject: Re: SocketTimeoutException in ODE
>
> Hi Jitendra,
>
> Your facade process is taking more than 2mins to respond. For the log
> snippet it's not clear where is it taking more time.
>
> I can only speculate,
> Probable causes could be your database is slow on committing transactions,
> or ODE is being loaded with requests and ODE scheduler is taking time to
> process them.
>
> What is the value of the parameter *ode-axi2.threads.pool.size* in your
> ode-axis2.properties file?
> You might have to play around this parameter by increasing it to higher
> values if jobs are piling up on the ODE scheduler.
>
> regards,
> sathwik
>
>
> On Tue, Mar 18, 2014 at 4:15 PM, Kharche, Jitendra <
> Jitendra.Kharche@geometricglobal.com> wrote:
>
> > Hi Sathwik,
> >
> > Sorry if am not clearly able to explain. To simplify, I have 2
> > processes - 1. A Façade Process, 2. My Actual/Target Process.
> >
> > Actual Process has following broad steps:
> > 1. Receive start request
> > 2. Reply to caller to indicate process started successfully 3. Invoke
> > an application adapter - async call 4. Receive feedback from
> > application adapter 5. further steps
> >
> > Façade Process has following steps:
> > 1. Receive start request
> > 2. Invoke Actual Process
> > 3. Reply to caller
> >
> >
> > I am pasting the exception I see in ode logs when I see the
> > SocketTimoutException on client.
> >
> > 2014-03-12 01:37:00,711 ERROR [ODEService] Timeout or execution error
> > when waiting for response to MEX {MyRoleMex#hqejbhcnphr942iow8mfqj
> > [Client hqejbhcnphr942iow8mfqi] calling
> > {http://a/b/processes/ProcessesFacade}ProcessesFacadeService.exportFee
> > dback(...)}
> > java.util.concurrent.TimeoutException: Message exchange
> >
> org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture@75f385cetimedout(120000
ms) when waiting for a response!
> > java.util.concurrent.TimeoutException: Message exchange
> >
> org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture@75f385cetimedout(120000
ms) when waiting for a response!
> >         at
> >
> org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture.get(MyRoleMessageExchangeImpl.java:245)
> >         at
> >
> org.apache.ode.axis2.ODEService.onAxisMessageExchange(ODEService.java:162)
> >         at
> >
> org.apache.ode.axis2.hooks.ODEMessageReceiver.invokeBusinessLogic(ODEMessageReceiver.java:69)
> >         at
> >
> org.apache.ode.axis2.hooks.ODEMessageReceiver.invokeBusinessLogic(ODEMessageReceiver.java:52)
> >         at
> >
> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:114)
> >         at
> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:173)
> >         at
> >
> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:173)
> >         at
> > org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:144)
> >         at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
> >         at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
> >         at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
> >         at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> >         at
> > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
> >         at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> >         at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> >         at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> >         at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> >         at
> >
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:611)
> >         at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> >         at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
> >         at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> >         at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> >         at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> >         at
> >
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
> >         at
> >
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
> >         at
> >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >         at java.lang.Thread.run(Thread.java:724)
> > 2014-03-12 01:37:00,789 ERROR [INVOKE] Failure during invoke: No
> > response received for invoke (mexId=hqejbhcnphr942iow8mfqn), forcing
> > it into a failed state.
> > 2014-03-12 01:37:00,789 INFO  [BpelRuntimeContextImpl] ActivityRecovery:
> > Registering activity 9, failure reason: No response received for
> > invoke (mexId=hqejbhcnphr942iow8mfqn), forcing it into a failed state.
> > on channel
> > 18
> >
> > Regards,
> > Jitendra
> >
> > -----Original Message-----
> > From: Sathwik B P [mailto:sathwik.bp@gmail.com]
> > Sent: Tuesday, March 18, 2014 3:40 PM
> > To: user@ode.apache.org
> > Subject: Re: SocketTimeoutException in ODE
> >
> > Can you attach the ODE logs.
> >
> > I don't think the process model was clear to me after your latest mail.
> > You said that there 2 recevie (bpel activites) in each process and
> > then in the latest mail you specify that there is receive-reply (bpel
> activities).
> >
> >
> > On Tue, Mar 18, 2014 at 2:14 AM, Kharche, Jitendra <
> > Jitendra.Kharche@geometricglobal.com> wrote:
> >
> > > Hi Sathwik,
> > >
> > > Thanks for quick reply.
> > >
> > > The façade process is request-response. When it receives a request,
> > > it invokes the target process. Each target process replies as soon
> > > as it receives a request (before starting its work) - so there is no
> > > blocking in the target process for the façade process. As soon as
> > > the façade receives the reply from the target process it replies
> > > back to the client - no extra processing in the façade process.
> > >
> > > >> client side raises the exception since it has not recived the
> > > >> response
> > > within the stiputaled time.
> > > This is obvious. However I am looking for the reasons for this. As I
> > > mentioned this happens rarely - not always and also randomly for any
> > > target process - not for any specific target process.
> > >
> > > Hope I am clear.
> > >
> > > Regards,
> > > Jitendra
> > >
> > > -----Original Message-----
> > > From: Sathwik B P [mailto:sathwik.bp@gmail.com]
> > > Sent: Tuesday, March 18, 2014 2:28 PM
> > > To: user@ode.apache.org
> > > Subject: Re: SocketTimeoutException in ODE
> > >
> > > Hi Jitendra,
> > >
> > > Is your facade process exposed as request-response or one-way mep?
> > >
> > > In case of request-response mep and if the facade process is a time
> > > consuming process, then you are bound to see read time out exceptions.
> > >
> > > Process execution within ODE is asyncronous, and hence the process
> > > continues to completion. But your transport channel on the client
> > > side raises the exception since it has not recived the response
> > > within the stiputaled time.
> > >
> > > regards,
> > > sathwik
> > >
> > >
> > > On Tue, Mar 18, 2014 at 1:27 AM, Kharche, Jitendra <
> > > Jitendra.Kharche@geometricglobal.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I am using ODE 1.3.6 on Tomcat 7.0.47 with 11 processes deployed
> > > > into
> > it.
> > > > On top of these I have a process that acts as a façade so that
> > > > clients have a single interface to call. The interface of the
> > > > façade process is a consolidation of interfaces provided by all
> processes.
> > > > This façade process then invokes the actual process.
> > > >
> > > > Each process is a long running using explicit correlation.
> > > > Each process has 2 receive activities - first receive starts the
> > > > process and second receive after receiving a completion reply from
> > > > the application adapter.
> > > > There is a scheduler to execute processes in scheduled manner. The
> > > > scheduler invokes the façade process which then invokes the target
> > > process.
> > > >
> > > > Randomly (once in 2-3 days for any random process), I get
> > > > SocketTimeoutException in client (the scheduler) as follows during
> > > > invocation of the process.
> > > > Sometimes it is also observed that, even though the client got the
> > > > exception the process started and completed successfully.
> > > >
> > > > What could be the cause?
> > > >
> > > > Thanks in advance.
> > > >
> > > > Exception in client:
> > > >
> > > >
> > > > javax.xml.ws.WebServiceException: Could not send Message.
> > > >
> > > >                 at
> > > > org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java
> > > > :1
> > > > 45
> > > > )
> > > >
> > > >                 at com.sun.proxy.$Proxy104.startEmpSync3D(Unknown
> > > > Source)
> > > >
> > > >                 at
> > > > com.geo.ml.scheduler.jobs.EmpSync3DJob.executeProcess(EmpSync3DJob
> > > > .j
> > > > av
> > > > a:115)
> > > >
> > > >                 at
> > > > com.geo.ml.scheduler.jobs.EmpSync3DJob.execute(EmpSync3DJob.java:4
> > > > 9)
> > > >
> > > >                 at
> > > > org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> > > >
> > > >                 at
> > > > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.
> > > > ja
> > > > va:557)
> > > >
> > > > Caused by: java.net.SocketTimeoutException: SocketTimeoutException
> > > > invoking [Façade Process URL]: Read timed out
> > > >
> > > >                 at
> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > > Method)
> > > >
> > > >                 at
> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstr
> > > > uc
> > > > to
> > > > rAccessorImpl.java:57)
> > > >
> > > >                 at
> > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Delegati
> > > > ng
> > > > Co
> > > > nstructorAccessorImpl.java:45)
> > > >
> > > >                 at
> > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > > >
> > > >                 at
> > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapE
> > > > xc
> > > > ep
> > > > tion(HTTPConduit.java:1467)
> > > >
> > > >                 at
> > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.clos
> > > > e(
> > > > HT
> > > > TPConduit.java:1452)
> > > >
> > > >                 at
> > > > org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:
> > > > 56
> > > > )
> > > >
> > > >                 at
> > > > org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:6
> > > > 59
> > > > )
> > > >
> > > >                 at
> > > > org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderE
> > > > nd
> > > > in
> > > > gInterceptor.handleMessage(MessageSenderInterceptor.java:62)
> > > >
> > > >                 at
> > > > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterc
> > > > ep
> > > > to
> > > > rChain.java:262)
> > > >
> > > >                 at
> > > > org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
> > > >
> > > >                 at
> > > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:463)
> > > >
> > > >                 at
> > > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:366)
> > > >
> > > >                 at
> > > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:319)
> > > >
> > > >                 at
> > > > org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96
> > > > )
> > > >
> > > >                 at
> > > > org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java
> > > > :1
> > > > 34
> > > > )
> > > >
> > > >                 ... 5 more
> > > >
> > > > Caused by: java.net.SocketTimeoutException: Read timed out
> > > >
> > > >                 at java.net.SocketInputStream.socketRead0(Native
> > > > Method)
> > > >
> > > >                 at
> > > > java.net.SocketInputStream.read(SocketInputStream.java:152)
> > > >
> > > >                 at
> > > > java.net.SocketInputStream.read(SocketInputStream.java:122)
> > > >
> > > >                 at
> > > > sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
> > > >
> > > >                 at
> > > > sun.security.ssl.InputRecord.read(InputRecord.java:480)
> > > >
> > > >                 at
> > > > sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
> > > >
> > > >                 at
> > > > sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:8
> > > > 84
> > > > )
> > > >
> > > >                 at
> > > > sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
> > > >
> > > >                 at
> > > > java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> > > >
> > > >                 at
> > > > java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> > > >
> > > >                 at
> > > > java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> > > >
> > > >                 at
> > > > sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
> > > >
> > > >                 at
> > > > sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
> > > >
> > > >                 at
> > > > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURL
> > > > Co
> > > > nn
> > > > ection.java:1323)
> > > >
> > > >                 at
> > > > java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:
> > > > 46
> > > > 8)
> > > >
> > > >                 at
> > > > sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(
> > > > Ht
> > > > tp
> > > > sURLConnectionImpl.java:338)
> > > >
> > > >                 at
> > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.hand
> > > > le
> > > > Re
> > > > sponseInternal(HTTPConduit.java:1604)
> > > >
> > > >                 at
> > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.hand
> > > > le
> > > > Re
> > > > sponse(HTTPConduit.java:1530)
> > > >
> > > >                 at
> > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.clos
> > > > e(
> > > > HT
> > > > TPConduit.java:1438)
> > > >
> > > >                 ... 15 more
> > > >
> > > >
> > > > Regards,
> > > > Jitendra
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message