各位达人,最近在生产环境上碰到一个很奇异的问题:
生产环境: weblogic815集群, oracle9i集群...
操作: 更改数据库的密码, 更改weblogic连接池的密码, 重新启动weblogic(受管和管理服务器), 未重启oracle(不能重启)
现象: 有一些操作正常, 但是有一些操作报错,错误信息如下
[code="java"]
java.sql.SQLException: 关闭的连接
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:125)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:162)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:227)
at oracle.jdbc.driver.PhysicalConnection.setAutoCommit(PhysicalConnectio
n.java:930)
at weblogic.jdbc.wrapper.PoolConnection.setAutoCommit(PoolConnection.jav
a:141)
at net.sf.hibernate.transaction.JDBCTransaction.toggleAutoCommit(JDBCTra
nsaction.java:104)
at net.sf.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction
.java:95)
at common.blh.XcBLH.execute(XcBLH.java:99)
at common.bizDelegate.FakeDelegate.invokeTask(FakeDelegate.java:58)
at common.bizDelegate.FakeDelegate.delegate(FakeDelegate.java:24)
at gov.gwssi.xcweb.common.XCAction.execute(XCAction.java:93)
at org.apache.struts.action.RequestProcessor.processActionPerform(Reques
tProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.ja
va:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:148
2)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run
(ServletStubImpl.java:1072)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubIm
pl.java:465)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.DoTimeoutFilter.doFilter(DoTimeoutFilt
er.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.SetCharacterEncodingFilter.doFilter(Se
tCharacterEncodingFilter.java:59)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationActio
n.run(WebAppServletContext.java:6987)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppSe
rvletContext.java:3892)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestIm
pl.java:2766)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
common.exception.XcBlException:
at gov.gwssi.xccore.blh.business.zhcx.PretreatYcfpZhcxBlh.performTask(Pr
etreatYcfpZhcxBlh.java:76)
at common.blh.XcBLH.execute(XcBLH.java:61)
at common.bizDelegate.FakeDelegate.invokeTask(FakeDelegate.java:58)
at common.bizDelegate.FakeDelegate.delegate(FakeDelegate.java:24)
at gov.gwssi.xcweb.common.XCAction.execute(XCAction.java:93)
at org.apache.struts.action.RequestProcessor.processActionPerform(Reques
tProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.ja
va:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:148
2)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run
(ServletStubImpl.java:1072)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubIm
pl.java:465)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.DoTimeoutFilter.doFilter(DoTimeoutFilt
er.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.SetCharacterEncodingFilter.doFilter(Se
tCharacterEncodingFilter.java:59)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationActio
n.run(WebAppServletContext.java:6987)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppSe
rvletContext.java:3892)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestIm
pl.java:2766)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
当前jsp的URI:/dkpz/default/syserror.jsp
OM DUAL" set up for pool "DkpzPool1" failed with exception: "java.sql.SQLExcepti
on: 必须登录到服务器".>
[/code]
敬请达人们指点
[b]问题补充：[/b]
各位达人,最近在生产环境上碰到一个很奇异的问题:
生产环境: weblogic815集群, oracle9i集群...
操作: 更改数据库的密码, 更改weblogic连接池的密码, 重新启动weblogic(受管和管理服务器), 未重启oracle(不能重启)
现象: 有一些操作正常, 但是有一些操作报错,错误信息如下
[code="java"]
ERROR ExecuteThread: '11' for queue: 'weblogic.kernel.Default' - Rollback failed
java.sql.SQLException: 关闭的连接
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:125)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:162)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:227)
at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.jav
a:1031)
at weblogic.jdbc.wrapper.PoolConnection_oracle_jdbc_driver_T4CConnection
.rollback(Unknown Source)
at net.sf.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction
.java:86)
at common.blh.XcBLH.execute(XcBLH.java:99)
at common.bizDelegate.FakeDelegate.invokeTask(FakeDelegate.java:58)
at common.bizDelegate.FakeDelegate.delegate(FakeDelegate.java:24)
at gov.gwssi.xcweb.common.XCAction.execute(XCAction.java:93)
at org.apache.struts.action.RequestProcessor.processActionPerform(Reques
tProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.ja
va:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:148
2)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run
(ServletStubImpl.java:1072)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubIm
pl.java:465)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.DoTimeoutFilter.doFilter(DoTimeoutFilt
er.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.SetCharacterEncodingFilter.doFilter(Se
tCharacterEncodingFilter.java:59)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationActio
n.run(WebAppServletContext.java:6987)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppSe
rvletContext.java:3892)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestIm
pl.java:2766)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
ERROR ExecuteThread: '11' for queue: 'weblogic.kernel.Default' - Could not toggle autocommit
java.sql.SQLException: 关闭的连接
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:125)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:162)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java
:227)
at oracle.jdbc.driver.PhysicalConnection.setAutoCommit(PhysicalConnectio
n.java:930)
at weblogic.jdbc.wrapper.PoolConnection.setAutoCommit(PoolConnection.jav
a:141)
at net.sf.hibernate.transaction.JDBCTransaction.toggleAutoCommit(JDBCTra
nsaction.java:104)
at net.sf.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction
.java:95)
at common.blh.XcBLH.execute(XcBLH.java:99)
at common.bizDelegate.FakeDelegate.invokeTask(FakeDelegate.java:58)
at common.bizDelegate.FakeDelegate.delegate(FakeDelegate.java:24)
at gov.gwssi.xcweb.common.XCAction.execute(XCAction.java:93)
at org.apache.struts.action.RequestProcessor.processActionPerform(Reques
tProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.ja
va:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:148
2)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run
(ServletStubImpl.java:1072)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubIm
pl.java:465)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.DoTimeoutFilter.doFilter(DoTimeoutFilt
er.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.SetCharacterEncodingFilter.doFilter(Se
tCharacterEncodingFilter.java:59)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationActio
n.run(WebAppServletContext.java:6987)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppSe
rvletContext.java:3892)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestIm
pl.java:2766)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
common.exception.XcBlException:
at gov.gwssi.xccore.blh.business.zhcx.PretreatYcfpZhcxBlh.performTask(Pr
etreatYcfpZhcxBlh.java:76)
at common.blh.XcBLH.execute(XcBLH.java:61)
at common.bizDelegate.FakeDelegate.invokeTask(FakeDelegate.java:58)
at common.bizDelegate.FakeDelegate.delegate(FakeDelegate.java:24)
at gov.gwssi.xcweb.common.XCAction.execute(XCAction.java:93)
at org.apache.struts.action.RequestProcessor.processActionPerform(Reques
tProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.ja
va:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:148
2)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run
(ServletStubImpl.java:1072)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubIm
pl.java:465)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:28)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.DoTimeoutFilter.doFilter(DoTimeoutFilt
er.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at gov.gwssi.xcweb.common.filters.SetCharacterEncodingFilter.doFilter(Se
tCharacterEncodingFilter.java:59)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.ja
va:27)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationActio
n.run(WebAppServletContext.java:6987)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppSe
rvletContext.java:3892)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestIm
pl.java:2766)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
当前jsp的URI:/dkpz/default/syserror.jsp
OM DUAL" set up for pool "DkpzPool1" failed with exception: "java.sql.SQLExcepti
on: 必须登录到服务器".>
[/code]
敬请达人们指点
[b]问题补充：[/b]
之前只设置了Test Reserved Connections
但是刚刚把
Test Reserved Connections
Test Created Connections
Test Released Connections
都设置上了也不行
[b]问题补充：[/b]
oracle没有超过连接数,任何时候都只有 特定的 业务模块才会出现这种现象,其它的模块没有
而有问题的业务的实现方式与正常的实现方式是一样的
[b]问题补充：[/b]
Web用的是Struts1.2, 持久层用的是Hibernate,封装的Session.
具体的流程是
BaseAction是一个基于Struts的模板模式,里面通过代理来调用业务逻辑方法
在代理中,首先获取一个hibernate的session,hibernate引用的是WEBLOGIC配置的数据源,用之前创建,用完后关闭(只关闭session,没有关闭对应的连接)
也就是说,在进行业务操作之前,都会新创建一个会话
另外,出现问题的业务模板与正常的业务模块都是基于这种框架
public abstract class BaseAction{
final ... execute(...){
...
prepareRequest();////这是个抽象方法,处理请求的数据
.....////这儿通过代理调用业务方法,即BaseBlh.performTask()
processResponse(...);/////抽象方法,处理业务生成的数据
abstract DTO prepareRequest(...);
abstract ActionForward processResponse(...);
}
public abstract class BaseBlh(){
abstract DTO performTask(Session session);
}
public BlhProxy{
public DTO performTask(...){
Session session = ////通过hibernate获取,用的是Weblogic配置的数据源
BaseBlh blh = ...../////2.通过反射,获取当前具体的业务类
DTO dto = blh.performTask(session);
session.close();
return dto;
////这里面省略了事务等等的操作
}
}
不知道我描述清楚了没有
再次谢谢
[b]问题补充：[/b]
另外,就是在进行数据库密码修改之前,一切正常
[b]问题补充：[/b]
to: bohemia
非常感谢你的关注, 今天去了一趟生产环境的现场, 通过日志(之前的异常信息是技术支持传过来的)发现, 在业务操作过程中, 有一个过程出现问题
抛出了 ora-00600 internal error code [...] [][][][]
异常
导致程序没能正常执行
但是存储过程编译没有出现任何问题,经过排查,发现,在存储过程中有一个游标
select column1, column2...., functionName(...) columnN from tableName;
无法执行,将其单独执行抛出ora-00600异常
后来将functionName(...) columnN这一列去掉后,查询正常.
此时, 发现, 通过plsql直接打开(利用右键)时,居然提示该函数不存在!!! 但是该函数的确存在,并且编译存储过程也没有任何问题. 通过其它方式打开该函数,尝试重新编译, 重新查询, 问题解决.
问题虽然解决,但是还是有几个疑问:
1. 为什么函数functionName(...)为出现这个问题
2. 为什么在oracle出现异常后连接会关闭
[b]问题补充：[/b]
ora-00600是一个oracle内容错误,出现错误的原因在oracle日志里面给的不是很详细,现在虽然找着了解决问题的办法.
另外,在业务操作的过程中,每一次操作之前获取一个连接,这个连接是有效的,但是在oracle抛出了那个ora-00600异常之后,这个连接居然"自动"断开了,很是费解, 因为业务操作过程是这样子的
1. 打开连接,开启事务
2.1 业务操作a 需要访问数据库
2.2 业务操作b 需要访问数据库
3. 提交(回滚)事务,关闭连接
在整个流程中,只有步骤3才会关闭连接,而在2.1操作抛出ora-00600错误后,再进行2.2操作时,就发现连接已经断开了
