生产环境线程死锁，服务假死。

Question

生产环境线程死锁，请问这是什么原因导致的，如何解决？

<2023-10-31 下午10时54分19秒 CST> <Critical> <WebLogicServer> <BEA-000394> <

DEADLOCK DETECTED:
==================

[deadlocked thread] QuartzScheduler_scheduler-localhost.localdomain1698650301518_ClusterManager:
-----------------------------------------------------------------------------------------------
Thread 'QuartzScheduler_scheduler-localhost.localdomain1698650301518_ClusterManager' is waiting to acquire lock 'org.apache.commons.dbcp.AbandonedObjectPool@5176ade8' that is held by thread 'Timer-2'

Stack trace:
------------
    org.apache.commons.pool.impl.GenericObjectPool.addObjectToPool(GenericObjectPool.java:1136)
    org.apache.commons.pool.impl.GenericObjectPool.returnObject(GenericObjectPool.java:1076)
    org.apache.commons.dbcp.AbandonedObjectPool.returnObject(AbandonedObjectPool.java:100)
    org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:80)
    org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(PoolingDataSource.java:180)
    org.springframework.jdbc.datasource.DataSourceUtils.doReleaseConnection(DataSourceUtils.java:333)
    org.springframework.jdbc.datasource.DataSourceUtils.releaseConnection(DataSourceUtils.java:294)
    org.springframework.scheduling.quartz.LocalDataSourceJobStore.closeConnection(LocalDataSourceJobStore.java:158)
    org.quartz.impl.jdbcjobstore.JobStoreSupport.cleanupConnection(JobStoreSupport.java:3603)
    org.quartz.impl.jdbcjobstore.JobStoreSupport.doCheckin(JobStoreSupport.java:3286)
    org.quartz.impl.jdbcjobstore.JobStoreSupport$ClusterManager.manage(JobStoreSupport.java:3870)
    org.quartz.impl.jdbcjobstore.JobStoreSupport$ClusterManager.run(JobStoreSupport.java:3907)

[deadlocked thread] Timer-2:
---------------------------
Thread 'Timer-2' is waiting to acquire lock 'org.apache.commons.dbcp.PoolableConnection@6c5b68ac' that is held by thread 'QuartzScheduler_scheduler-localhost.localdomain1698650301518_ClusterManager'

Stack trace:
------------
    org.apache.commons.dbcp.AbandonedTrace.addTrace(AbandonedTrace.java:172)
    org.apache.commons.dbcp.AbandonedTrace.init(AbandonedTrace.java:89)
    org.apache.commons.dbcp.AbandonedTrace.<init>(AbandonedTrace.java:79)
    org.apache.commons.dbcp.DelegatingStatement.<init>(DelegatingStatement.java:60)
    org.apache.commons.dbcp.DelegatingConnection.createStatement(DelegatingConnection.java:161)
    org.apache.commons.dbcp.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:327)
    org.apache.commons.dbcp.PoolableConnectionFactory.validateObject(PoolableConnectionFactory.java:308)
    org.apache.commons.pool.impl.GenericObjectPool.evict(GenericObjectPool.java:1217)
    org.apache.commons.pool.impl.GenericObjectPool$Evictor.run(GenericObjectPool.java:1341)
    java.util.TimerThread.mainLoop(Timer.java:512)
    java.util.TimerThread.run(Timer.java:462)

> 
<2023-10-31 下午10时54分19秒 CST> <Critical> <Health> <BEA-310006> <Critical Subsystem core has failed. Setting server state to FAILED.
Reason: Thread deadlock detected> 
<2023-10-31 下午10时54分19秒 CST> <Critical> <WebLogicServer> <BEA-000385> <Server health failed. Reason: health of critical service 'core' failed> 
<2023-10-31 下午10时54分19秒 CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED> 
<2023-10-31 下午10时54分19秒 CST> <Error> <> <BEA-000000> <THREAD DUMP from JVM taken at 'Tue Oct 31 22:54:19 2023'


2023-10-31 22:54:19
Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.10-b01 mixed mode):

"MultiThreadedHttpConnectionManager cleanup" daemon prio=10 tid=0x00007fb4e410d000 nid=0x14734 in Object.wait() [0x00007fb405cb8000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x0000000629963418> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
    - locked <0x0000000629963418> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
    at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)

Answer 1

该回答通过自己思路及引用到GPTᴼᴾᴱᴺᴬᴵ搜索,得到内容具体如下：
线程死锁是指在程序运行过程中，两个或多个线程互相等待对方释放资源，导致程序无法继续执行。在生产环境中，线程死锁可能导致服务假死，影响用户体验。

解决这个问题的方法有以下几种：

1. 优化代码逻辑，避免出现循环等待的情况。例如，可以使用信号量、计数器等同步机制来控制线程的执行顺序。

2. 调整线程池参数，如核心线程数、最大线程数、空闲线程存活时间等，以减少线程竞争和死锁的可能性。

3. 使用定时任务或者调度器（如Quartz）来管理线程，确保线程按照预期的顺序执行。

4. 如果可能的话，尝试升级到更高版本的Java虚拟机（JVM），因为新版本的JVM可能会提供更好的线程管理和死锁检测功能。

5. 在开发和测试阶段，使用调试工具（如VisualVM、JConsole等）来监控线程状态和资源占用情况，以便及时发现并解决问题。

---

如果以上回答对您有所帮助，点击一下采纳该答案～谢谢