springboot项目 异常io输出

Question

腾讯云linux 服务器 中 tomcat 部署springboot，过几个小时自动宕机

打印了当时的IO日志，如下

ps:4.11-5.25之间，io日志消失了，怀疑4.11开始已经出问题了。

这是tomcat 崩溃时候的日志

16-May-2024 04:19:18.310 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:07:25] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:07:25: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,345,201,577,740,177 lastInboundActivity=10,345,239,459,302,303 time=10,345,222,111,353,126 lastPing=10,345,201,745,951,564
16-May-2024 04:19:02.550 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:06:47] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:06:47: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,345,138,020,134,454 lastInboundActivity=10,345,107,264,819,380 time=10,345,158,331,801,992 lastPing=10,345,138,219,121,945
16-May-2024 04:25:47.763 INFO [http-nio-8094-exec-3] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
 Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
    java.lang.IllegalArgumentException: Invalid character found in method name [0x160x030x010x000xee0x010x000x000xea0x030x03+}0xe60xc6c0xb40xf8N0xc740xd6d0xc370x01:H0x1d0x9c>O)0xf4(0xa30x020xd90xd10xa4M*0x09 ]. HTTP method names must be tokens
        at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:434)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:513)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:885)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1688)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:750)
16-May-2024 04:25:48.180 INFO [http-nio-8094-exec-9] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
 Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
    java.lang.IllegalArgumentException: Invalid character found in method name [0x160x030x010x000xca0x010x000x000xc60x030x03z0xc80xc00x180x8fC0xa7~b0xf6r0xff0x97_T0x8f>0xcc0xd00x9f0xab0x8f'0xbb0xf50xeb0x100xa2oV0xed0x860x000x00h0xcc0x140xcc0x130xc0/0xc0+0xc000xc0,0xc00x110xc00x070xc0'0xc0#0xc00x130xc00x090xc0(0xc0$0xc00x140xc00x0a0xcc0x150x000x9e0x000x9f0x00g0x00k0x0030x0090x000x9c0x000x9d0x000x050x000x040x00<0x00=0x00/0x0050xc00x120x000x160x000x0a0x000x030x000x080x000x060x000x140x000x110x000x190x000x170x0020xc00x080x000x120x000x130x000x150x0080x00@0x00f0x00j0x000xa20x000xa30x010x000x0050x000x050x000x050x010x000x000x000x000x000x0a0x000x080x000x060x000x170x000x180x000x190x000x0b0x000x020x010x000x000x0d0x000x0c0x000x0a0x040x010x040x030x020x010x020x030x020x020xff0x010x000x010x000x000x0f0x000x010x01...]. HTTP method names must be tokens
        at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:434)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:513)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:885)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1688)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:750)
2024-05-16 04:28:36.859  WARN 2012468 --- [SchedulerThread] c.a.druid.pool.DruidAbstractDataSource   : discard long time none received connection. , jdbcUrl : jdbc:mysql://82.157.118.175:8089/bingshan?useUnicode=true&characterEncoding=utf-8&useSSL=false&nullCatalogMeansCurrent=true&serverTimezone=Asia/Shanghai&allowMultiQueries=true&autoReconnect=true, version : 1.2.9, lastPacketReceivedIdleMillis : 592881
16-May-2024 05:01:15.544 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:07:25] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:07:25: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,347,989,158,140,917 lastInboundActivity=10,347,958,923,084,791 time=10,348,010,489,720,378 lastPing=10,347,989,375,569,495
16-May-2024 05:01:15.544 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:06:47] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:06:47: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,347,991,880,345,976 lastInboundActivity=10,347,958,918,439,116 time=10,348,012,611,210,848 lastPing=10,347,992,190,194,151
16-May-2024 05:05:54.387 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:07:25] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:07:25: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,348,275,290,233,708 lastInboundActivity=10,348,254,198,775,554 time=10,348,295,117,503,857 lastPing=10,348,275,435,866,375
16-May-2024 05:06:26.322 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:06:47] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:06:47: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,348,289,472,857,438 lastInboundActivity=10,348,259,985,166,481 time=10,348,314,516,778,771 lastPing=10,348,289,686,103,460
16-May-2024 05:09:53.886 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:07:25] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:07:25: Timed out as no write activity, keepAlive=20,000,000,000 lastOutboundActivity=10,348,417,018,321,122 lastInboundActivity=10,348,419,246,500,697 time=10,348,501,210,565,068 lastPing=10,348,275,435,866,375
2024-05-16 05:10:33.023 ERROR 2012468 --- [5/2024 20:07:25] com.jeeplus.mqtt.paho.ClientMQTT         : connectionLost,重连失败
2024-05-16 05:14:20.997 ERROR 2012468 --- [5/2024 20:06:47] com.jeeplus.mqtt.paho.ClientMQTT         : connectionLost,重连失败
16-May-2024 05:14:21.027 INFO [http-nio-8095-exec-3] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
 Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
    java.lang.IllegalArgumentException: Invalid character found in method name [0x160x030x010x01>0x010x000x01:0x030x030xaca0xa30xa9'0xb40xf940xb7a0xa3;"B'0x1b0xf70xe20x95]0xd30x9e0x0c0xebR0xaf0x820x010x13]0xc2d ]. HTTP method names must be tokens
        at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:434)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:513)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:885)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1688)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:750)
16-May-2024 05:23:44.675 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:07:25] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:07:25: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,349,345,746,165,219 lastInboundActivity=10,349,318,746,168,782 time=10,349,367,062,935,126 lastPing=10,349,346,067,153,594
16-May-2024 05:36:53.796 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:07:25] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:07:25: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,350,092,867,019,570 lastInboundActivity=10,350,061,396,835,841 time=10,350,113,426,532,169 lastPing=10,350,093,025,138,612
16-May-2024 05:45:28.956 SEVERE [MQTT Ping: client_cloud_15/05/2024 20:07:25] org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity client_cloud_15/05/2024 20:07:25: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=10,350,638,145,266,675 lastInboundActivity=10,350,609,224,232,201 time=10,350,671,466,673,078 lastPing=10,350,638,385,062,613
2024-05-16 06:03:17.006 ERROR 2012468 --- [5/2024 20:07:25] com.jeeplus.mqtt.paho.ClientMQTT         : connectionLost,重连失败

能帮忙看看啥原因吗，最近经常部署没多久，就崩了。（前几次是因为tomcat shutdown的时候，springboot里面那些进程没关掉，说是啥内存泄露，设置成守护进程也没啥用）

Answer 1

以下内容由CHATGPT及阿里嘎多学长共同生成、有用望采纳：

---

您想要解决的问题

您提问的目的是解决在腾讯云Linux服务器上部署的Spring Boot项目在Tomcat环境中运行一段时间后自动宕机的问题。

图片内容

• 第一张图片：显示了系统在某个时间点的磁盘读写情况，包括PID、用户、磁盘读取速度、磁盘写入速度、SWAPIN以及对应的COMMAND。
• 第二张图片：显示了另一个时间点的磁盘读写情况，格式与第一张图片相似。

问题出现原因

根据您提供的信息和日志，问题可能出现的原因包括：

1. MQTT连接超时：日志中多次出现MQTT客户端由于没有活动而超时的情况。
2. HTTP请求解析错误：服务器在解析HTTP请求头时遇到了错误，这可能是由于接收到了非法的请求。
3. 数据库连接超时：Druid数据库连接池丢弃了长时间未接收到连接的连接。
4. 内存泄露：您提到之前的崩溃是因为Spring Boot中的进程在Tomcat关闭时没有正确关闭，这可能暗示了内存泄露问题。

问题解决方案

1. 检查MQTT配置和网络问题：确保MQTT客户端的配置正确，包括keepAlive时间设置，同时检查网络连接是否稳定。
2. 增强异常处理：在Spring Boot应用中增强对异常的捕获和处理，避免因异常导致整个应用的崩溃。
3. 优化数据库连接：检查数据库连接池的配置，确保有合理的超时设置和连接回收策略。
4. 内存和性能监控：使用监控工具监控应用的内存使用情况，查找可能的内存泄露。
5. 日志分析：定期分析应用日志，以便及时发现和解决问题。

提供代码

由于问题可能由多个原因引起，没有特定的代码可以解决所有问题，但是以下是一些通用的建议：

• 异常捕获：使用Spring Boot的@ControllerAdvice和@ExceptionHandler注解来全局处理异常。

@ControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(Exception.class)
    public ResponseEntity<String> handleException(Exception e) {
        // 处理异常逻辑
        return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body("Global Exception Handler: " + e.getMessage());
    }
}

• 数据库连接池配置：在application.properties或application.yml中配置Druid数据库连接池。

# application.properties
spring.datasource.druid.max-idle=10
spring.datasource.druid.min-idle=5
spring.datasource.druid.max-active=50
spring.datasource.druid.initial-size=5
spring.datasource.druid.validation-query=SELECT 1
spring.datasource.druid.test-on-borrow=false
spring.datasource.druid.test-on-return=false
spring.datasource.druid.test-while-idle=true
spring.datasource.druid.time-between-eviction-runs-millis=60000
spring.datasource.druid.min-evictable-idle-time-millis=300000
spring.datasource.druid.max-wait=10000
spring.datasource.druid.remove-abandoned=true
spring.datasource.druid.remove-abandoned-timeout=300
spring.datasource.druid.log-abandoned=true

代码运行方式

将上述代码片段添加到您的Spring Boot项目中，并根据需要调整配置。

代码预期运行结果

应用将能够更好地处理异常，减少因异常导致的宕机。数据库连接将更加稳定，减少超时情况。

推荐相关链接

• Spring Boot官方文档
• Druid数据库连接池文档
• MQTT协议官方文档
• Linux性能监控工具