spring-boot-starter-data-mongodb 并发查询慢问题，请大家指导下，看是什么问题

Question

问题描述：
最近在学习mongodb，发现mongodb并发查询比较慢，集合的数据大小2G左右，总共文档6万+，其中每个文档包含1000左右的子文档，单次查询响应时间在400ms左右，10个线程并发查询响应时间在1800ms左右，并发线程数增加，对应的查询响应时间也在增加

spring-boot-starter-data-mongodb版本

   <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
            <version>2.2.1.RELEASE</version>
   </dependency>

spring boot 配置

spring:
  data:
    mongodb:
      host: 127.0.0.1
      port: 27017
      authentication-database: admin
      database: xxx
      connections-per-host: 200
      min-connections-per-host: 20
      max-connection-idel-time: 0
      threadsAllowedToBlockForConnectionMultiplier: 30

java 代码

       // 查询条件字段用 xxx_1、2、3 代替
      Criteria criteria = new Criteria();
        criteria.and("xxx_1").is("xxx");
        criteria.and("xxx_2").in(list);

        DateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

        criteria.andOperator(Criteria.where("xxx_3").gte(format.parse("2020-08-05 00:00:00")),
                Criteria.where("xxx_3").lte(format.parse("2021-08-05 23:59:59")));
        Query query = new Query(criteria);
        // 省略查询字段，不包含子文档
        query.fields(...);
        long startTime = System.currentTimeMillis();
        List<TestBO> queryList = mongoTemplate.find(query, TestBO.class);
        System.out.println("查询耗时：" + (System.currentTimeMillis() - startTime));
        System.out.println(queryList.size());

mongodb 脚本执行计划

查询条件字段 xxx_1、xxx_2、xxx_3 为联合索引
{
    plannerVersion: NumberInt("1"),
    namespace: "xxx.xxx",
    indexFilterSet: false,
    parsedQuery: {
        $and: [
            {
                xxx_1: {
                    $eq: "xxx"
                }
            },
            {
                xxx_3: {
                    $lte: ISODate("2021-07-29T23:59:59.000Z")
                }
            },
            {
                xxx_2: {
                    $in: [
                        ...
                    ]
                }
            }
        ]
    },
    queryHash: "C16527B7",
    planCacheKey: "4538160D",
    winningPlan: {
        stage: "PROJECTION_SIMPLE",
        transformBy: {
            /*查询字段*/
        },
        inputStage: {
            stage: "FETCH",
            inputStage: {
                stage: "IXSCAN",
                keyPattern: {
                    xxx_1: NumberInt("1"),
                    xxx_2: NumberInt("1"),
                    xxx_3: NumberInt("1")
                },
                indexName: "index_name",
                isMultiKey: false,
                multiKeyPaths: {
                    xxx_1: [ ],
                    xxx_2: [ ],
                    xxx_3: [ ]
                },
                isUnique: false,
                isSparse: false,
                isPartial: false,
                indexVersion: NumberInt("2"),
                direction: "forward",
                indexBounds: {
                    xxx_1: [
                        ...
                    ],
                    xxx_2: [
                        ...
                    ],
                    xxx_3: [
                        ...
                    ]
                }
            }
        }
    },
    rejectedPlans: [
        {
            stage: "PROJECTION_SIMPLE",
            transformBy: {
                /*查询字段*/
            },
            inputStage: {
                stage: "FETCH",
                filter: {
                    $and: [
                        {
                            xxx_1: {
                                $eq: "xxx"
                            }
                        },
                        {
                            xxx_3: {
                                $lte: ISODate("2021-07-29T23:59:59.000Z")
                            }
                        }
                    ]
                },
                inputStage: {
                    stage: "IXSCAN",
                    keyPattern: {
                        xxx_2: NumberInt("1")
                    },
                    indexName: "r_index",
                    isMultiKey: false,
                    multiKeyPaths: {
                        xxx_2: [ ]
                    },
                    isUnique: false,
                    isSparse: false,
                    isPartial: false,
                    indexVersion: NumberInt("2"),
                    direction: "forward",
                    indexBounds: {
                        xxx_2: [
                            ...
                        ]
                    }
                }
            }
        },
        {
            stage: "PROJECTION_SIMPLE",
            transformBy: {
                /*查询字段*/
            },
            inputStage: {
                stage: "FETCH",
                filter: {
                    $and: [
                        {
                            xxx_1: {
                                $eq: "xxx"
                            }
                        },
                        {
                            xxx_2: {
                                $in: [
                                    ...
                                ]
                            }
                        }
                    ]
                },
                inputStage: {
                    stage: "IXSCAN",
                    keyPattern: {
                        xxx_3: NumberInt("1")
                    },
                    indexName: "xxx_3",
                    isMultiKey: false,
                    multiKeyPaths: {
                        xxx_3: [ ]
                    },
                    isUnique: false,
                    isSparse: false,
                    isPartial: false,
                    indexVersion: NumberInt("2"),
                    direction: "forward",
                    indexBounds: {
                        xxx_3: [
                            "(true, new Date(1627603199000)]"
                        ]
                    }
                }
            }
        }
    ]
}

Answer 1

问题算是解决了，没有找到特别好的解决方式，最后用的是非映射对象的方式解决的，相同的查询条件，单个请求查询平均在200ms内，10到30个线程并发查询循环10次，查询时间差不多也在1000ms内响应

    Bson eq = Filters.eq("xxx_1", "xxx_1");
        Bson in = Filters.in("xxx_2", xxx_2);
        // 查询条件
        Bson and = Filters.and(eq, in);
        BasicDBObject fieldsObject = new BasicDBObject();
        // 指定返回字段
        fieldsObject.put("xxx_1", 1);
        ...
        FindIterable<Document> quota = mongoTemplate.getCollection("Test").find(and).projection(Document.parse(fieldsObject.toString()));
        List<TestBO> qlist = new ArrayList<>();
        for(Document document : quota) {
            TestBO q = new TestBO();
            // document.get("") ;
            // 通过逐个对象取值然后在赋值给具体的对象，试过用Fastjson的方式转换，但是效果更慢
            q.setxxx(document.getString("xxx"));
            qlist.add(q);
        }