m0_37373563
二_叔
采纳率20%
2021-03-05 16:08

从数据库查询20w数据太慢,怎么用多线程并行查询汇总list集合

有这么一个需求,数据库20w数据,如果直接从数据库一次性读取比较慢,怎么用多线程 分批次并行读取,然后将List<entity>结果汇总

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

3条回答

  • m0_37373563 二_叔 1月前
        @Autowired
        private RedisActiveTestsMapper redisActiveTestsMapper;
    
    public List<List<RedisActiveTests>> selectMult(RedisActiveTests redisActiveTests) {
            System.out.println("----程序开始运行----");
            Date date1 = new Date();
            int taskSize = 10;
            // 创建一个线程池
            ExecutorService pool = Executors.newFixedThreadPool(taskSize);
            // 创建多个有返回值的任务
            List<Future> list = new ArrayList<Future>();
            for (int i = 0; i < taskSize; i++) {
                Callable c = new ThredQuery(redisActiveTestsMapper);
                // 执行任务并获取Future对象
                Future f = pool.submit(c);
                // System.out.println(">>>" + f.get().toString());
                list.add(f);
            }
            // 关闭线程池
            pool.shutdown();
    
            // 获取所有并发任务的运行结果
            for (Future f : list) {
                // 从Future对象上获取任务的返回值,并输出到控制台
                try {
                    String s = f.get().toString();
                    System.out.println(">>>" +s );
                }catch (Exception e){
    
                }
    
            }
    
            Date date2 = new Date();
            System.out.println("----程序结束运行----,程序运行时间【" + (date2.getTime() - date1.getTime()) + "毫秒】");
            return new ArrayList<>();
    
    
    
    
    
    
    
    public class ThredQuery implements Callable<Object>
    {
        private RedisActiveTestsMapper redisActiveTestsMapper;
        public ThredQuery(RedisActiveTestsMapper redisActiveTestsMapper) {
            this.redisActiveTestsMapper = redisActiveTestsMapper;
        }
    
        @Override
        public Object call() throws Exception {
            int i=1;
            System.out.println(">>>" + i + "任务启动");
            Date dateTmp1 = new Date();
            Thread.sleep(1000);
            Date dateTmp2 = new Date();
            long time = dateTmp2.getTime() - dateTmp1.getTime();
            List<RedisActiveTests> selectnan = redisActiveTestsMapper.selectnan(new RedisActiveTests());
            System.out.println(">>>" + i + "任务终止");
            i++;
            return JSONArray.toJSONString(selectnan) + "任务返回运行结果,当前任务时间【" + time + "毫秒】";
        }
    }
    点赞 评论 复制链接分享
  • qq_27471405 小小鱼儿小小林 1月前

    20w相对来说数据量还是很少的,对查询条件加联合索引试试

    点赞 评论 复制链接分享
  • a1767028198 a1767028198 1月前

    countdownlatch就能让主线程等待,子线程全部执行完再执行主线程代码!或者jdk8的Future,也能让主线程等待

    点赞 评论 复制链接分享

为你推荐