weixin_39681724
2020-12-27 04:05 阅读 1

Coverage Threshold parameter

An optional Threshold parameter could be given e.g. --threshold=90 and when the merged threshold is >90% the script could return 0, otherwise return 1.

Would be great for CI.

该提问来源于开源项目:gcovr/gcovr

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

6条回答 默认 最新

  • weixin_39681724 weixin_39681724 2020-12-27 04:05

    eg.

     python
    parser.add_option(
        "--threshold",
        help="Coverage % threshold value (range:0-100), below this retval:1, otherwise:0, default 90%",
        action="store",
        dest="cov_threshold",
        default=90
    )
    

    at the end:

     python
    print("\nSummary:");
    cov_global = print_summary(covdata);
    
    # retval based on threshold parameter
    
    if (options.cov_threshold is not None):
      thresh = int(options.cov_threshold)
      if (thresh <= 100) and (thresh > 0):
        if cov_global < thresh:
          print("failed, not enough coverage (got:%d, expected min:%d)" % (cov_global, thresh))
          sys.exit(1); # failed, not enough coverage
      else:
        print("Please give value 0-100 for options.threshold", thresh);
        sys.exit(1); # everything was fine
      print("\nMinimum covarage was:%d%%\n" % thresh);
    
    print("\nResult stored in index.html");
    print("\nFinished running");
    sys.exit(0); # everything was fine
    
    点赞 评论 复制链接分享
  • weixin_39608613 weixin_39608613 2020-12-27 04:05

    It may be difficult to figure out if the non-zero exit status was caused by another error.

    You could use the --print-summary option and let your CI system evaluate the result.

    点赞 评论 复制链接分享
  • weixin_39801879 weixin_39801879 2020-12-27 04:05

    , I guess this can be closed c.f. 3cf2cd469f2cddcb982c856d9e45b6b5347e488e

    点赞 评论 复制链接分享
  • weixin_39891272 weixin_39891272 2020-12-27 04:05

    Such a parameter (--fail-under-line and --fail-under-branch) was implemented in #173.

    What this issue suggests in addition:

    • validating that the provided coverage percentage is 0 < threshold <= 100.
    • reporting when the coverage target was missed.

    Both are good ideas, though gcovr should only create such output if it's in verbose mode, or if it is printing a textual report (default format or summary report).

    I'm therefore leaving this issue open for now.

    点赞 评论 复制链接分享
  • weixin_39801879 weixin_39801879 2020-12-27 04:05

    Reporting: what shall be done ?

    I think that it shall output a message on stderr when the coverage target was missed since we're returning a non-zero exit code. e.g.

    python
    print("failed, not enough line coverage (got:%d, expected min:%d)" % (cov_line_global, line_thresh))
    

    I'm not sure about adding messages for success but I stay open to the idea. In that case: Other information output shall be printed on stdout in verbose mode. e.g.

    python
    print("Minimum line coverage was:%d%%\n" % thresh);
    

    If thinking about adding it in textual report/summary report then what about HTML output ? Any other ideas ?

    点赞 评论 复制链接分享
  • weixin_39762441 weixin_39762441 2020-12-27 04:05

    For me the exit code is for run ok or not ok. To check the result in the CI you can analyse the XML output.

    点赞 评论 复制链接分享

相关推荐