Behat的行为测试覆盖率

Question

I would like to approximate test coverage of behaviour using Behat in PHP.

Obviously, unlike code coverage, behavioural coverage is not precisely quantifiable, however, we can know what behaviours we are planning to accommodate for from the start. For example, we might know that a user can 1) buy a product, 2) comment on the product. For this we would want to write two BDD tests.

To begin with, we could create placeholders and use the standard @wip tag to mark each test as work in progress.

I have two related questions:

Is it possible to have Behat report on the total number of scenarios, and the total number of scenarios with a given tag (i.e. @wip)?

Is it possible for Behat to know that count, even if you use the filter "~@wip" to ignore those tests?

Answer 1

You can use hooks to get information about scenarios, including their tags, however you can only do this in @beforeFeature or @beforeSuite.

Unfortunately this means it will not report on scenarios with tags that are being filtered out.

Here is my initial solution.

class Context extends BehatContext {

    protected static $scenarioCount = 0;

    protected static $scenarioCountWip = 0;

    /**
     * Count scenarios and WIP
     * @BeforeScenario
     */
    public function countScenario(Behat\Behat\Event\ScenarioEvent $scenario)
    {
        self::$scenarioCount++;
        if(in_array('wip', $scenario->getScenario()->getTags())) {
            self::$scenarioCountWip++;
        }
    }

    /**
     * @AfterSuite
     */
    public static function coverageReport()
    {
        echo 'Scenarios:      '.self::$scenarioCount.PHP_EOL;
        echo ' - in progress: '.self::$scenarioCountWip.PHP_EOL;
        echo ' - coverage:    '.(100/self::$scenarioCount)*(self::$scenarioCount-self::$scenarioCountWip).PHP_EOL;
        echo PHP_EOL;
    }

}

The feature file:

Feature: Test Count

  @wip
  Scenario: Work in progress

  @someTag
  Scenario: Scenarion with tag

  Scenario: Final scenario

The result when running all tests:

Feature: Test Count

  @wip
  Scenario: Work in progress

  @someTag
  Scenario: Scenarion with tag

  Scenario: Final scenario

Scenarios:      3
 - in progress: 1
 - coverage:    66.666666666667

3 scenarios (3 passed)
No steps

This the result whit @wip filtered out:

Feature: Test Count

  @someTag
  Scenario: Scenarion with tag

  Scenario: Final scenario

Scenarios:      2
 - in progress: 0
 - coverage:    100

2 scenarios (2 passed)
No steps

I'll leave the question open in the hope someone finds a better solution.