weixin_39725154
weixin_39725154
2020-12-03 03:40

Implement mutation switching 👽🔀

Intro

Mutation switching is a technique that can be used to improve the performance of mutation testing. It is used in Stryker.NET as well as Stryker4s. We believe this has a lot of benefits for JavaScript/TypeScript as well.

In short, with mutation switching all mutants will be placed into the source code at once. Only one mutant can be active at the same time. Here is a small example:

js
/* 1*/ // original
/* 2*/ function add(a, b) {
/* 3*/   return a + b;
/* 4*/ }
/* 5*/ 
/* 6*/ // Might be instrumented to:
/* 7*/ function add(a, b) {
/* 8*/   if (global.activeMutant === 0) {
/* 9*/      // block mutation, empty function
/*10*/   } else {
/*11*/     return global.activeMutant === 1? a - b: a+b;
/*12*/   }
/*13*/ }

Potential performance improvements

Although JavaScript is not a compiled language, there is still huge performance improvements to be had.

  • Less I/O. This can be a big performance overhead, since not only the mutated file has to be cleared from node's require cache, but the entire object graph that is dependent on that file, which in practice means clearing the entire cache (100's of files for running Stryker on Stryker). For karma it means a reload between test runs with sometimes MB's of files to reload.
  • One-time transpiling. A lot of projects depend on some kind of transpiler/bundler, also to run the tests. Think of TypeScript, Webpack, Vue CLI, Angular CLI, Jest-ts, ts-node, etc. They will all have to transpile only once, instead of once per mutant.
  • Hot reload. Test runners won't have to clear their test suite between tests (there is a caveat here regarding static code, see hurdles).
  • Test coverage analysis. Since we're instrumenting the code, it's pretty much free to add mutation coverage instrumentation in the code. For example, line 11 could be mutated to: return global.activeMutant === 1 ? a - b : (global__coverMutant__(1), a + b);. The __coverMutant__ here is an example of how we can count coverage analysis. Mutants that aren't covered, don't need to be tested. If we add the context of which test is running at that point in time, we can only run the tests that cover a particular mutant. We currently have a very complex coverage analysis in place, which we can totally get rid of once we have this new test coverage analysis system in place. It will work for a lot more use cases, no matter which transpiler or bundler or even minified you use. However, we will need to change all test runners to report this new mutation coverage statistic. There is also complication here for static code, see hurdles.

Mutation switch instrumenter PoC.

I've created a Proof of Concept of mutation switching. It works for a couple of our mutations and I'm confident we can add them all. It has a lot of benefits:

  • It supports both JS and TS with one AST (thanks to babeljs).
  • It supports JS, TS, Vue and HTML file formats (thanks to angular-html-parser).
  • Support mutant coverage instrumentation.

Hurdles

There are some hurdles to overcome.

  • We need to make sure that mutated code can be transpiled by the supported transpilers out there. Webpack, Babel, TypeScript, and friends will have to produce JavaScript to run. This is not trivial since we are definitely introducing type errors into the source code.
  • We will still need to type check each mutant. A.t.m. this is the job of the transpiler. We will need to change that, as the transpiler will have to produce code no matter what. I would suggest adding a new plugin type for that "TypeChecker". We might want to not make it a public API at first, to save time and maintenance.
  • Test runners will have to support at least running the tests. Reporting the mutation coverage and hot reload are a nice to have, although Stryker might actually slow down without it (due to larger file sizes).
  • There's also the concept of static code. Code that is executed once when imported and then never again. If we mutate in that part, we would need to detect it and turn off hot reload for those scenarios. Detecting static code can be done by reporting mutant coverage. Mutants that were covered before a test runs can be considered "static".
  • Our current way of test filtering in Mocha will no longer work in combination with hot reload.

Rethinking Transpilers

We currently have a Transpiler API. It is responsible for transpiling the code for the initial test run, as well as transpiling each mutant. Stryker takes care to load the transpiler and keep it alive, so the transpiler can use in-memory --watch functionality if it can.

Examples of transpilers: * Webpack transpiler * Babeltranspiler * TypeScript transpiler

Specific transpiler plugins are no longer needed when using mutation switching. We only need to transpiler once (with all mutants in the code), so there is no need to painstakingly transpile each mutant. The amount of maintenance is simply not worth the effort to keep this transpiler API, if only to run it once.

Instead, we should allow a user to configure a "build command" (name pending). It can be tsc -b for example, or webpack --configFile webpack.conf.js. This build command is then executed inside the sandbox before the initial test run starts. This sandbox is reused for the mutant test runs.

When copying files to the sandbox, we might need to change them slightly. Take this tsconfig file for example:

json
{
  "extends": "../../tsconfig.settings.json",
  "references": [ 
    { "path": "../api/tsconfig.src.json" },
    { "path": "../util/tsconfig.src.json" }
  ]
}

Simply picking this file up and dropping it in the sandbox won't work. Assuming the sandbox is copied 2 directories deep, we need to preprocess it to be:

json
{
  "extends": "../../../../tsconfig.settings.json",
  "references": [ 
    { "path": "../../../api/tsconfig.src.json" },
    { "path": "../../../util/tsconfig.src.json" }
  ]
}

The goal of a transpiler

Migration scenario

I think a small step-by-step migration is out of the question, as approximately half of our code will have to be rewritten, or completely scrapped. Our public API will also need to be changed in a couple of places too.

If we want to keep feature parity with the current features, we will at least need support for transpiling using babel, webpack, typescript, jest, ts-node, karma-webpack, mocha-webpack, vue cli, angular CLI and react scripts and test runner support for karma, jasmine, mocha, and jest (I think we can drop WCT).

A high-level overview of the new way of working of a mutation test run: 1. Read config 1. Instrument the code 1. Parse 1. Generate mutants 1. Place mutants 1. Print the code in memory 1. Transpile the code (once) 1. Fill one sandbox, run tests once for the control run (initial test run) 1. Check mutants using the configured "check" plugins 1. Test the remaining mutants in a sandbox

Investigation progress

There's a lot of stuff we need to figure out. We'll have to investigate how all features are possible. Note: there is a lot of stuff "nice to have" here, so we shouldn't block implementation.

  • [x] Source code instrumentation
    • [x] Source code mutation
    • [x] Mutation coverage instrumentation
    • [x] Support for vue, ts and JS
  • [x] TypeCheckers
    • [x] TypeScript. we should be able to converts parts of the current TypeScript transpiler to a type checker
  • [x] Transpilers prefix every ts/js file with `// -nocheck
    • [x] TypeScript we can use this work around to transpile using typescript
    • [x] Babel doesn't perform type checking
    • [x] Angular CLI. as long as we don't mutate decorators, we should be able to prefix every ts/js file with // -nocheck (works from TS3.7)
    • [x] Web pack: Turn off type checking for typescript related plugins. prefix every ts/js file with `// -nocheck
    • [x] VueCLI _ Turn off type checking for typescript related plugins_
  • [x] Test runners
    • [x] Command: How do we activate a specific mutant when using the command test runner? will be best effort using env variable
    • [x] Mocha
    • [x] Report mutation coverage just reporting the global object
    • 🤷‍♂️ [nice to have] Hot reload (will be implemented in https://github.com/mochajs/mocha/pull/4234)
    • 🤷‍♂️ [nice to have as long as we don't have hot reload] Support test filtering (investigated in https://github.com/mochajs/mocha/issues/4183), although maybe we can use grep
    • [x] Jest
    • [x] Turn off type checking for typescript code can not be done by simply removing ts-jest, however, we can use the typescript workaround
    • 🤷‍♂️ [nice to have, we don't support coverage a.t.m.] Report mutation coverage (we can use workaround proposed in https://github.com/facebook/jest/issues/7421)
    • 🤷‍♂️ [nice to have, we don't support this a.t.m.] Run individual tests
    • 🤷‍♂️ [nice to have] Hot reload
    • [x] Karma
    • [x] Turn off type checking for typescript code in karma plugins we can use the typescript workaround
    • [x] Report mutation coverage can be done with custom middleware and framework? See https://github.com/stryker-mutator/stryker/issues/1514#issuecomment-625971578
    • [x] Run individual tests (can be done with the current setup, but won't support hot reloading)
    • 🤷‍♂️ [nice to have] Hot reload (probably needs custom framework plugin and middleware plugin)
    • [x] Jasmine node
    • [x] Report mutation coverage just reporting the global object
    • [x] Run individual tests (current setup will work)
    • 🤷‍♂️ [nice to have] Hot reload

Implementation plan

Implementation should be done in steps. To make it easier on ourselves, we should allow us to break parts of Stryker in the meantime. I suggest therefore to work in a feature branch. We'll be implementing mutation switching on that branch.

In the meantime, our master branch will keep getting security patches, but we want to keep the functional changes to a minimum.

Question: Do we want to keep Dependabot during this time? I think it is better to point Dependabot to the feature branch instead of master.

The plan:

Preparation

  • [x] Add e2e tests for missing use cases. Be sure to add code that will result in type errors when mutation switching is implemented.
  • [x] Karma with karma-webpack and typescript (we can copy a small sample from mutation-testing-elements) (#2183)
  • [x] Jest with ts (#2177)
  • [x] React with jest and tsx (#2184 )
  • [x] Vue-cli with mocha
  • [x] Vue-cli with jest?
  • [x] mocha using ts-node
  • 🤷‍♂️ A setup with Aurelia ?
  • [x] Remove html-reporter package (not used anymore)
  • [x] Remove integration tests for api package (chore to maintain, without any actual benefit)
  • [x] Setup github workflow to publish to `` for long-lived feature branches.
  • [x] Create a long-lived feature branch epic/mutation-switching
  • [x] Set Dependabot to update on that branch instead of master

Implementation

  • [x] I think we should add the new source code instrumentation in a new package: -mutator/instrumenter (name pending). That way we keep it out of the Stryker core and it will all be a bit more manageable. We might choose to allow a new "instrumenter" plugin later, for now, I would suggest to let Stryker have a full-fledged dependency on it!
  • [x] Add instrumenter package - [x] Add Parsers - [x] Add MutantPlacers - [x] Add Mutators - [x] Add Ast instrumenters - [x] Add Printers
  • [x] Use instrumenter from -mutator/core
  • [x] Remove old coverage analysis code and the dependency on instanbul
  • [x] The "Mutator" api will disappear. That means:
  • [x] We should remove the mutator from directory from -mutator/api
  • [x] We should remove the -mutator/javascript-mutator
  • [x] We should remove the -mutator/vue-mutator
  • [x] We should remove the mutator from -mutator/typescript. Since we will create a new transpiler, we can remove this entire package.
  • ~~[x] Add a new transpiler called in -mutator/typescript-transpiler that uses this work around. It should ignore all type errors and work for both project references and plain projects.~~ no longer needed, since we're scrapping the entire transpiler api
  • [x] Add a new "checker" plugin type (name pending).
  • [x] Add new api. Proposal: * init(files: []): Promise<void>
    It should be fed the primary files and to check for errors in the initial test run (no errors) * check(mutant: Mutant): Promise<MutantStatus>
    It should be fed one mutant at a time that is than type checked, or filtered (see #1980 )
  • [x] Implement -mutator/typescript-checker (name pending)
  • [x] Update the "test-runner" API
  • [x] The test runner will be responsible to activate a specific mutant. We should also add a flag to say weather coverage should be reported, so it is not reported when testing mutants (only after initial test run).
  • [x] Implement in -mutator/karma-runner
  • [x] Implement in -mutator/jasmine-runner
  • [x] Implement in -mutator/mocha-runner
  • [x] Implement in -mutator/jest-runner
  • [x] Implement in -mutator/wct-runner
  • [x] Implement in command runner
  • [x] Re-imagine transpilers
  • [x] Remove `-mutator/webpack-transpiler
  • [x] Remove -mutator/babel-transpiler
  • [x] Add preprocessors to core (private api), add a typescript preprocessor that adds // -nocheck and updates tsconfig.json files.
  • [x] Add a build command option to core
  • [x] Karma runner: report mutation coverage
  • [x] Karma runner: support activating a mutant
  • [x] Mocha runner: report mutation coverage. Please remove the OptionsEditor.
  • [x] Mocha runner: support activating a mutant
  • [x] Jasmine runner: report mutation coverage.
  • [x] Jasmine runner: support activating a mutant
  • [x] Core
  • [x] Implement new dry-run process (initial test run)
  • [x] Implement new mutation run process
  • [x] Implement preprocessors
    • [x] tsconfig files
    • [x] Add // -nocheck at the top of each js/ts file.

I'll be keeping this post up to date with our current progress.

该提问来源于开源项目:stryker-mutator/stryker

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

29条回答

  • weixin_39725154 weixin_39725154 5月前

    As for the karma mutation coverage reporting. This seems pretty trivial with Karma's plugin system. A small PoC:

    js
    // stryker-karma-plugin.js
    module.exports = {
      'framework:stryker': ['factory', strykerFramework],
      'middleware:stryker': ['value', strykerMiddleware],
      'middleware:body-parser': ['value', require('body-parser').json()]
    }
    
    function strykerFramework (config) {
      config.files.unshift({
        pattern: require.resolve('./adapter.js'), 
        included: true, 
        watched: false,
        served: true
      });
    }
    
    strykerFramework.inject = ['config'];
    
    function strykerMiddleware(request, _response, next) {
      if (request.method === 'POST' && request.url === '/stryker-info') {
        console.log('Received from stryker:')
        console.log(request.body);
      } else {
        next();
      }
    }
    
    // adapter.js
    const originalComplete = window.__karma__.complete.bind(window.__karma__);
    window.__karma__.complete = (...args) => {
      fetch('/stryker-info', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          test: true
        })
      }).then(() => {
        originalComplete(...args);
      });
    }
    

    Activate with:

    js
    // karma.conf.js
    
    module.exports = function(config){
      config.set({
        files: ['src/*.js', 'test/*.js'],
        frameworks: [
          'jasmine',
          'stryker' // Activate the "stryker" framework
        ],
        plugins: [
          'karma-*',
          require.resolve('./stryker-karma-plugin') // Add plugin
        ],
        singleRun: true,
        browsers: [
          'Chrome'
        ],
        middleware: [
          'body-parser', // body-parser used to parse json
          'stryker' // Activate the "stryker" middleware
        ]
      })
    }
    
    
    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    Yow! We've started work on this in epic/mutation-switching.

    Now first, let me get a coffee ☕

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    About the transpilers plugin. I was wondering if we still need the complexity we now have. Couldn't we just copy files to the sandbox and run the transpiler there? We could allow people to configure an arbitrary command to be run before we're running files.

    The files that are copied to the sandbox should still be transformed somehow. For example, we should add // -nocheck to each typescript and javascript file. We should also transform tsconfig.json files to make relative paths work. For example, "extends": "../../tsconfig.settings.json" becomes "extends": "../../../../tsconfig.settings.json". Maybe we can use this transform plugin and add a command transpiler, which runs a simple command once.

    -vrijswijk thoughts?

    点赞 评论 复制链接分享
  • weixin_39974409 weixin_39974409 5月前

    If TypeScript projects can be supported as easily as they can now, I'm all for it :)

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    This would also be the case for the webpack and babel transpiler. All 3 can be replaced by a simple command from the command line.

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    I've updated the issue with our current idea for transpilers. The basic idea is to remove them in favor of letting the user configure a buildCommand (name pending)

    点赞 评论 复制链接分享
  • weixin_39760434 weixin_39760434 5月前

    I was thinking, if just before merging mutation switching to master could we make performance tests, we could use them later on blog post or somewhere (like if you are going to make a talk, you could mention it or sth)

    点赞 评论 复制链接分享
  • weixin_39855658 weixin_39855658 5月前

    This sounds very promising. Can you already make a rough estimation for when this feature will be implemented? I'm currently doing a case study on the practical usage of mutation testing with an existing (relatively big) TS project and try to find out if and how we can apply mutation testing in a reasonable and efficient way. Sadly my initial test with Stryker took nearly 2 days (!) to complete (running on a laptop with i7-6600U) so it would be very interesting to see how mutation switching (eventually along with coverage analysis for jest) may impact the total execution time in this case and if that would make it more feasible. I would also be willing to help and contribute to Stryker (e.g. this feature) if you need help and I've got enough time.

    点赞 评论 复制链接分享
  • weixin_39927993 weixin_39927993 5月前

    it would be very interesting to see how mutation switching (eventually along with coverage analysis for jest) may impact the total execution time in this case

    You should be able to install Stryker from GitHub and give it a try (maybe easier said than done).

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    For anyone that wants to help. We're using the 4.0 milestone for this.

    https://github.com/stryker-mutator/stryker/milestone/9

    点赞 评论 复制链接分享
  • weixin_39804603 weixin_39804603 5月前

    Wow, looking forward for this ❤️

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    If anyone wants to give it a try, we've released a beta 🤗 https://stryker-mutator.io/blog/2020-07-13/announcing-stryker-4-beta-mutation-switching

    I'm hoping to release v4 in a few weeks 🤐

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    Sadly my initial test with Stryker took nearly 2 days (!) to complete (running on a laptop with i7-6600U) so it would be very interesting to see how mutation switching (eventually along with coverage analysis for jest) may impact the total execution time in this case and if that would make it more feasible

    Wow! I think you are my personal hero 🦸‍♂️.

    Mutation switching by itself will not have a significant impact on Jest projects unfortunately. You might have some performance improvement because Jest would be able to cache build results, but definitely not an order of magnitude. However, it does open up the way for coverage analysis in jest. Hot reload should theoretically also be possible with Jest, but not with their current public api alone. We'll be focussing on jest next, after we've released v4.

    点赞 评论 复制链接分享
  • weixin_39760434 weixin_39760434 5月前

    I think I'll have to check it on my BigMath project soon :D

    点赞 评论 复制链接分享
  • weixin_39927993 weixin_39927993 5月前

    I tried this with Prettier on an 8 vCPU cloud instance, see https://github.com/brodybits/prettier/pull/42 but encountered some issues:

    • memory issues with a couple large source files
    • timeout error message on initial test run, even with --timeoutMS 5000000 command line option, then went quiet

    I think https://github.com/brodybits/prettier/pull/42 should be enough to reproduce my issues, but please let me know if there is anything else I can do to help isolate them.

    点赞 评论 复制链接分享
  • weixin_39855658 weixin_39855658 5月前

    As far as I know --timoutMS is not used for the initial test run but for the actual mutation testing. Currently (or at least in the old stable version) the maximum time for the inital test run is hard coded as 5 minutes. Also see my issue: https://github.com/stryker-mutator/stryker/issues/2302

    点赞 评论 复制链接分享
  • weixin_39760434 weixin_39760434 5月前

    there could be some memory leak 🤔

    点赞 评论 复制链接分享
  • weixin_39927993 weixin_39927993 5月前

    For Prettier, the initial test run went much faster when switching from Jest to the command runner. But for some reason I am getting many test failures from the initial test run. I have a feeling that something is going very wrong with what the mutation switching does to the JavaScript and will try to investigate it another day.

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    I complement your bravery, running Stryker on such huge open source projects. But yes, why shouldn't we?

    Unsupported type ConditionalExpression

    See #2400

    TypeError: Property id of DeclareModule expected node

    See #2399 (this one should not be that hard, string literal mutator shouldn't mutate DeclareModule identifiers.

    For Prettier, the initial test run went much faster when switching from Jest to the command runner. But for some reason I am getting many test failures from the initial test run.

    See #2401

    🎉

    点赞 评论 复制链接分享
  • weixin_39927993 weixin_39927993 5月前

    Thanks . I think the other major thing with Prettier is the resource usage and long prep time I had encountered on a large cloud instance, as I listed in https://github.com/stryker-mutator/stryker/issues/2401#issuecomment-676436431. I did clean up my attempt in https://github.com/brodybits/prettier/pull/42, in case it may help at all.

    I can try to test new beta releases from time to time, otherwise leaving it in your hands for now. Yeah what a monster mutating Prettier!

    P.S. I did minimize my long comment as "duplicate", since it is now covered by separate issues.

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    thanks for giving the beta a spin!

    There are fewer mutations

    This is probably because we don't generate mutants anymore that we couldn't place (i.e. babel doesn't allow invalid syntax). A good example is property keys:

    js
    const foo = {
      'bar': baz
    }
    

    In this example, the string literal bar was mutated in 3.0 but no longer in 4, because inserting a ternary operator there isn't allowed there. We could place the mutant with a switch case for example, but I decided against it, because we don't mutate regular property names, so why mutate them when they appear as literals?

    I believe this is a known issue with Stryker that has now been solved.

    Awesome! 🍺

    I am now running 15 instances, where I used to run 16.

    Yes, during the refactoring of the concurrency (for the checker api), I've decided to reserve 1 core for the stryker main process by default. You can override it with --concurrency. Note: the max concurrency is no longer capt by Stryker, so don't use a too high concurrency.

    The biggest thing for me though is that it's actually a lot slower.

    Hmm interesting. Let's take a closer look at your use case and the possible performance improvements with mutation switching:

    • Less I/O. This improvement might not have a big impacting on your use case. 🤷‍♂️
    • One-time transpiling. You don't use transpiling ❌
    • Hot reload. This would be a big performance improvement I think, but alas, this isn't implemented yet. ❌
    • Test coverage analysis. This is something Stryker 3 already supported with istanbul for your use case, correct? So no performance gain here. ❌

    I was planning to pickup hot reload after the 4.0 release, but with this performance impact in mind, maybe we should already implement it in the beta? What do you think?

    Just to be sure, you are running with --coverageAnalysis perTest correct?

    点赞 评论 复制链接分享
  • weixin_39979516 weixin_39979516 5月前

    I decided against it, because we don't mutate regular property names, so why mutate them when they appear as literals?

    That was actually a minor annoyance for me. A good decision as far as I'm concerned

    I've decided to reserve 1 core for the stryker main process by default. You can override it with --concurrency

    I'll give it a run with --concurrency as a performance check. I'll also keep an eye on my CPU usage while it's running. My gut tells me that the reserved core makes sense, but I'm curious anyway 😊

    I was planning to pickup hot reload after the 4.0 release, but with this performance impact in mind, maybe we should already implement it in the beta? What do you think?

    The way you phrased it makes me think that hot reload is not a breaking change, right?

    The thing is that it's worse for me, but a lot better for others in this thread. In fact, Stryker suddenly became feasible for bigger projects.

    Thinking about this makes me curious about what the impact is on a small project with transpiling. Something like a React/Angular/Vue/... front-end written in Typescript. If someone knows about an open-source project like this, I'll be glad to run it in 3 and 4.beta.

    In fact, I've been wondering about how Stryker scales in general. The relation between mutation count and runtime feels exponential-ish. Would you be up for including some benchmarks in the repo? I'm not sure how to set it up yet, but I have some ideas.

    If the only projects that are worse off are those like mine, then I think its a no-brainer to release asap. When It's not as clear, and it rarely is, you can also release 4 with the note in the changelog that it comes with a performance hit for smaller projects.

    Just to be sure, you are running with --coverageAnalysis perTest correct?

    Yes, I am.

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    This would require a major change in how mutation is done. I've discussed this face to face with and the easiest thing will be to build it next to the current mutation implementation and let people opt-in with an --expermimentalMutationSwitching command line flag (or something like it). That way we can actually release it before we remove the old implementation while keeping it all backward compatible.

    We'll be starting development on this next week probably. We'll update this issue along the way.

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    I've been working on a PoC for a couple of weeks now and it seems to be working with some success! https://github.com/nicojs/mutation-switch-instrumenter#mutation-switch-instrumenter

    I'll be using that PoC to see if we can run tests on Angular/Vue/React projects with Karma/Mocha/Jest, etc.

    点赞 评论 复制链接分享
  • weixin_39760434 weixin_39760434 5月前

    Sounds great ! If you want, you can use my BigMath library to check the performance of this solution :)

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    Not so fast there, cowboy 🤠. I've updated the original issue text to reflect the current progress in research and thoughts. As you can see, still some work to be done before we can start implementing.

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    I've started to work on an implementation plan (see original issue text)

    点赞 评论 复制链接分享
  • weixin_39687814 weixin_39687814 5月前

    Hi !

    I'm impressed with your Mutation switching new approach. Looking forward for it!

    By the way I was thinking about another way of applying mutations in JS/TS - using debugger API. On the first glance the flow can be following: 1. set the breakpoint on the required line of source code 2. run tests 3. when debugger paused on the breakpoint - capture variable values and mutate it 4. resume execution and check tests result

    Seems like such approach has it's own pros and cons. Pros: - no instrumentation step: easier integration with frameworks

    Cons: - less mutations: we can only change variable values and function return value

    What do you think about it?

    点赞 评论 复制链接分享
  • weixin_39725154 weixin_39725154 5月前

    That's interesting. It does seem seriously limited. I'm also not sure if it is truly easier to implement since you'll need to inspect the test runner process, calculate the breakpoint position back to the original source code, and than understand the code enough to know which value you want to debug.

    If you want to talk more about it, let's do that on slack: https://app.slack.com/client/TTHUR6NNP

    点赞 评论 复制链接分享

相关推荐