CI test matrix failures

Question

CI test matrix has some know failures with certain vscode variants.

Related

• Hang at getmac.exe message (update: this error isn't the cause of the hang, just happens to be the last message printed)
• https://github.com/microsoft/vscode-remote-release/issues/2719
• https://github.com/aws/aws-toolkit-vscode/pull/1193#issuecomment-659490004
• https://github.com/microsoft/vscode/issues/102428
• Fixed the getmac.exe error (by installing getmac.exe in CI), but doesn't actually fix our builds: https://github.com/aws/aws-toolkit-vscode/issues/1207#issuecomment-666609262
• https://github.com/aws/aws-toolkit-vscode/issues/1201 ?
• https://github.com/aws/aws-toolkit-vscode/issues/1071 ?

Exit code -4058

• Exit code -4058 indicates a node.js crash/segfault. This is strictly a node.js bug, but we need to either work around it or root-cause it and report it upstream.
• observed in CI jobs:
• Windows Unit Tests [insiders]

Starting tests: C:\codebuild\tmp\output\src630281967\src\github.com\aws\aws-toolkit-vscode-staging\dist\src\test\index.js
Test error: Error: spawn C:\codebuild\tmp\output\src630281967\src\github.com\aws\aws-toolkit-vscode-staging\.vscode-test\vscode-insiders\Code - Insiders.exe ENOENT
Exit code:   -4058

Exit code 3221226356

• commit https://github.com/aws/aws-toolkit-vscode/commit/e691811d87f00050450f4973181578961e5e9467 (2020-08-17)
• Windows CI job failure Exit code: 3221226356 (heap corruption)
  

  
      Cannot read property 'toJSON' of null: TypeError: Cannot read property 'toJSON' of null
          at file:///C:/codebuild/tmp/output/src125341715/src/github.com/aws/aws-toolkit-vscode/.vscode-test/vscode-1.48.0/resources/app/out/vs/workbench/workbench.desktop.main.js:3930:297
          at Array.map (<anonymous>)
          at file:///C:/codebuild/tmp/output/src125341715/src/github.com/aws/aws-toolkit-vscode/.vscode-test/vscode-1.48.0/resources/app/out/vs/workbench/workbench.desktop.main.js:3930:288
        TypescriptLambdaHandlerSearch
          √ finds export declared functions in Typescript code
          √ ignores class declarations in Typescript code
          √ ignores interface declarations in Typescript code
          √ finds module.exports declared functions in javascript code
          √ ignores class declarations in javascript code
        1158 passing (16s)
      [main 2020-08-18T00:53:33.744Z] Lifecycle#kill()
      [main 2020-08-18T00:53:33.747Z] Lifecycle#window.on('closed') - window ID 1
      [main 2020-08-18T00:53:33.747Z] Lifecycle#onWillShutdown.fire()
      [main 2020-08-18T00:53:33.749Z] Lifecycle#app.on(window-all-closed)
      [main 2020-08-18T00:53:33.750Z] Lifecycle#app.on(before-quit)
      [main 2020-08-18T00:53:33.750Z] Lifecycle#onBeforeShutdown.fire()
      [main 2020-08-18T00:53:33.757Z] onBeforeShutdown {
        lastActiveWindow: undefined,
        lastPluginDevelopmentHostWindow: undefined,
        openedWindows: []
      }
      [main 2020-08-18T00:53:33.757Z] Lifecycle#app.on(will-quit)
      [main 2020-08-18T00:53:33.765Z] [storage :memory:] close()
      Exit code:   3221226356
    </anonymous>

~~Error: TypeError~~ Fixed

• observed in CI jobs:
• Windows Unit Tests [insiders] https://github.com/aws/aws-toolkit-vscode/pull/1206:
• Fixed: pulling newer VSCode versions (instead of pinning to 1.46) fixes the ERR_INVALID_ARG_TYPE error.

[main 2020-07-23T16:03:32.425Z] update#setState idle
Error: TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
    at h._doHandleExtensionTests (c:\codebuild\tmp\output\src251412895\src\github.com\aws\aws-toolkit-vscode\.vscode-test\vscode-insiders\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:876:885)
    at processTicksAndRejections (internal/process/task_queues.js:94:5)
native-keymap: Could not create ITfThreadMgr.
Exit code:   1
...
npm ERR! aws-toolkit-vscode.12.0-SNAPSHOT test: `npm run testCompile && ts-node ./test-scripts/test.ts`

该提问来源于开源项目：aws/aws-toolkit-vscode

Answer 1

Fixed by https://github.com/microsoft/vscode/issues/102642 :

ERR_INVALID_ARG_TYPE when testing extension on 1.47.1

seems to be related to the recent CVE fix https://portal.msrc.microsoft.com/en-us/security-guidance/advisory/CVE-2020-1416 where the node path resolution introduced a dependency on process.env.HOMEDRIVE and process.env.HOMEPATH to additionally check for preventing module loads from the Users directory.

https://github.com/microsoft/vscode/pull/102966 claims to fix the issue but it's not clear if this made it into our CI yet.