ptvs debugger looks for visualstudio_py_debugger in user code

Oh, here's one thing that I ought to ask first. Does this call the handler on a background thread; and if so, does the thread get started by native code?

To make breakpoints work, we need to set up settrace hook separately on every thread before any Python code runs there. For normal threads, we do so by detouring some of the bits in _threading. But if native code starts the thread itself, there's no way for us to get notified and inject the hooks. I recall that was a problem with PyQt apps that used QThread.

The short answer is IDK!

The long answer is that there are likely at least Python and Managed threads. I'm not sure about Native threads. This is using Webview component from WinForms, and you probably know better than me what kind of threads get launched .

On Tue, Mar 28, 2017, 5:45 PM Pavel Minaev wrote:

Oh, here's one thing that I ought to ask first. Does this call the handler on a background thread; and if so, does the thread get started by native code?

To make breakpoints work, we need to set up settrace hook separately on every thread before any Python code runs there. For normal threads, we do so by detouring some of the bits in _threading. But if native code starts the thread itself, there's no way for us to get notified and inject the hooks. I recall that was a problem with PyQt apps that used QThread.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Microsoft/PTVS/issues/2328#issuecomment-289928226, or mute the thread https://github.com/notifications/unsubscribe-auth/AHgZ5RLedM6ppLA3Gwf9FiVBYmJOjamZks5rqY16gaJpZM4Mo7eA .

for non-Python threads is it possible to use settrace like in pydevd? I see that it is available in ptvsd, but PTVS remote debugging does not work with pythonnet (#552).

Both ptvsd and the regular debugger do settrace on the thread on which they initially execute. But there's no way to do just settrace, without initializing the debugger (which can only be done once). There's no reason why it can't be done - we just need to decouple thread creation from thread registration, and provide a public API for the latter, so that you could do something like import ptvsd; ptvsd.enable_debugging_on_thread().

I'll add a note to ~~#750~~#749 so we do that as part of the refactoring, and close this issue. I'll also add a note that we can probably hook threads automatically on Python 3.6 using the JIT API, but there isn't really any other way to do it without providing a public API and telling people to call it.

there are 2 issues in this thread: one for unreasonable call-stack and one for not hitting the breakpoints. So did you confirm that the second issue is caused by foreign threads? Is there a separate issue for unreasonable call-stack?

https://github.com/Microsoft/PTVS/issues/2328#issuecomment-289892778

Interesting. Do you have any info about why it broke there? Was there an exception?

At a guess, you've overridden sys.stdout or sys.stderr and something that we are writing to that is failing. But without any exception info it's hard to tell.

this popup for visualstudio_py_debugger only appears the first time. It is likely the same issue as here:

https://github.com/Microsoft/PTVS/issues/1083

Anyway I'm not able to debug with PTVS on_navigate method from pywebview, because PTVS skips it completely:

https://github.com/r0x0r/pywebview/blob/winforms-dpi-fix/webview/winforms.py#L83

jupyter does mess with stdout and stderr for the linked issue, but not pywebview.

I had to debug pywebview calls from C# side using pythonnet code. It is in the linked issue.

In both cases, the problem is that our debugger is hitting a problem with pythonnet code, or the pythonnet code is raising an exception and the first actual Python code on the stack is us. So I believe we're breaking correctly, but because of the extra pythonnet stuff on the stack, we aren't showing the right debugging information.

Some info about why the debugger is breaking there would be very helpful. Perhaps it is an exception? Or there is a breakpoint set somewhere else? We can "fix" trying to show code in our debugger, but it's very likely that we'll then just show a random breakpoint in code higher up the callstack, while the actual problem is deeper.

no exceptions at this method entry - the exception is much later

I tried stepping through with F11 and also continue with F5 up to breakpoint and both cases PTVS fails to debug.

Note that on_nagivate is called as an event:

https://github.com/r0x0r/pywebview/blob/winforms-dpi-fix/webview/winforms.py#L68

One more note - I'm debugging using attach to process with Python engine. I have not tried other debugging options.

I think we need to split this one into two different issues, since there are really two bugs here - one is that you're seeing that call stack in the screenshot, and another is that your breakpoints in the method aren't hit. It may be the same root cause, but I'm not so sure about that.

Can you give a simple standalone (or minimal) repro for this, so that I can debug?

Hi Pavel,

Here are instructions:

pip install git+https://github.com/r0x0r/pywebview-dpi-fix

This should also install latest pythonnet from PyPI.

import webview
webview.create_window("It works, Jim!", "http://www.flowrl.com")

The breakpoint locations are linked above.