weixin_39875842
2020-11-28 18:24 阅读 11

Compilation to WASM?

Compiling the whole Rustc to WASM is a pretty big undertaking for many reasons.

However, Miri doesn't need an actual codegen and many other parts of the whole Rustc, so I wonder how realistic it would be to compile it and the pieces it depends on to WASM instead? Are there any obvious blockers?

Mostly opening this to gauge interest and estimate complexity, as I believe there is an interest in running Rust directly in the browser on playground-like websites.

P.S. Despite what I said in the first sentence, this was actually done for Clang a while ago - https://tbfleming.github.io/cib/ - which includes LLVM compiled to WASM that, in turn, generates more WASM dynamically during runtime. In theory, it should be possible to do the same for Rust, especially since they share LLVM, but for now having just an interpreter could already be an interesting starting goal.

该提问来源于开源项目:rust-lang/miri

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

65条回答 默认 最新

  • weixin_39793794 weixin_39793794 2020-11-28 18:24

    I think a first step would be to create a new codegen backend which doesn't actually do any codegen and just dumps the metadata. That way we should be able to build a rustc which doesn't depend on llvm or other C code.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I think a first step would be to create a new codegen backend which doesn't actually do any codegen and just dumps the metadata.

    I had created one in the past but it bitrotted and it was unused, so I removed it in https://github.com/rust-lang/rust/pull/58847. If you copy https://github.com/bjorn3/rustc_codegen_cranelift/blob/11d816c/src/lib.rs#L182-L190 into provide and provide_extern it should work. (Also change target_features_whitelist to contain the same as cg_llvm https://github.com/rust-lang/rust/blob/2d401fb4dc89eaef5b8f31330636094f9c26b4c4/src/librustc_codegen_llvm/llvm_util.rs#L249, otherwise stdsimd wont compile.)

    Another thing necessary is replacing the dlopen for loading the codegen backend with a regular extern crate. Eg replace the match at https://github.com/rust-lang/rust/blob/08bfe16129b0621bc90184f8704523d4929695ef/src/librustc_interface/util.rs#L271 with _ => || Box::new(MetadataOnlyCodegenBackend) or however you call the backend.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I want to do the same for https://github.com/bjorn3/rustc_codegen_cranelift/, but I want it to pass the rustc test suite first and cranelift doesnt support wasm output yet.

    点赞 评论 复制链接分享
  • weixin_39517797 weixin_39517797 2020-11-28 18:24

    Intriguing. :) I should add one warning though: Miri isn't a fast interpreter. It's really slow. So I don't think it is actually a good environment to use to run code, I see it as more useful for debugging and testing.

    But, don't let me stop you! I just felt I should give you a fair warning. And if ideas like this leak to people making Miri lightning fast while maintaining all the UB checking, I'll be even more happier. :D

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Miri isn't a fast interpreter. It's really slow. So I don't think it is actually a good environment to use to run code, I see it as more useful for debugging and testing.

    That's understandable, but I think it's ought to be good enough for typical playground snippets :)

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    I think a first step would be to create a new codegen backend which doesn't actually do any codegen and just dumps the metadata. That way we should be able to build a rustc which doesn't depend on llvm or other C code.

    I guess that's one way, although I was wondering if Miri actually needs the main rustc crate or maybe it could be possible to depend only on some of the finer-grained rustc_* crates and avoid including codegen altogether?

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    The codegen backend is necessary for rustc_driver to work. Using a dummy codegen backend (<100LOC mostly copyable from the code I mentioned in https://github.com/rust-lang/miri/issues/722#issuecomment-489019303) is a lot easier than duplicating all the things rustc_driver does (>>1000LOC).

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Fair enough.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I am currently trying to compile rustc for wasm (https://github.com/bjorn3/rust/tree/compile_rustc_for_wasm), but I am hitting a compiler bug: https://github.com/rust-lang/rust/issues/60540.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    I've rebased your branch onto master, updated deps and fixed cfg's from target_env to target_os - you can check it out at https://github.com/RReverser/rust/tree/compile_rustc_for_wasm.

    Eventually it compiled successfully, but then ran into the same runtime validation issue with invalid code generated by Rust.

    However, I recompiled in release mode and then it passed validation!

    That got me thinking it should work now, but running the generated file with wasmtime or wasmer now seems to just hang. Some infinite loop somewhere perhaps?

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Oh... maybe it's just been taking so long (especially the compilation part). I've tried wasmer with --backend singlepass instead now, and it has actually worked!

    
    $ ./wasmer run target/wasm32-unknown-wasi/release/rustc_binary.wasm --backend singlepass
    Usage: rustc [OPTIONS] INPUT
    
    Options:
        -h, --help          Display this message
            --cfg SPEC      Configure the compilation environment
        -L [KIND=]PATH      Add a directory to the library search path. The
                            optional KIND can be one of dependency, crate, native,
                            framework or all (the default).
        -l [KIND=]NAME      Link the generated crate(s) to the specified native
                            library NAME. The optional KIND can be one of
                            static, dylib, or framework. If omitted, dylib is
                            assumed.
            --crate-type [bin|lib|rlib|dylib|cdylib|staticlib|proc-macro]
                            Comma separated list of types of crates
                            for the compiler to emit
            --crate-name NAME
                            Specify the name of the crate being built
            --edition 2015|2018
                            Specify which edition of the compiler to use when
                            compiling code.
            --emit [asm|llvm-bc|llvm-ir|obj|metadata|link|dep-info|mir]
                            Comma separated list of types of output for the
                            compiler to emit
            --print [crate-name|file-names|sysroot|cfg|target-list|target-cpus|target-features|relocation-models|code-models|tls-models|target-spec-json|native-static-libs]
                            Comma separated list of compiler information to print
                            on stdout
        -g                  Equivalent to -C debuginfo=2
        -O                  Equivalent to -C opt-level=2
        -o FILENAME         Write output to <filename>
            --out-dir DIR   Write output to compiler-chosen filename in <dir>
            --explain OPT   Provide a detailed explanation of an error message
            --test          Build a test harness
            --target TARGET Target triple for which the code is compiled
        -W, --warn OPT      Set lint warnings
        -A, --allow OPT     Set lint allowed
        -D, --deny OPT      Set lint denied
        -F, --forbid OPT    Set lint forbidden
            --cap-lints LEVEL
                            Set the most restrictive lint level. More restrictive
                            lints are capped at this level
        -C, --codegen OPT[=VALUE]
                            Set a codegen option
        -V, --version       Print version info and exit
        -v, --verbose       Use verbose output
    
    Additional help:
        -C help             Print codegen options
        -W help             Print 'lint' options and default settings
        -Z help             Print unstable compiler options
        --help -v           Print the full set of options rustc accepts
    </dir></filename>
    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Oh... maybe it's just been taking so long (especially the compilation part).

    Yes, it takes several minutes to compile it using wasmtime with cranelift as backend.

    However, I recompiled in release mode and then it passed validation!

    🎉 🎉 🎉

    and it has actually worked!

    I tried actually compiling something, but it errors with:

    
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: StringError("operation not supported on wasm yet") }', src/libcore/result.rs:999:5
    note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
    

    I am currently trying to figure out were it errors.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Places needing patching:

    • librustc_interface/util.rs: spawn_thread_pool must not spawn a thread.
    • librustc/session/filesearch.rs: get_or_default_sysroot must not be called, as it needs std::env::current_exe. I am now passing an explicit --sysroot.
    • librustc/session/mod.rs: build_session_ local working_dir is created using std::env::current_dir.

    Edit: pushed bjorn3/rust (based on 's branch).

    Now it errors at

    
    thread 'main' panicked at 'unknown codegen backend llvm', src/librustc_interface/util.rs:277:18
    

    Which is expected, as I had to remove the codegen backend dynamic loader. Will try to get https://github.com/bjorn3/rustc_codegen_cranelift to work with it.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Places needing patching

    FWIW previously (before even filing this issue) I tried compiling rustc with Emscripten instead, which should, in theory, reduce number of these places to patch, as it supports a bit more than WASI does. Haven't gotten too far though, because I tried to build completely unpatched rustc and there were few things that still didn't compile and probably needed similar fixes as in your branch.

    Which is expected, as I had to remove the codegen backend dynamic loader. Will try to get bjorn3/rustc_codegen_cranelift to work with it.

    I thought the plan was to build it without any codegen, just with miri? Or do you want to build an actual full rustc?

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I thought the plan was to build it without any codegen, just with miri? Or do you want to build an actual full rustc?

    I want them both. :) I currently have rustc_codegen_cranelift hooked up, but rustc gives an error before rustc_codegen_cranelift can do actual codegen: can't find crate for `std`. Supporting miri will need that error to be fixed too.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    can't find crate for std

    Yeah for that I think you'll need to do the proper build (via x.py build) to build all components. I haven't had much luck with that yet due to failures in other crates which probably also need to be patched similarly to rustc itself.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Yeah for that I think you'll need to do the proper build (via x.py build) to build all components.

    Seems like it doesn't even reach the rustc version check for the libraries. I added --sysroot $(rustc --print sysroot) and disabled the rustc version check, but it still gives the same error.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Switching from wasmer to wasmtime fixed it. It even got to the beginning of codegen.

    Edit: filled https://github.com/wasmerio/wasmer/issues/434.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I am currently working on making miri compile for wasi, which this issue was actually about.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    It seems to trap while calling the ecx.run. :(

    
    error while processing main module ../../target/wasm32-unknown-wasi/release/rustc_binary.wasm: Instantiation error: Trap occurred while invoking start function: wasm trap at 0x2a881eb82
    

    I pushed the wip stuff to my branch.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Left a comment on your MIRI commit on your branch.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    But also, I'm not sure why rustc is now depending on miri... shouldn't it be the other way around? (like in non-WASI version)

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I did that to be able to prevent having to recompile every rustc crate, which is slow and to prevent having to copy all files in the dir layout rustc wants a sysroot to be.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    I'm not sure I understand what you're saying... neither should be affected by which crate you compile as an entry point.

    I've changed my local copy of Rust & MIRI to do just that, and got miri.wasm successfully, but yeah, also hitting some trap.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I meant that I had already compiled all crates in rust/target. When switching to miri as crate root, I would have to recompile all crates into miri/target. By keeping rustc-binary as crate root, I could reuse rust/target as target dir.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    When switching to miri as crate root, I would have to recompile all crates into miri/target

    Not when you use miri in src/tools/miri path - then it reuses same workspace.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Yes, but I didn't think about that possibility. I just used the miri I had already cloned outside my rust clone.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Yeah, in conjunction with your comment on that commit it makes sense now. Shouldn't be hard to switch.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Things being patched (how patched / how to prevent in the future):

    • [x] measureme (rust-lang/measureme#43)
    • [ ] jobserver (alexcrichton/jobserver-rs#13)
    • [ ] backtrace (remove dep or stub out on wasm, alexcrichton/backtrace-rs#178)
    • [x] flate2 (enable rust_backend feature and disable default features, https://github.com/alexcrichton/flate2-rs/pull/194)
    • [ ] std::env::current_dir (use . instead)
    • [ ] std::path::Path::canonicalize (just don't)
    • [ ] rustc_interface::util::get_resident (remove)
    • [ ] memmap (use Vec<u8> on WASI https://github.com/danburkert/memmap-rs/issues/88#issuecomment-492320205)
    • [ ] threads (run in one thread)
    • [ ] several other things
    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    What's wrong with flate2? It worked well in WASM for me (in other projects). See https://github.com/alexcrichton/flate2-rs/issues/161.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    It didn't compile. I believe I will have to enable the rust_backend feature and disable the default features.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Btw, when using Cranelift backend instead, the trap has extra detail:

    thread 'main' panicked at 'wasm trap occured: memory out-of-bounds access', src/bin/wasmer.rs:359:51

    Not that it's very helpful without a stacktrace.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    That makes me suspect https://github.com/rust-lang/rust/blob/efa3c27f0ff21960b9309f8036dbf3e7416b9e52/src/librustc_mir/interpret/memory.rs#L743-L768. It is the only block of unsafe code in rustc_mir::interpret.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I tried compiling wasi rustc with debuginfo to get a backtrace with lldb, but wasmtime panics while processing the debuginfo (https://github.com/CraneStation/wasmtime/issues/144)

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Wow, this seems to be a decent stress test that uncovers all sorts of bugs in WASM tooling. Thanks for working on it!

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Yeah, I expected rustc itself to be the hardest part of compiling it for WASM, not the tools. :)

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    https://github.com/rust-lang/rust/pull/60831 got merged recently, haven't been able to confirm that the miscompilation in debug mode is fixed though.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    measureme fix has been merged too, worth updating the list above?

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Got some logs for the miri SIGSEGV:

    
    [...]
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] (_0.2: std::marker::PhantomData) = move _21
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] StorageDead(_21)
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] StorageDead(_20)
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] StorageDead(_19)
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] StorageDead(_8)
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] StorageDead(_2)
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] return
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::eval_context] LEAVING(7) core::slice::<impl>::iter
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::eval_context] CONTINUING(6) std::vec::Vec::<u8>::extend_from_slice
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] // bb1
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] StorageDead(_5)
    [2019-05-15T16:50:10Z INFO  rustc_mir::interpret::step] _0 = const std::vec::SpecExtend::spec_extend(move _3, move _4) -> bb2
    Process 22977 stopped
    * thread #1: tid = 22977, 0x00007ffdf4f28dd2 JIT(0x7ffe097f4010), name = 'wasmtime', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
        frame #0: 0x00007ffdf4f28dd2 JIT(0x7ffe097f4010)
    ->  0x7ffdf4f28dd2: movslq (%rcx,%rax,4), %rax
        0x7ffdf4f28dd6: addq   %rax, %rcx
        0x7ffdf4f28dd9: jmpq   *%rcx
        0x7ffdf4f28ddc: movl   0x4fc(%rsp), %eax
    (lldb) bt
    * thread #1: tid = 22977, 0x00007ffdf4f28dd2 JIT(0x7ffe097f4010), name = 'wasmtime', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
      * frame #0: 0x00007ffdf4f28dd2 JIT(0x7ffe097f4010)
        frame #1: 0x00007ffdf4f17ed0 JIT(0x7ffe097f4010)
        frame #2: 0x00007ffdf6c23e05 JIT(0x7ffe097f4010)
        frame #3: 0x00007ffdf6c21d8b JIT(0x7ffe097f4010)
        frame #4: 0x00007ffdf4e6880d JIT(0x7ffe097f4010)
        frame #5: 0x00007ffdf6b6842a JIT(0x7ffe097f4010)
        frame #6: 0x00007ffdf6acbb0a JIT(0x7ffe097f4010)
        frame #7: 0x00007ffdf6c3fdaa JIT(0x7ffe097f4010)
        frame #8: 0x00007ffdf4708a2e JIT(0x7ffe097f4010)
        frame #9: 0x00007ffdf6bf9418 JIT(0x7ffe097f4010)
        frame #10: 0x00007ffdf6bf615d JIT(0x7ffe097f4010)
        frame #11: 0x00007ffdf592b819 JIT(0x7ffe097f4010)
        frame #12: 0x00007ffdf67368cb JIT(0x7ffe097f4010)
        frame #13: 0x00007ffdf44e2bbd JIT(0x7ffe097f4010)
        frame #14: 0x00007ffdf5351dff JIT(0x7ffe097f4010)
        frame #15: 0x00007ffdf459e5c7 JIT(0x7ffe097f4010)
        frame #16: 0x00007ffdf5344b6b JIT(0x7ffe097f4010)
        frame #17: 0x00007ffdf42f33a3 JIT(0x7ffe097f4010)
        frame #18: 0x00007ffdee912992 JIT(0x7ffe097f4010)
        frame #19: 0x00007ffdeebdcc83 JIT(0x7ffe097f4010)
        frame #20: 0x00007ffdedbf895c JIT(0x7ffe097f4010)
        frame #21: 0x00007ffdedbfa6d8 JIT(0x7ffe097f4010)
        frame #22: 0x00007ffdeef72e1a JIT(0x7ffe097f4010)
        frame #23: 0x00007ffdedbfa4bc JIT(0x7ffe097f4010)
        frame #24: 0x00007ffdedc039fe JIT(0x7ffe097f4010)
        frame #25: 0x00007ffdedf2fd72 JIT(0x7ffe097f4010)
        frame #26: 0x00007ffdedf64859 JIT(0x7ffe097f4010)
        frame #27: 0x00007ffdedf1d207 JIT(0x7ffe097f4010)
        frame #28: 0x00007ffdee0e194e JIT(0x7ffe097f4010)
        frame #29: 0x00007ffdedbf3f23 JIT(0x7ffe097f4010)
        frame #30: 0x00007ffdedbd620d JIT(0x7ffe097f4010)
        frame #31: 0x00007ffdf7f9ffdc JIT(0x7ffe097f4010) at lib.rs:85
        frame #32: 0x00007ffdedbd38ce JIT(0x7ffe097f4010)
        frame #33: 0x00007ffdedc07f92 JIT(0x7ffe097f4010)
        frame #34: 0x00007ffdedbf6111 JIT(0x7ffe097f4010)
        frame #35: 0x00007ffdf7f8e8ff JIT(0x7ffe097f4010) at rt.rs:49
        frame #36: 0x00007ffdf7f98b38 JIT(0x7ffe097f4010) at rt.rs:49
        frame #37: 0x00007ffdf7f9ffdc JIT(0x7ffe097f4010) at lib.rs:85
        frame #38: 0x00007ffdf7f9b954 JIT(0x7ffe097f4010) at panicking.rs:272
        frame #39: 0x00007ffdedc084e5 JIT(0x7ffe097f4010)
        frame #40: 0x00007ffdedbc486b JIT(0x7ffe097f4010)
        frame #41: 0x0000555555676071 wasmtime`wasmtime_call + 433
        frame #42: 0x0000555555670823 wasmtime`wasmtime_runtime::instance::Instance::invoke_function::h1366e5c600361498 + 147
        frame #43: 0x0000555555673274 wasmtime`wasmtime_runtime::instance::InstanceHandle::new::hf352a471df9e241a + 8644
        frame #44: 0x000055555560d1c8 wasmtime`wasmtime_jit::instantiate::instantiate::hbbe129f222e6a659 + 712
    </u8></impl>

    It is not really helpful, as https://github.com/CraneStation/wasmtime/issues/144 forces me to compile it without debuginfo.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    What about wasmer? Does it use debug information at least for the stacktrace?

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Wasmer seems to have some problems with WASI compliance: https://github.com/wasmerio/wasmer/issues/434 it used to not be able to read dirs and now it can't read /home/bjorn/.cache/miri/HOST/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd.so.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    You might want to pass a more precise nested path in --dir - at least that did trick for me before in a similar issue.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Passing --dir $MIRI_SYSROOT/lib/rustlib/x86_64-unknown-linux-gnu/lib as extra argument or replacing the --dir $MIRI_SYSROOT doesn't work.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    I see, interesting. Btw, do you think libstd.so is anyway going to help here? I suppose for execution in Wasm we'd need to compile libstd to Wasm as well...

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I suppose for execution in Wasm we'd need to compile libstd to Wasm as well...

    Actually not, we only need to read the metadata rustc stores in it, so by telling rustc to "compile" the executable to be interpreted with the target libstd was compiled for should just work:tm:.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    I see. Does that mean we'll still need to ship the whole libstd.so to browsers and such or only the metadata parts?

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    It actually first tries to load libstd.rlib. This is just an ar archive, so it is possible to remove all object files from it and just keep the rust.metadata.bin file it contains in the archive.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    At least the combination of https://github.com/rust-lang/rust/commit/709120b32146e74c19ecb53fd58b2b108fa9096a and the wasmer fix in https://github.com/wasmerio/wasmer/pull/446 seems to fix the miri SIGSEGV.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Using wasmtime with that commit still gives SIGSEGV.

    点赞 评论 复制链接分享
  • weixin_39928480 weixin_39928480 2020-11-28 18:24

    What about wasmer? Does it use debug information at least for the stacktrace?

    To get debug information from wasmer run it like:

    
    cargo run --release --features debug -- ...
    

    I'll add it to my todo list to document this somewhere (and also improve the signal:noise ratio while I'm at it)

    Wasmer's WASI implementation is being prioritized based on need right now (until it becomes more stable and we define some spec tests), so please don't hesitate to file issues on Wasmer or ping us if there's anything we can do to help!

    I'm super excited about your work here -- thanks for doing it!

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    To get debug information from wasmer run it like:

    
    cargo run --release --features debug -- ...
    

    Good to know!

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    Filled https://github.com/CraneStation/wasmtime/issues/150 for the wasmtime crash.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Found out that flate2 incorrectly tries to compile C library when targeting WASI, sent a PR at https://github.com/alexcrichton/flate2-rs/pull/194.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    cargo run --release --features debug -- ...

    Why is it optional btw? Could it be included in public prebuilt binaries?

    点赞 评论 复制链接分享
  • weixin_39928480 weixin_39928480 2020-11-28 18:24

    cargo run --release --features debug -- ...

    Why is it optional btw? Could it be included in public prebuilt binaries?

    The current debug level is extremely verbose and can be a bottleneck during execution. If we enabled dynamic toggling in the current system, we'd have to investigate the performance impacts. My intuition is that they'll prevent optimizations, but we might be able to do something clever with generics to keep the debug-free case the same.

    More concretely, I opened a PR (wasmerio/wasmer#447) to move some of the noisiest messages to their own flag (trace) and we can look in to having a less noisy level that could be enabled by default.

    What types of debug info would you be interested in seeing?

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I think this is a misinterpretation of debuginfo. I thought you meant the debuginfo which a debugger uses for backtraces and showing the correct line for the current instruction, while you meant log messages.

    点赞 评论 复制链接分享
  • weixin_39928480 weixin_39928480 2020-11-28 18:24

    That's correct! I didn't know wasmtime had that feature -- that's very cool! We've had some discussions around integrating with debuggers but don't have anything yet.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Integration with debuggers might be a big feature and require much more time, but resolving backtracks to function names shouldn't be too hard and would already significantly improve experience when debugging issues.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    The wasmtime crash seems to happen during codegen_fulfill_obligation.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Hi, did you have any further luck on this?

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I didn't continue with this. I lost interest in continuing this. Feel free to use changes.

    Nb: I suspect the wasmtime crash to be a stackoverflow, but I havent confirmed this.

    点赞 评论 复制链接分享
  • weixin_39875842 weixin_39875842 2020-11-28 18:24

    Ah, too bad. Thanks for all the work though.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I have rebased my branch as compile_rustc_wasm3.

    点赞 评论 复制链接分享
  • weixin_39694016 weixin_39694016 2020-11-28 18:24

    what's the status of this? The issue you note on the current state of the compile_rustc_wasm3 branch has since been fixed upstream.

    点赞 评论 复制链接分享
  • weixin_39762464 weixin_39762464 2020-11-28 18:24

    I haven't done anything more on this since my last comment. You can rebase my branch if you want to experiment with this. I doubt that I will be accepted upstream with the patches it needed. I needed to disable stuff like threads. Upstreaming the patches for platform specific stuff is more likely to succeed.

    点赞 评论 复制链接分享

相关推荐