weixin_39518002
weixin_39518002
2020-12-27 11:57

build translate fail under gcc 5.3.1

when i build translate with cuda 9.0, i meet below issues, did anyone meet the same issue?

-- The C compiler identification is GNU 5.3.1 -- The CXX compiler identification is GNU 5.3.1 -- Check for working C compiler: /opt/rh/devtoolset-4/root/usr/bin/cc -- Check for working C compiler: /opt/rh/devtoolset-4/root/usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /opt/rh/devtoolset-4/root/usr/bin/c++ -- Check for working CXX compiler: /opt/rh/devtoolset-4/root/usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
-- Caffe2: Found gflags with new-style gflags target. -- Caffe2: Cannot find glog automatically. Using legacy find. -- Found glog: /usr/include
-- Caffe2: Found glog (include: /usr/include, library: /usr/lib64/libglog.so) -- Caffe2: Found protobuf with new-style protobuf targets. -- Caffe2: Protobuf version 3.5.0 -- Found CUDA: /usr/local/cuda (found suitable version "9.0", minimum required is "7.0") -- Caffe2: CUDA detected: 9.0 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 9.0 -- Found CUDNN: /usr/local/cuda/include
-- Found cuDNN: v7.1.2 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so) -- Autodetected CUDA architecture(s): 6.0 -- Added CUDA NVCC flags for: -gencode;arch=compute_60,code=sm_60 -- Summary: -- CMake version : 3.11.1 -- CMake command : /home/wliao2/local/bin/cmake -- System name : Linux -- C++ compiler : /opt/rh/devtoolset-4/root/usr/bin/c++ -- C++ compiler version : 5.3.1 -- CXX flags : -std=c++11 -O2 -fPIC -Wno-narrowing -- Caffe2 version : 0.8.2 -- Caffe2 include path : /home/wliao2/anaconda3/envs/translate/include -- Have CUDA : -- Configuring done -- Generating done -- Build files have been written to: /home/wliao2/translate/pytorch_translate/cpp/build (translate) [xxx]$ make 2>&1 | tee MAKE_OUT Scanning dependencies of target translation_decoder [ 16%] Building CXX object CMakeFiles/translation_decoder.dir/Decoder.cpp.o /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:38:18: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_string(encoder_model, "", "Encoder model path"); ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:39:18: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_string(decoder_step_model, "", "Decoder step model path"); ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:40:18: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_string(source_vocab_path, "", "Source vocab file"); ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:41:18: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_string(target_vocab_path, "", "Target vocab file"); ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:43:15: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_int(beam_size, -1, "Beam size"); ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:44:18: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_double( ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:50:15: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_int( ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:57:16: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_bool( ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:61:16: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_bool( ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:65:16: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_bool( ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:69:18: error: expected constructor, destructor, or type conversion before ‘(’ token C10_DEFINE_double( ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp: In function ‘int main(int, char)’: /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:78:7: error: ‘FLAGS_source_vocab_path’ is not a member of ‘c10’ if (c10::FLAGS_source_vocab_path.empty() || ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:79:7: error: ‘FLAGS_target_vocab_path’ is not a member of ‘c10’ c10::FLAGS_target_vocab_path.empty() || ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:80:7: error: ‘FLAGS_encoder_model’ is not a member of ‘c10’ c10::FLAGS_encoder_model.empty() || ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:81:7: error: ‘FLAGS_decoder_step_model’ is not a member of ‘c10’ c10::FLAGS_decoder_step_model.empty()) { ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:85:38: error: ‘FLAGS_source_vocab_path’ is not a member of ‘c10’ << "(source_vocab_path='" << c10::FLAGS_source_vocab_path ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:86:40: error: ‘FLAGS_target_vocab_path’ is not a member of ‘c10’ << "', target_vocab_path='" << c10::FLAGS_target_vocab_path ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:87:36: error: ‘FLAGS_encoder_model’ is not a member of ‘c10’ << "', encoder_model='" << c10::FLAGS_encoder_model ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:88:41: error: ‘FLAGS_decoder_step_model’ is not a member of ‘c10’ << "', decoder_step_model='" << c10::FLAGS_decoder_step_model << "')"; ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:92:41: error: ‘FLAGS_source_vocab_path’ is not a member of ‘c10’ std::make_shared<:dictionary>(c10::FLAGS_source_vocab_path); ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:94:41: error: ‘FLAGS_target_vocab_path’ is not a member of ‘c10’ std::make_shared<:dictionary>(c10::FLAGS_target_vocab_path); ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:96:7: error: ‘FLAGS_beam_size’ is not a member of ‘c10’ c10::FLAGS_beam_size, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:97:7: error: ‘FLAGS_max_out_seq_len_mult’ is not a member of ‘c10’ c10::FLAGS_max_out_seq_len_mult, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:98:7: error: ‘FLAGS_max_out_seq_len_bias’ is not a member of ‘c10’ c10::FLAGS_max_out_seq_len_bias, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:101:7: error: ‘FLAGS_encoder_model’ is not a member of ‘c10’ c10::FLAGS_encoder_model, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:102:7: error: ‘FLAGS_decoder_step_model’ is not a member of ‘c10’ c10::FLAGS_decoder_step_model, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:103:7: error: ‘FLAGS_reverse_source’ is not a member of ‘c10’ c10::FLAGS_reverse_source, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:104:7: error: ‘FLAGS_stop_at_eos’ is not a member of ‘c10’ c10::FLAGS_stop_at_eos, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:105:7: error: ‘FLAGS_append_eos_to_source’ is not a member of ‘c10’ c10::FLAGS_append_eos_to_source, ^ /home/wliao2/translate/pytorch_translate/cpp/Decoder.cpp:106:7: error: ‘FLAGS_length_penalty’ is not a member of ‘c10’ c10::FLAGS_length_penalty); ^ make[2]: </:dictionary></:dictionary> [CMakeFiles/translation_decoder.dir/Decoder.cpp.o] Error 1 make[1]: [CMakeFiles/translation_decoder.dir/all] Error 2 make: * [all] Error 2

该提问来源于开源项目:pytorch/translate

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

6条回答

  • weixin_39518002 weixin_39518002 3月前

    it seems not to be the gcc version issue, it should be the caffe2/core/flags.h changed, in the translate, it include c10 library which use C10_DEFINE_xxx, while in pythorch-caffe2-cuda9.0-cudnn7.0 package, it still using CAFFE2_DEFINE_xxx

    点赞 评论 复制链接分享
  • weixin_39782545 weixin_39782545 3月前

    I am also having this issue on Ubuntu 18.04 with cuda 9. Does anyone have a more up to date docker image they could add to dockerhub?

    点赞 评论 复制链接分享
  • weixin_39518002 weixin_39518002 3月前

    i did not find one, i git clone the latest pytorch code and build with setup.py, then you need change the makefile to set the caffe2_dir to you build to make the translate to use your build caffe2. but you still will meet some interface issue between translate and latest pytorch, you need fix it one by one.

    点赞 评论 复制链接分享
  • weixin_39782545 weixin_39782545 3月前

    Thanks!

    Do you know if it is possible to train the model without this stuff then export it later? Also do you know if the transformer model will export to onnx?

    点赞 评论 复制链接分享
  • weixin_39842475 weixin_39842475 3月前

    -fire, it should be possible to export the transformer. Once you have trained a transformer model, you can use a similar command to https://github.com/pytorch/translate/blob/master/pytorch_translate/examples/export_iwslt14.sh

    点赞 评论 复制链接分享
  • weixin_39545895 weixin_39545895 3月前

    -fire, it should be possible to export the transformer. Once you have trained a transformer model, you can use a similar command to https://github.com/pytorch/translate/blob/master/pytorch_translate/examples/export_iwslt14.sh

    which caffe2 version should I use? latest version has too much API changes.

    点赞 评论 复制链接分享