问题描述
问题描述:
在ubuntu 20.04 amd64架构的主机上生成的镜像,无法在 ubuntu 20.04 arm64架构的 NVIDIA Xavier NX上使用。
(此问题通过数次重新生成镜像解决)通过docker save -o xxx.tar xxx:v1
命令存储的docker不能再另一台设备上运行,load
命令报错,import
命令成功但不能运行。
此镜像在原主机盒另一台amd64架构的主机上运行正常。
操作流程
以下是我的操作流程,
Dockerfile:
FROM python:3.9.13
ADD . /ppo_eval
WORKDIR /ppo_eval
RUN pip install -r requirements.txt
ENV PATH=$PATH:/ppo_eval/ppo_eval
ENV PYTHONPATH=/ppo_eval/ppo_eval
CMD ["python", "./ppo_eval/evaluate/evaluate_async.py"]
在主机上的存储命令:
sudo docker save -o ppo_docker.tar ppo_docker:v1
存成了ppo_docker.tar
,之后使用sudo cp -b
拷贝到USB,再通过USB拷贝到另一台设备。
主要报错
拷贝好后,首先尝试使用load
命令加载:
sudo docker load -i ppo_docker.tar
报错为:
open /var/lib/docker/tmp/docker-import-3631489028/blobs/json: no such file or directory
检查docker images
,确实没装上
修改过程
重新生成后,可以运行。
user@ubuntu:~$ sudo docker load -i ppo_docker.tar
[sudo] password for user:
655ed1b7a428: Loading layer 129.2MB/129.2MB
a89d1d47b5a1: Loading layer 11.31MB/11.31MB
9ff6e4d46744: Loading layer 19.31MB/19.31MB
8d51c618126f: Loading layer 156.5MB/156.5MB
54acb5a6fa0b: Loading layer 538.1MB/538.1MB
100796cdf3b1: Loading layer 19.04MB/19.04MB
874448b54658: Loading layer 52.58MB/52.58MB
a3f7911df18a: Loading layer 4.608kB/4.608kB
6153a9b1f580: Loading layer 10.78MB/10.78MB
5f283f210830: Loading layer 45.95MB/45.95MB
5f70bf18a086: Loading layer 1.024kB/1.024kB
c657a4b8de7e: Loading layer 7.804GB/7.804GB
Loaded image: ppo_docker:v1
之后使用 run 命令:
user@ubuntu:~$ sudo docker run ppo_docker:v1
[sudo] password for user:
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
exec /usr/local/bin/python: exec format error
参考网上的方法,改成用 --platform
user@ubuntu:~$ sudo docker run ppo_docker:v1 --platform linux/arm64/v8
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "--platform": executable file not found in $PATH: unknown.
ERRO[0002] error waiting for container: context canceled
解决
b121-legion@b121legion-LEGION-REN9000K-34IRZ:~/Projects/ppo_docker$ sudo docker build -t ppo_docker:arm64 . --platform linux/arm64/v8
[sudo] password for b121-legion:
[+] Building 163.1s (8/8) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 253B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9.13 4.2s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 7.60kB 0.0s
=> [1/4] FROM docker.io/library/python:3.9.13@sha256:a1f8d596b9400f5af381e6e8482bf97424bf0c1e8f41361a95497c8 39.6s
=> => resolve docker.io/library/python:3.9.13@sha256:a1f8d596b9400f5af381e6e8482bf97424bf0c1e8f41361a95497c89 0.0s
=> => sha256:a1f8d596b9400f5af381e6e8482bf97424bf0c1e8f41361a95497c89d5514dfc 2.35kB / 2.35kB 0.0s
=> => sha256:580b0402c5a830f8703e9d50ae9c3ae56719f275cb54a48252a0e3e7ced3c02a 8.53kB / 8.53kB 0.0s
=> => sha256:a1d98e120b809269e56de468d2b91569789c521011e4d9b1806e43fd04462de2 4.94MB / 4.94MB 2.7s
=> => sha256:6229556c9db680ee1ab01c9b7401f19a68707b0e25d91652a0f84ee2a1e03d39 2.22kB / 2.22kB 0.0s
=> => sha256:cd84405c8b9e7a8c3d580c2148d25120dd697ea61e1cb55a62f33e67988b7043 53.69MB / 53.69MB 33.5s
=> => sha256:7cb6be5911b40ca548e48c10b09cb2312f1698b4c84f09711c69389a94b1a8be 10.66MB / 10.66MB 2.2s
=> => sha256:db608c3c3ce3568556faf63acbc2069811afe242faf8992ccf0ab1986ee38e4f 54.68MB / 54.68MB 8.3s
=> => sha256:ef10f752bfb9bb5888d549687e75a51ad78e1523ee87958e8b464cbd4dee4bb2 189.71MB / 189.71MB 18.4s
=> => sha256:d0a39d2c0f04e93b41d969037f123877e0dae7c88550a4d5a1920ce5eb6ee980 6.16MB / 6.16MB 9.6s
=> => sha256:e54876cf7264cfcd60d64f825fa68dc25a42f317f467f49fe2b1e2f765380346 17.92MB / 17.92MB 13.4s
=> => sha256:e3fe5a493bc59c7963374657dc82d59c20b7cb78316b6d22ab560eed6477058e 232B / 232B 14.9s
=> => sha256:352c3727478dc1246befaa35e89267ace6babe5b32cf3917089bbc184c90d24d 2.88MB / 2.88MB 17.5s
=> => extracting sha256:cd84405c8b9e7a8c3d580c2148d25120dd697ea61e1cb55a62f33e67988b7043 0.9s
=> => extracting sha256:a1d98e120b809269e56de468d2b91569789c521011e4d9b1806e43fd04462de2 0.1s
=> => extracting sha256:7cb6be5911b40ca548e48c10b09cb2312f1698b4c84f09711c69389a94b1a8be 0.1s
=> => extracting sha256:db608c3c3ce3568556faf63acbc2069811afe242faf8992ccf0ab1986ee38e4f 0.8s
=> => extracting sha256:ef10f752bfb9bb5888d549687e75a51ad78e1523ee87958e8b464cbd4dee4bb2 2.5s
=> => extracting sha256:d0a39d2c0f04e93b41d969037f123877e0dae7c88550a4d5a1920ce5eb6ee980 0.1s
=> => extracting sha256:e54876cf7264cfcd60d64f825fa68dc25a42f317f467f49fe2b1e2f765380346 0.2s
=> => extracting sha256:e3fe5a493bc59c7963374657dc82d59c20b7cb78316b6d22ab560eed6477058e 0.0s
=> => extracting sha256:352c3727478dc1246befaa35e89267ace6babe5b32cf3917089bbc184c90d24d 0.2s
=> [2/4] ADD . /ppo_eval 0.3s
=> [3/4] WORKDIR /ppo_eval 0.1s
=> ERROR [4/4] RUN pip install -r requirements.txt 118.7s
------
> [4/4] RUN pip install -r requirements.txt:
8.446 Collecting appdirs==1.4.4
9.057 Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
9.429 Collecting astor==0.8.1
9.626 Downloading astor-0.8.1-py2.py3-none-any.whl (27 kB)
10.07 Collecting asttokens==2.2.1
10.28 Downloading asttokens-2.2.1-py2.py3-none-any.whl (26 kB)
10.74 Collecting backcall==0.2.0
11.01 Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)
11.53 Collecting colorama==0.4.6
11.72 Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
12.08 Collecting cycler==0.11.0
12.28 Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
12.63 Collecting decorator==5.1.1
12.84 Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
13.29 Collecting executing==1.2.0
13.56 Downloading executing-1.2.0-py2.py3-none-any.whl (24 kB)
14.12 Collecting filelock==3.12.2
14.38 Downloading filelock-3.12.2-py3-none-any.whl (10 kB)
16.19 Collecting greenlet==2.0.2
16.43 Downloading greenlet-2.0.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (607 kB)
16.99 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 607.6/607.6 KB 1.2 MB/s eta 0:00:00
17.71 Collecting ipython==8.14.0
17.90 Downloading ipython-8.14.0-py3-none-any.whl (798 kB)
18.25 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 798.7/798.7 KB 2.4 MB/s eta 0:00:00
18.59 Collecting jedi==0.19.0
19.00 Downloading jedi-0.19.0-py2.py3-none-any.whl (1.6 MB)
19.11 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 14.4 MB/s eta 0:00:00
19.52 Collecting Jinja2==3.1.2
19.81 Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
19.93 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 KB 3.4 MB/s eta 0:00:00
20.58 Collecting kiwisolver==1.4.4
20.77 Downloading kiwisolver-1.4.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.4 MB)
21.05 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 5.2 MB/s eta 0:00:00
21.71 Collecting MarkupSafe==2.1.3
21.90 Downloading MarkupSafe-2.1.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB)
23.44 Collecting matplotlib==3.3.4
23.70 Downloading matplotlib-3.3.4.tar.gz (37.9 MB)
26.25 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.9/37.9 MB 15.4 MB/s eta 0:00:00
32.15 Preparing metadata (setup.py): started
45.96 Preparing metadata (setup.py): finished with status 'done'
46.22 Collecting matplotlib-inline==0.1.6
46.43 Downloading matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
46.80 Collecting mpmath==1.3.0
47.05 Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
47.11 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 KB 10.5 MB/s eta 0:00:00
47.52 Collecting networkx==3.1
47.76 Downloading networkx-3.1-py3-none-any.whl (2.1 MB)
47.91 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 14.8 MB/s eta 0:00:00
49.84 Collecting numpy==1.26.2
49.87 Using cached numpy-1.26.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.2 MB)
50.56 Collecting packaging==23.1
50.94 Downloading packaging-23.1-py3-none-any.whl (48 kB)
50.98 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 KB 1.9 MB/s eta 0:00:00
52.52 Collecting pandas==1.2.5
53.19 Downloading pandas-1.2.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.0 MB)
56.87 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.0/10.0 MB 2.7 MB/s eta 0:00:00
57.67 Collecting pandasgui==0.2.14
57.90 Downloading pandasgui-0.2.14.tar.gz (218 kB)
57.96 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 218.2/218.2 KB 5.2 MB/s eta 0:00:00
58.06 Preparing metadata (setup.py): started
63.18 Preparing metadata (setup.py): finished with status 'done'
63.53 Collecting parso==0.8.3
63.74 Downloading parso-0.8.3-py2.py3-none-any.whl (100 kB)
63.78 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.8/100.8 KB 3.3 MB/s eta 0:00:00
64.11 Collecting pickleshare==0.7.5
64.26 Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
66.27 Collecting Pillow==10.0.0
66.92 Downloading Pillow-10.0.0-cp39-cp39-manylinux_2_28_aarch64.whl (3.2 MB)
67.65 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 4.4 MB/s eta 0:00:00
68.34 Collecting plotly==5.16.1
68.65 Downloading plotly-5.16.1-py2.py3-none-any.whl (15.6 MB)
70.70 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.6/15.6 MB 7.2 MB/s eta 0:00:00
71.98 Collecting prompt-toolkit==3.0.39
72.16 Downloading prompt_toolkit-3.0.39-py3-none-any.whl (385 kB)
72.23 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 385.2/385.2 KB 6.6 MB/s eta 0:00:00
72.60 Collecting pure-eval==0.2.2
72.96 Downloading pure_eval-0.2.2-py3-none-any.whl (11 kB)
73.97 Collecting pyarrow==12.0.1
74.18 Downloading pyarrow-12.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (36.4 MB)
83.28 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.4/36.4 MB 3.4 MB/s eta 0:00:00
84.28 Collecting Pygments==2.16.1
84.52 Downloading Pygments-2.16.1-py3-none-any.whl (1.2 MB)
84.62 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 13.6 MB/s eta 0:00:00
85.07 Collecting pynput==1.7.6
85.35 Downloading pynput-1.7.6-py2.py3-none-any.whl (89 kB)
85.39 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 KB 3.2 MB/s eta 0:00:00
85.97 Collecting pyparsing==3.1.1
86.18 Downloading pyparsing-3.1.1-py3-none-any.whl (103 kB)
86.22 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.1/103.1 KB 4.0 MB/s eta 0:00:00
86.61 Collecting PyQt5==5.15.9
86.99 Downloading PyQt5-5.15.9.tar.gz (3.2 MB)
87.20 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 16.1 MB/s eta 0:00:00
89.51 Installing build dependencies: started
116.1 Installing build dependencies: finished with status 'done'
116.1 Getting requirements to build wheel: started
117.1 Getting requirements to build wheel: finished with status 'done'
117.1 Preparing metadata (pyproject.toml): started
118.1 Preparing metadata (pyproject.toml): finished with status 'error'
118.1 error: subprocess-exited-with-error
118.1
118.1 × Preparing metadata (pyproject.toml) did not run successfully.
118.1 │ exit code: 1
118.1 ╰─> [30 lines of output]
118.1 pyproject.toml: line 7: using '[tool.sip.metadata]' to specify the project metadata is deprecated and will be removed in SIP v7.0.0, use '[project]' instead
118.1 Traceback (most recent call last):
118.1 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 156, in prepare_metadata_for_build_wheel
118.1 hook = backend.prepare_metadata_for_build_wheel
118.1 AttributeError: module 'sipbuild.api' has no attribute 'prepare_metadata_for_build_wheel'
118.1
118.1 During handling of the above exception, another exception occurred:
118.1
118.1 Traceback (most recent call last):
118.1 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
118.1 main()
118.1 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
118.1 json_out['return_val'] = hook(**hook_input['kwargs'])
118.1 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 160, in prepare_metadata_for_build_wheel
118.1 whl_basename = backend.build_wheel(metadata_directory, config_settings)
118.1 File "/tmp/pip-build-env-lavkddb7/overlay/lib/python3.9/site-packages/sipbuild/api.py", line 46, in build_wheel
118.1 project = AbstractProject.bootstrap('wheel',
118.1 File "/tmp/pip-build-env-lavkddb7/overlay/lib/python3.9/site-packages/sipbuild/abstract_project.py", line 87, in bootstrap
118.1 project.setup(pyproject, tool, tool_description)
118.1 File "/tmp/pip-build-env-lavkddb7/overlay/lib/python3.9/site-packages/sipbuild/project.py", line 587, in setup
118.1 self.apply_user_defaults(tool)
118.1 File "project.py", line 68, in apply_user_defaults
118.1 super().apply_user_defaults(tool)
118.1 File "/tmp/pip-build-env-lavkddb7/overlay/lib/python3.9/site-packages/pyqtbuild/project.py", line 70, in apply_user_defaults
118.1 super().apply_user_defaults(tool)
118.1 File "/tmp/pip-build-env-lavkddb7/overlay/lib/python3.9/site-packages/sipbuild/project.py", line 237, in apply_user_defaults
118.1 self.builder.apply_user_defaults(tool)
118.1 File "/tmp/pip-build-env-lavkddb7/overlay/lib/python3.9/site-packages/pyqtbuild/builder.py", line 69, in apply_user_defaults
118.1 raise PyProjectOptionException('qmake',
118.1 sipbuild.pyproject.PyProjectOptionException
118.1 [end of output]
118.1
118.1 note: This error originates from a subprocess, and is likely not a problem with pip.
118.1 error: metadata-generation-failed
118.1
118.1 × Encountered error while generating package metadata.
118.1 ╰─> See above for output.
118.1
118.1 note: This is an issue with the package mentioned above, not pip.
118.1 hint: See above for details.
118.2 WARNING: You are using pip version 22.0.4; however, version 23.3.1 is available.
118.2 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
------
Dockerfile:4
--------------------
2 | ADD . /ppo_eval
3 | WORKDIR /ppo_eval
4 | >>> RUN pip install -r requirements.txt
5 | ENV PATH=$PATH:/ppo_eval/ppo_eval
6 | ENV PYTHONPATH=/ppo_eval/ppo_eval
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements.txt" did not complete successfully: exit code: 1
这里的error是因为arm64架构的板子装不上qt5及相关库,不过这就是另一个问题了。
剩下的见
附
主机为 ubuntu 20.04 64位
报错的设备为 NVIDIA Xavier NX,ubuntu 20.04
Docker均通过官方 test_docker.sh 安装