基于PaddleGAN快速让你的照片动起来

Question

基于PaddleGAN快速让你的照片动起来
本项目基于PaddleGAN实现的FirstOrder与Wav2lip,。FirstOrder是输入一个模板视频与一张照片，就可以使照片里面的人物唱出模板视频里的歌曲，前段时间很火的 「蚂蚁呀嘿」就是用这个方法做的；还有另一个方法就是使用Wav2lip，输入照片和音频就可以直接让照片根据音频的内容动起来。

方案一：FirstOrder
1.下载PaddleGAN代码
In [ ]
# 从github上克隆PaddleGAN代码
!git clone https://gitee.com/paddlepaddle/PaddleGAN
In [ ]
# 安装所需安装包
%cd PaddleGAN/
!pip install -r requirements.txt
!pip install imageio-ffmpeg
%cd applications/
2.将驱动视频迁移至照片中
大家可以上传自己准备的视频和图片，并在下面的代码中source_image参数和driving_video参数分别换成自己的图片和视频路径，然后点击运行，就可以完成动作表情迁移，程序运行成功后，会在ouput文件夹生成名为result.mp4的视频文件，该文件即为动作迁移后的视频。

本项目中提供了原始图片和驱动视频供展示使用。具体的各参数使用说明如下

driving_video: 驱动视频，视频中人物的表情动作作为待迁移的对象
source_image: 原始图片，视频中人物的表情动作将迁移到该原始图片中的人物上
relative: 指示程序中使用视频和图片中人物关键点的相对坐标还是绝对坐标，建议使用相对坐标，若使用绝对坐标，会导致迁移后人物扭曲变形
adapt_scale: 根据关键点凸包自适应运动尺度
output：设置输出视频的存放文件夹
In [ ]
!export PYTHONPATH=$PYTHONPATH:/home/aistudio/PaddleGAN && python -u tools/first-order-demo.py  --driving_video /home/aistudio/PlanA_FirstOrder/zhiming.mp4 \
                                                                                                --source_image /home/aistudio/PlanA_FirstOrder/trumps.png \
                                                                                                --relative \
                                                                                                --adapt_scale \
                                                                                                --output /home/aistudio/PlanA_FirstOrder/output/
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  def convert_to_list(value, n, name, dtype=np.int):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/scipy/linalg/__init__.py:217: DeprecationWarning: The module numpy.dual is deprecated.  Instead of using dual, use the functions directly from numpy or scipy.
  from numpy.dual import register_func
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/scipy/special/orthogonal.py:81: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  from numpy import (exp, inf, pi, sqrt, floor, sin, cos, around, int,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/scipy/io/matlab/mio5.py:98: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  from .mio5_utils import VarReader5
[04/08 21:00:32] ppgan INFO: Found /home/aistudio/.cache/ppgan/vox-cpk.pdparams
W0408 21:00:32.665690  1292 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0408 21:00:32.673117  1292 device_context.cc:372] device: 0, cuDNN Version: 7.6.
100%|█████████████████████████████████████████| 345/345 [00:12<00:00, 28.21it/s]
3.为生成的视频加上音乐
In [ ]
# add audio
!pip install moviepy
In [ ]
#为生成的视频加上音乐
from moviepy.editor import *

videoclip_driving = VideoFileClip("/home/aistudio/PlanA_FirstOrder/zhiming.mp4") # 打开驱动视频
videoclip_result = VideoFileClip("/home/aistudio/PlanA_FirstOrder/output/result.mp4") # 打开PaddleGan刚刚生成的视频

audio_driving = videoclip_driving.audio # 提取驱动视频里的声音
video = videoclip_result.set_audio(audio_driving) # 将提取出来的声音加到生成视频的音轨中

video.write_videofile("/home/aistudio/PlanA_FirstOrder/output/final_result.mp4", audio_codec="aac") # 保存视频
生成的视频将放置在“PlanA_FirstOrder/output/”目录下，下载后即可查看效果~

方案二：Wav2lip
（若运行了方案一，请先点击代码执行器—>重启执行器）

1.安装必要的资源包
In [1]
%cd /home/aistudio/work

# 安装所需安装包
!mkdir sndfile
%cd sndfile
!wget http://www.mega-nerd.com/libsndfile/files/libsndfile-1.0.28.tar.gz
!tar xzvf libsndfile-1.0.28.tar.gz

%cd libsndfile-1.0.28
!./configure --prefix=/home/aistudio/build_libs CFLAGS=-fPIC --enable-shared 
!make
!make install
In [7]
%cd /home/aistudio/PaddleGAN
!pip install -r requirements.txt
%cd applications/
2.根据音频直接合成唇形动作
只需在如下命令中的face参数和audio参数分别换成自己的视频和音频路径，然后运行如下命令，就可以生成和音频同步的视频。

程序运行完成后，会在当前文件夹下生成文件名为outfile参数指定的视频文件，该文件即为和音频同步的视频文件。本项目中提供了demo展示所用到的视频和音频文件。具体的参数使用说明如下：

face: 原始视频，视频中的人物的唇形将根据音频进行唇形合成--通俗来说，想让谁说话
audio：驱动唇形合成的音频，视频中的人物将根据此音频进行唇形合成--通俗来说，想让这个人说什么
In [11]
!export PYTHONPATH=$PYTHONPATH:/home/aistudio/PaddleGAN && python tools/wav2lip.py --face /home/aistudio/PlanB_Wav2lip/MonaLisa.jpeg \
                                                                                    --audio /home/aistudio/PlanB_Wav2lip/TA.mp3 \
                                                                                    --outfile /home/aistudio/PlanB_Wav2lip/output/result.mp4
生成的视频将放置在“PlanB_Wav2lip/output/”目录下，下载后即可查看效果~

Answer 1

• 文章：基于PaddleGAN精准唇形合成实现持枪人讲电话已关机 中也许有你想要的答案，请看下吧