weixin_39619858
2020-12-25 18:55 阅读 1

Compression

Hi, I experimented with kopia and compression.

I did a kopia policy set --global --compression=pgzip and later with s2-parallel-4.
After thar I did a tar cvfz of that directory, that showed that kopia did not compress much.

The results:

Linux

  • 54446 Downloads/ (the source in MB)
  • 52440 /media/kees/WD500G/kbackup (kopia no compression)
  • 51230 /media/kees/WD500G/kbackup2 (kopia pgzip compression)
  • 51307 /media/kees/WD500G/kbackup3 (kopia s2-parallel-4)
  • 52090 /media/kees/WD500G/rbackup (restic backup no compression)
  • 31774 downloads.tgz (tar'd directory with gzip default compression)

Did I something wrong?

Regards,

Kees

该提问来源于开源项目:kopia/kopia

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

4条回答 默认 最新

  • weixin_39946364 weixin_39946364 2020-12-25 18:55

    What kind of files were those?

    Kopia compresses each file content individually (and large files are split into contents of about 2-8mb) so compression cannot leverage any cross-file similarities.

    On the other hand tar bundles all files together and then gzip compresses them as a single large file so it can achieve great compression if you have lots of similar files that have related content (such as source files).

    Compare that to zip which compresses each file individually and depending of file sizes you should see roughly similar results.

    点赞 评论 复制链接分享
  • weixin_39619858 weixin_39619858 2020-12-25 18:55

    thank you for your reply.

    It's about: * 13GB mkv files * 19GB ISO's * 8GB jpg files * 1.5GB pdf files * 2GB zip files * 1.7GB tar.xz files

    I zipped the directory and indeed, it was about the size of the backups: 51420MB, which confirms your explenation. I'll do an other trial with a better compressible directory.

    Thanks again,

    Kind regards,

    Kees

    点赞 评论 复制链接分享
  • weixin_39900023 weixin_39900023 2020-12-25 18:55

    Most of those files seem not compressible since they are already compressed. The ISOs may be compressible depending on the contents of the image. ISOs from OS installers are likely non compressible since they contain a lot of already compressed installer packages.

    Cheers.

    点赞 评论 复制链接分享
  • weixin_39619858 weixin_39619858 2020-12-25 18:55

    -lopez that is correct, the zip only did 3-4% on the iso's. Funny thuough tar + gzip = a single large file being compressed, does so much more compression!

    Regards,

    Kees

    点赞 评论 复制链接分享

相关推荐