Linux/Backups

• Links: MongoDb/MongoBackupRestore

• Note: There is a lot of opensource tools for making linux backups, ranging from rsync to btrfs syncs.
  • I was looking for a simple cost effective sollution that would be future proof, (Not need some special software year's later to recover.)

Backup setup

1. Archive vm using tar. Builds backup file from filesystem, allows for including and excluding directories.
2. Monthly Compress and stream full backup to S3 tar.xz (By streaming i don't need additional local disk space)
3. Daily Compress and stream incremental change tar backup based off the initial monthly back and stream to s3.
4. Setup s3 bucket to move backups older than a week to Glacier cold storage, as they will probably not be needed, and its really cheap.

Components used in backup plan

• tar - https://en.wikipedia.org/wiki/Tar_(computing)

  • with --listed-incremental it can keep a record of monthly full backup and then only backup changed file's for the rest of the month.
• xz compression utility, tar gets piped through here
• aws s3 - to upload/stream tar to s3, reading from stdin.

Advantages of this setup

1. tar is really standard way to pack files, and preserve file permisions.
2. tar file can be downloaded from s3, and opend up to get hold of single file.
3. by using the incremental file feature of tar, backup size is reduced, but any day can be reconstructed by just downloading 2 tar files.
4. by streaming the output of tar to xv to aws s3, the backups get created while not occupying any disk space, great for small vm's on internet with limited storage, or full storage.

5. all three tools are available on Linux, Windows, Mac. xz - https://en.wikipedia.org/wiki/XZ_Utils

CategoryLinux