The rcat utility generates a multi-volume tar archive and uploads each volume to cloud storage via rclone. The entire process is automated, and therefore is perfectly suitable for a cronjob or as part of any scheduled backup routine.
Since each volume is only retained for the duration of the file transfer, it greatly streamlines the process of backing up and restoring large amounts of data (like an entire filesystem) when free hard drive space would otherwise be insufficient.
While there are a couple third-party tools with functionality similar to rcat, they are coded in Python and are much more sophisticated by comparison:
I prefer the simplest possible implementation when it comes to backing up important data, particularly since customization might be necessary. Hence, I decided a shell script in combination with the multi-volume feature of tar was best suited for this purpose.
The command syntax for a backup operation is as follows:
- ~/rcat/backup.csh CHUNK_SIZE LOCAL_PATH CLOUD_PATH/CHUNK_NAME.tar
- CHUNK_SIZE - the maximum file size for each volume of the tar archive, specified as a whole number in kilobytes, or optionally in megabytes or gigabytes by suffixing "M" or "G" respectively; sizes are base-2, thus 1M = 1024 kilobytes and 1G = 1024 megabytes
- LOCAL_PATH - the directory or file to backup; absolute paths will be stripped of the leading slash as is the default behavior of tar
- CLOUD_PATH - the location where rclone will store the tar archive, typically to the cloud
- CHUNK_NAME - the base filename for each volume of the tar archive, appended with the ".tar" extension; the resulting files will be named as "CHUNK_NAME.partX.tar", where "X" is the respective volume number.
Here are some example command invocations for backing up a directory and a file:
- ~/rcat/backup.csh 1G /tmp/worlds/jt2 remote:weeklies/tmp_worlds_jt2.tar
~/rcat/backup.csh 100M auth.txt remote:dailies/minetest/worlds/jt2/auth.txt.tar
- ~/rcat/restore.csh CLOUD_PATH/CHUNK_NAME.tar
- CLOUD_PATH - the location where rclone will retrieve the tar archive, typically from the cloud
- CHUNK_NAME - the base filename for each volume of the tar archive, appended with the ".tar" extension. The files must be named as "CHUNK_NAME.partX.tar," where "X" is the respective volume number.
Here are the corresponding command invocations for restoring a directory and a file:
- ~/rcat/restore.csh remote:weeklies/pub_worlds_jt2.tar
~/rcat/restore.csh remote:dailies/minetest/worlds/jt2/auth.txt.tar
- Backup: /tmp/worlds/jt2 -> Restore: ./tmp/worlds/jt2
- Backup: ./auth.txt -> Restore: ./auth.txt
Repository:
https://bitbucket.org/sorcerykid/rcat
Download Archive (.zip)
Download Archive (.tar.gz)
Dependencies:
Rclone can be downloaded from https://rclone.org/downloads/
Installation Instructions:
- Unzip the archive into your home directory.
- Rename the rcat-master directory to "rcat".
- Edit the CACHE_PATH setting in "backup.csh" accordingly.
- Edit the CACHE_PATH setting in "restore.csh" accordingly.
The MIT License