Difference between revisions of "BackupPc"

From YobiWiki
Jump to navigation Jump to search
 
(20 intermediate revisions by the same user not shown)
Line 18: Line 18:
   
 
To prepare backuppc to access the hosts:
  
To prepare backuppc to access the hosts:
  +
<pre>
−
su - backuppc
  
  +
su - backuppc
−
ssh-keygen -t rsa -N ''
  
  +
ssh-keygen -t rsa -N ''
  +
</pre>
   
−
For a full restoration from scratch, see the backuppc paragraph in [[HardDrives]]
 +
For a full restoration from scratch, see the backuppc paragraph in [[Harddrive]]
   
 
Whenever you change the config files, be sure to have the proper permissions:
  
Whenever you change the config files, be sure to have the proper permissions:
Line 35: Line 37:
 
* scp /var/lib/backuppc/.ssh/id_rsa.pub root@<host_to_backup>:
  
* scp /var/lib/backuppc/.ssh/id_rsa.pub root@<host_to_backup>:
 
* on the host:
  
* on the host:
−
** mv id_rsa.pub ~/.ssh/authorized_keys
 +
** cat id_rsa.pub >> ~/.ssh/authorized_keys
 
** prepend the line in authorized_keys by sth like<br>from="<my_backup_server>" <br> so that only the backup host can log with this key.
  
** prepend the line in authorized_keys by sth like<br>from="<my_backup_server>" <br> so that only the backup host can log with this key.
 
** install a ssh server if not yet done
  
** install a ssh server if not yet done
 
** install rsync
  
** install rsync
 
* Try once as user backuppc to log as root on your host, first to accept the fingerprint, second to check if you are in without prompt for pwd
  
* Try once as user backuppc to log as root on your host, first to accept the fingerprint, second to check if you are in without prompt for pwd
  +
* Reload hosts in backuppc: /etc/init.d/backuppc reload
  +
  +
===Troubleshooting===
  +
Backup failed and you've only a very poor error message?
  +
  +
For example:
  +
Got fatal error during xfer (Unable to read 4 bytes)
  +
  +
There are more complete logfiles in /var/lib/backuppc/pc/*yourmachine*/XferLOG.bad.z
  +
<br>But they're compressed with libz (I think) so to read the file:
  +
$ /usr/share/backuppc/bin/BackupPC_zcat /var/lib/backuppc/pc/*yourmachine*/XferLOG.bad.z
  +
To continue on the same example, here we got:
  +
full backup started for directory /blabla
  +
Running: /usr/bin/ssh -q -x -l root yourmachine /usr/bin/rsync --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive --bwlimit=1500 --ignore-times . /blabla
  +
Xfer PIDs are now 10100
  +
Read EOF: Connection reset by peer
  +
Tried again: got 0 bytes
  +
Done: 0 files, 0 bytes
  +
Got fatal error during xfer (Unable to read 4 bytes)
  +
Backup aborted (Unable to read 4 bytes)
  +
Not saving this as a partial backup since it has fewer files than the prior one (got 0 and 0 files versus 0)
  +
Ok ssh command failed. Let's try it:
  +
# su - backuppc
  +
$ /usr/bin/ssh -l root yourmachine ls
  +
  +
The authenticity of host 'yourmachine (1.2.3.4)' can't be established.
  +
RSA key fingerprint is bl:ab:la
  +
Are you sure you want to continue connecting (yes/no)? yes
  +
So that's was the issue! That's what happen when you miss the last step in the paragraph about adding a host, just here above!
   
 
===To invert tunnelling (if from behind a firewall)===
  
===To invert tunnelling (if from behind a firewall)===
Line 52: Line 83:
 
*** Port 2222
  
*** Port 2222
 
===[[Harddrive#Restoring_from_backuppc|Restoring]]===
  
===[[Harddrive#Restoring_from_backuppc|Restoring]]===
  +
===Deleting backups===
  +
I was used to do it manually but there is a nice script [http://sourceforge.net/apps/mediawiki/backuppc/index.php?title=How_to_delete_backups here], probably handling it less wildly than what I did...
  +
 
===Tips===
  
===Tips===
 
When upgrading to rsync 2.6.7 on the clients you could face some problems.
  
When upgrading to rsync 2.6.7 on the clients you could face some problems.
Line 58: Line 92:
 
<br>Change it in config.pl and other host config files: $Conf{RsyncArgs} and $Conf{RsyncRestoreArgs}
  
<br>Change it in config.pl and other host config files: $Conf{RsyncArgs} and $Conf{RsyncRestoreArgs}
 
<br>cf http://sourceforge.net/mailarchive/forum.php?thread_id=10176480&forum_id=503
  
<br>cf http://sourceforge.net/mailarchive/forum.php?thread_id=10176480&forum_id=503
  +
  +
===Notes on new install===
  +
On the [[Fit PC]]
  +
  +
http://cdimage.debian.org/mirror/cdimage/unofficial/non-free/cd-including-firmware/current/i386/iso-cd/firmware-8.6.0-i386-netinst.iso
  +
  +
LVM
  +
* VG fitpc-vg
  +
** LV swap 2G => crypto; random key
  +
*** swap
  +
** LV root 23G => ext4; label ROOT; use as /
  +
** LV home 500G => crypto
  +
*** ext4; label HOME; use as /home; noatime,nodirtime; usage:news (more inodes)
  +
  +
Mounting encrypted home & swap failed during install -> mark them as "do not use" and see later.
  +
  +
* web server ; ssh server ; standard utils
  +
  +
Reboot
  +
  +
Encrypt swap:
  +
  +
Create /etc/crypttab
  +
cswap1 /dev/disk/by-id/dm-name-fitpc--vg-swap /dev/urandom swap,cipher=aes-xts-plain64,size=256,hash=sha1
  +
  +
Add to /etc/fstab
  +
/dev/mapper/cswap1 none swap sw 0 0
  +
Mount and format
  +
cryptdisks_start cswap1
  +
mkswap /dev/mapper/cswap1
  +
Activate swap
  +
swapon -a
  +
free
  +
  +
Handle encrypted home:
  +
  +
Add to /etc/crypttab
  +
home_crypt /dev/disk/by-id/dm-name-fitpc--vg-home none luks,noauto
  +
Add to /etc/fstab
  +
/dev/mapper/home_crypt /home_crypt ext4 noatime,nodiratime,noauto,errors=remount-ro 0 0
  +
Create fs
  +
cryptdisks_start home_crypt
  +
mkfs.ext4 -i 4096 /dev/mapper/home_crypt
  +
mount /home_crypt
  +
  +
Edit /etc/hosts
  +
127.0.1.1 fitpc.xxx.be fitpc
  +
192.168.1.2 fitpc
  +
...
  +
  +
Install stuff
  +
apt-get install mc screen backuppc
  +
Move backuppc
  +
systemctl disable backuppc
  +
/etc/init.d/backuppc stop
  +
cp -a /var/lib/backuppc/ /home_crypt/
  +
rm -rf /var/lib/backuppc/
  +
ln -s /home_crypt/backuppc /var/lib/
  +
/etc/init.d/backuppc start
  +
  +
/root/backuppc :
  +
#!/bin/bash
  +
cryptdisks_start home_crypt
  +
mount /home_crypt
  +
/etc/init.d/backuppc start
  +
  +
Fix /etc/backuppc/apache.conf:
  +
Options ExecCGI FollowSymlinks
  +
=>
  +
Options +ExecCGI +FollowSymlinks
  +
then
  +
service apache restart
  +
  +
Update web passwd and create key:
  +
htpasswd /etc/backuppc/htpasswd backuppc
  +
su - backuppc
  +
ssh-keygen -t ed25519 -N ''
  +
exit
  +
  +
Sensors:
  +
apt-get install lm-sensors sensord
  +
sensors-detect # use default, no need for optional scans
  +
# -> coretemp
  +
echo -e "chip \"acpitz-virtual-0\"\n ignore temp1\n ignore temp" > /etc/sensors.d/fitpc-sensors.conf
  +
Read sensors:
  +
sensors
  +
acpitz-virtual-0
  +
Adapter: Virtual device
  +
  +
coretemp-isa-0000
  +
Adapter: ISA adapter
  +
Core 0: +51.0°C (crit = +90.0°C)
  +
Edit /etc/default/sensord
  +
ALARM_INTERVAL=1m
  +
LOG_INTERVAL=30m
  +
SYSLOG_FACILITY=daemon
  +
RRD_FILE=/var/log/sensord.rrd
  +
RRD_INTERVAL=5m
  +
RRD_LOADAVG=yes
  +
  +
service sensord restart
  +
  +
Emails:
  +
Change /etc/exim4/update-exim4.conf.conf
  +
dc_eximconfig_configtype='smarthost'
  +
dc_smarthost='smtp.belgacom.net'
  +
  +
update-exim4.conf
  +
service exim4 restart
  +
  +
Change /etc/email-addresses (= From:)
  +
root: some_valid_email
  +
backuppc: some_valid_email
  +
<!--
  +
root: phil-fitpc@....
  +
backuppc: phil-backuppc@....
  +
phil: phil@....
  +
-->
  +
  +
Add to /etc/aliases (= To: for backuppc users)
  +
myuser: some_valid_email
  +
<!--
  +
phil: phil@....
  +
andrea: andrea@....
  +
-->
  +
  +
Edit /etc/mailname
  +
xxx.be
  +
  +
Fix localhost backup (auth errors):
  +
apt-get install sudo
  +
echo "backuppc ALL = NOPASSWD: /bin/tar" > /etc/sudoers.d/backuppc
  +
Then edit /etc/backuppc/localhost.pl
  +
$Conf{TarClientCmd} = '/usr/bin/env LC_ALL=C $tarPath -c -v -f - -C $shareName'
  +
=>
  +
$Conf{TarClientCmd} = '/usr/bin/env LC_ALL=C /usr/bin/sudo $tarPath -c -v -f - -C $shareName'
  +
  +
Add a machine:
  +
* Add machine in /etc/hosts
  +
192.168.x.x mymachine
  +
* From machine: (assuming fitpc2 is in its /etc/hosts, warning some devices don't seem to support the "from=")
  +
sudo apt-get install openssh-server rsync
  +
(echo -n "from=\"<IP_of_fitpc2>\" ";ssh root@fitpc2 cat /var/lib/backuppc/.ssh/id_ed25519.pub) |sudo tee -a /root/.ssh/authorized_keys
  +
* Test from the server twice (once to accept fingerprint, once to check):
  +
su -c "ssh root@mercure_eth" - backuppc
  +
* Add it to /etc/backuppc/hosts
  +
mymachine 0 myuser backuppc
  +
* Create a file /etc/backuppc/<host>.pl with the same name as in /etc/hosts and /etc/backuppc/hosts and with variables from config.pl that you want to adapt for your host
  +
* Reload
  +
service backuppc reload

Latest revision as of 01:09, 6 January 2017

Setup

  • apt-get install apache2 backuppc libfile-rsyncp-perl
  • add to /etc/aliases
backuppc: root

If you use a dedicated disk or partition:

  • prefer reiserfs over ext3 to avoid any inode shortage problem.
  • mount it with "noatime" for better performances
  • If you've to move an existing setup, move /var/lib/backuppc to the dedicated partition but preserve hardlinks (use cp -a), then create /var/lib/backuppc symlink

To create/change http passwords: 

 htpasswd /etc/backuppc/htpasswd backuppc

To create a new user: 

htpasswd /etc/backuppc/htpasswd user

add to /etc/aliases 

user: user@email.com

To prepare backuppc to access the hosts: 

su - backuppc
ssh-keygen -t rsa -N ''

For a full restoration from scratch, see the backuppc paragraph in Harddrive

Whenever you change the config files, be sure to have the proper permissions: 

chown backuppc:www-data /etc/backuppc/*
chmod go-rwx /etc/backuppc/*pl

Apache2

ln -s /etc/backuppc/apache.conf /etc/apache2/sites-enabled/backuppc

Add a host for backup

  • add it to /etc/backuppc/hosts
  • create a file such as /etc/backuppc/<host>.pl with the same name and with variables from config.pl that you want to adapt for your host
  • scp /var/lib/backuppc/.ssh/id_rsa.pub root@<host_to_backup>:
  • on the host:
    • cat id_rsa.pub >> ~/.ssh/authorized_keys
    • prepend the line in authorized_keys by sth like
      from="<my_backup_server>"
      so that only the backup host can log with this key.
    • install a ssh server if not yet done
    • install rsync
  • Try once as user backuppc to log as root on your host, first to accept the fingerprint, second to check if you are in without prompt for pwd
  • Reload hosts in backuppc: /etc/init.d/backuppc reload

Troubleshooting

Backup failed and you've only a very poor error message?

For example: 

Got fatal error during xfer (Unable to read 4 bytes)

There are more complete logfiles in /var/lib/backuppc/pc/*yourmachine*/XferLOG.bad.z
But they're compressed with libz (I think) so to read the file: 

$ /usr/share/backuppc/bin/BackupPC_zcat /var/lib/backuppc/pc/*yourmachine*/XferLOG.bad.z

To continue on the same example, here we got: 

full backup started for directory /blabla
Running: /usr/bin/ssh -q -x -l root yourmachine /usr/bin/rsync --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive --bwlimit=1500 --ignore-times . /blabla
Xfer PIDs are now 10100
Read EOF: Connection reset by peer
Tried again: got 0 bytes
Done: 0 files, 0 bytes
Got fatal error during xfer (Unable to read 4 bytes)
Backup aborted (Unable to read 4 bytes)
Not saving this as a partial backup since it has fewer files than the prior one (got 0 and 0 files versus 0)

Ok ssh command failed. Let's try it: 

# su - backuppc
$ /usr/bin/ssh -l root yourmachine ls

The authenticity of host 'yourmachine (1.2.3.4)' can't be established.
RSA key fingerprint is bl:ab:la
Are you sure you want to continue connecting (yes/no)? yes

So that's was the issue! That's what happen when you miss the last step in the paragraph about adding a host, just here above!

To invert tunnelling (if from behind a firewall)

  • On the host:
    • ssh -R 2222:localhost:22 <backup_server>
      Make it passwordless with keys
    • To make it automated: install autossh
      Add somewhere in the startup scripts:
      su <my_local_user> -c 'autossh -N -f -M 29001 -R 2222:localhost:22 <backup_server>' &
    • Or in /etc/network/interfaces
      up su <my_local_user> -c 'autossh -N -f -M 29001 -R 2222:localhost:22 <backup_server>' &
      down su <my_local_user> -c 'killall autossh'
  • On the server:
    • backuppc ~/.ssh/config:
      • Host <host_to_backup_as_said_to_backuppc>
      • ~HostName <localhost or ip_pub for vservers>
      • Port 2222

Restoring

Deleting backups

I was used to do it manually but there is a nice script here, probably handling it less wildly than what I did...

Tips

When upgrading to rsync 2.6.7 on the clients you could face some problems.
This is due to a different interpretation of the option --devices
Use -D instead for sth compatible with all rsync versions.
Change it in config.pl and other host config files: $Conf{RsyncArgs} and $Conf{RsyncRestoreArgs}
cf http://sourceforge.net/mailarchive/forum.php?thread_id=10176480&forum_id=503

Notes on new install

On the Fit PC

http://cdimage.debian.org/mirror/cdimage/unofficial/non-free/cd-including-firmware/current/i386/iso-cd/firmware-8.6.0-i386-netinst.iso

LVM

  • VG fitpc-vg
    • LV swap 2G => crypto; random key
      • swap
    • LV root 23G => ext4; label ROOT; use as /
    • LV home 500G => crypto
      • ext4; label HOME; use as /home; noatime,nodirtime; usage:news (more inodes)

Mounting encrypted home & swap failed during install -> mark them as "do not use" and see later.

  • web server ; ssh server ; standard utils

Reboot

Encrypt swap:

Create /etc/crypttab 

 cswap1          /dev/disk/by-id/dm-name-fitpc--vg-swap       /dev/urandom    swap,cipher=aes-xts-plain64,size=256,hash=sha1

Add to /etc/fstab 

 /dev/mapper/cswap1  none        swap    sw            0       0

Mount and format 

 cryptdisks_start cswap1
 mkswap /dev/mapper/cswap1

Activate swap 

 swapon -a
 free

Handle encrypted home:

Add to /etc/crypttab 

 home_crypt      /dev/disk/by-id/dm-name-fitpc--vg-home       none luks,noauto

Add to /etc/fstab 

 /dev/mapper/home_crypt     /home_crypt     ext4    noatime,nodiratime,noauto,errors=remount-ro 0 0

Create fs 

 cryptdisks_start home_crypt
 mkfs.ext4 -i 4096 /dev/mapper/home_crypt
 mount /home_crypt

Edit /etc/hosts 

127.0.1.1 fitpc.xxx.be fitpc
192.168.1.2 fitpc
...

Install stuff 

 apt-get install mc screen backuppc

Move backuppc 

 systemctl disable backuppc
 /etc/init.d/backuppc stop
 cp -a /var/lib/backuppc/ /home_crypt/
 rm -rf /var/lib/backuppc/
 ln -s /home_crypt/backuppc /var/lib/
 /etc/init.d/backuppc start

/root/backuppc : 

 #!/bin/bash
 cryptdisks_start home_crypt
 mount /home_crypt
 /etc/init.d/backuppc start

Fix /etc/backuppc/apache.conf: 

Options ExecCGI FollowSymlinks
=>
Options +ExecCGI +FollowSymlinks

then 

service apache restart

Update web passwd and create key: 

htpasswd /etc/backuppc/htpasswd backuppc
su - backuppc
ssh-keygen -t ed25519 -N 
exit

Sensors: 

apt-get install lm-sensors sensord
sensors-detect # use default, no need for optional scans
# -> coretemp
echo -e "chip \"acpitz-virtual-0\"\n        ignore temp1\n        ignore temp" > /etc/sensors.d/fitpc-sensors.conf

Read sensors: 

sensors
acpitz-virtual-0
Adapter: Virtual device

coretemp-isa-0000
Adapter: ISA adapter
Core 0:       +51.0°C  (crit = +90.0°C)

Edit /etc/default/sensord 

ALARM_INTERVAL=1m
LOG_INTERVAL=30m
SYSLOG_FACILITY=daemon
RRD_FILE=/var/log/sensord.rrd
RRD_INTERVAL=5m
RRD_LOADAVG=yes

service sensord restart

Emails: Change /etc/exim4/update-exim4.conf.conf 

dc_eximconfig_configtype='smarthost'
dc_smarthost='smtp.belgacom.net'

update-exim4.conf
service exim4 restart

Change /etc/email-addresses (= From:) 

root: some_valid_email
backuppc: some_valid_email

Add to /etc/aliases (= To: for backuppc users) 

myuser: some_valid_email

Edit /etc/mailname 

xxx.be

Fix localhost backup (auth errors): 

apt-get install sudo
echo "backuppc ALL = NOPASSWD: /bin/tar" > /etc/sudoers.d/backuppc

Then edit /etc/backuppc/localhost.pl 

$Conf{TarClientCmd} = '/usr/bin/env LC_ALL=C $tarPath -c -v -f - -C $shareName'
=>
$Conf{TarClientCmd} = '/usr/bin/env LC_ALL=C /usr/bin/sudo $tarPath -c -v -f - -C $shareName'

Add a machine:

  • Add machine in /etc/hosts
192.168.x.x mymachine
  • From machine: (assuming fitpc2 is in its /etc/hosts, warning some devices don't seem to support the "from=")
sudo apt-get install openssh-server rsync
(echo -n "from=\"<IP_of_fitpc2>\" ";ssh root@fitpc2 cat /var/lib/backuppc/.ssh/id_ed25519.pub) |sudo tee -a /root/.ssh/authorized_keys
  • Test from the server twice (once to accept fingerprint, once to check):
su -c "ssh root@mercure_eth" - backuppc
  • Add it to /etc/backuppc/hosts
mymachine 0       myuser   backuppc
  • Create a file /etc/backuppc/<host>.pl with the same name as in /etc/hosts and /etc/backuppc/hosts and with variables from config.pl that you want to adapt for your host
  • Reload
service backuppc reload
Retrieved from "https://wiki.yobi.be/index.php?title=BackupPc&oldid=10259"