Vserver administration: Difference between revisions

From YobiWiki
Jump to navigation Jump to search
← Older editNewer edit →
Content deleted Content added
mNo edit summary
Line 1: Line 1:
==Introduction==
Official homepage: [http://linux-vserver.org/ Linux VServer Project]

Good introduction:
* [http://linux-vserver.org/index.php?page=Linux-VServer-Paper Linux-VServer Technology]
* [http://linux-vserver.org/index.php?page=Linux-VServer-Paper-French La Technologie Linux-VServer]

Debian support:
apt-cache search vserver
kernel-patch-vserver - context switching virtual private servers - kernel patch
[http://www.nongnu.org/util-vserver/ util-vserver] - tools for Virtual private servers and context switching
vserver-debiantools - Tools to manage debian virtual servers

Misc:
* [http://www.lri.fr/~fragile/IMG/pdf/Quetier.pdf Benchmark Comparisons between UML, VMWare, vserver and Xen (pdf)]

==Kernel compilation==
===The Debian way===
I followed instructions given in
* /usr/share/doc/kernel-patch-vserver/README.Debian
* [http://linux-vserver.org/Step-by-Step+Guide+2.6 Step-by-step 2.6]
* [http://deb.riseup.net/vserver/preparing/ Debian vservers]
* [http://arnofear.free.fr/linux/vserver-1.php Debian and vserver, french howto]
* [http://lena.franken.de/linux/debian_and_vserver/ Debian and vserver]
<pre>
apt-get install kernel-patch-vserver linux-source-2.6.16 kernel-package fakeroot
cd /usr/src
tar xjf linux-source-2.6.16.tar.bz2
cd /usr/src/linux-source-2.6.16
cp config-2.6.16-1-amd64-k8 .config
export PATCH_THE_KERNEL=YES
make-kpkg --rootcmd fakeroot \
--revision custom01 \
--added-patches vserver \
--append-to-version +vserver \
--initrd \
binary-arch
"Virtual root device support" -> **y**
"Legacy kernel API" -> y
"Show a Legacy Version ID" -> n
"Disable Legacy Networking Kernel API" -> n
"Enable Proc Security" -> y
"Enable Hard CPU Limits" -> y
"Limit the IDLE task" -> n
"Persistent Inode Context Tagging" -> UID24/GID24 (32/32 probably not yet supported on Reiserfs)
"Tag NFSD User Auth and Files" -> n
"VServer Debugging Code" -> n
</pre>
Install kernel and reboot
===Vanilla with GrSec, still the Debian way===
I used linux-2.6.17.14.tar.bz2 + patch-2.6.17.14-vs2.0.2.1-grsec2.1.9.diff
<br>and the config of the Debian kernel config-2.6.17-2-vserver-amd64
make oldconfig
I activated HARDCPU limits and misc PAX & GRSEC stuff ([http://people.linux-vserver.org/~harry/_README_ this page] can help):
<pre>
CONFIG_VSERVER_HARDCPU=y
CONFIG_VSERVER_HARDCPU_IDLE=y
CONFIG_PAX=y
CONFIG_PAX_SOFTMODE=y
CONFIG_PAX_PT_PAX_FLAGS=y
CONFIG_PAX_HAVE_ACL_FLAGS=y
CONFIG_PAX_NOEXEC=y
CONFIG_PAX_PAGEEXEC=y
CONFIG_PAX_MPROTECT=y
CONFIG_PAX_ASLR=y
CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y
CONFIG_PAX_MEMORY_SANITIZE=y
CONFIG_GRKERNSEC=y
CONFIG_GRKERNSEC_CUSTOM=y
CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_IO=y
CONFIG_GRKERNSEC_PROC_MEMMAP=y
CONFIG_GRKERNSEC_BRUTE=y
CONFIG_GRKERNSEC_MODSTOP=y
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=3
CONFIG_GRKERNSEC_ACL_TIMEOUT=30
CONFIG_GRKERNSEC_PROC=y
CONFIG_GRKERNSEC_PROC_USER=y
CONFIG_GRKERNSEC_PROC_ADD=y
CONFIG_GRKERNSEC_LINK=y
CONFIG_GRKERNSEC_FIFO=y
CONFIG_GRKERNSEC_CHROOT=y
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
CONFIG_GRKERNSEC_CHROOT_UNIX=y
CONFIG_GRKERNSEC_CHROOT_NICE=y
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
CONFIG_GRKERNSEC_RESLOG=y
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y
CONFIG_GRKERNSEC_PROC_IPADDR=y
CONFIG_GRKERNSEC_EXECVE=y
CONFIG_GRKERNSEC_SHM=y
CONFIG_GRKERNSEC_DMESG=y
CONFIG_GRKERNSEC_RANDPID=y
CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_SYSCTL=y
CONFIG_GRKERNSEC_FLOODTIME=10
CONFIG_GRKERNSEC_FLOODBURST=4
</pre>
make-kpkg --rootcmd fakeroot --us --uc --initrd kernel-image
And I got a linux-image-2.6.17.14-grsec2.1.9-vs2.0.2.1_2.6.17.14-grsec2.1.9-vs2.0.2.1-10.00.Custom_amd64.deb
==Host preparation==
<pre>
apt-get install util-vserver vserver-debiantools
wget http://vserver.13thfloor.at/Stuff/SCRIPT/testme.sh
chmod +x testme.sh
./testme.sh
dd bs=1024k count=1024 if=/dev/zero of=1gb.test
modprobe loop
losetup /dev/loop0 ./1gb.test
./testfs.sh [ -F reiser ] -D /dev/loop0 -M /mnt
losetup -d /dev/loop0
modprobe -r loop
</pre>
There is no error at this point but as I'm using Reiserfs, I have to activate manually the extended attributes (for lsattr/chattr) by adding the following option to /etc/fstab lines: "attrs" (?? also option acl ??)
<br>Test: lsattr <mount point of a Reiserfs>
===Change the vserver base path===
* /etc/vservers/.defaults/vdirbase -> /var/lib/vservers
* I change it to /home/vservers, fix the above symlink
* Re-create the "chroot barrier": %%%setattr --barrier /home/vservers%%%showattr /home -> B for vservers
* Some tools could have /var/lib/vservers hardcoded, for safety I create a symlink /var/lib/vservers pointing to /home/vservers
==Manipulating vservers==
===Create a vserver===
Edit /etc/vservers/newvserver-vars:
<pre>
# cf http://amd64.debian.net/README.mirrors.html
MIRROR="http://ftp.belnet.be/debian-amd64/debian"
INTERFACE="<my_if>"
ARCH="amd64"
</pre>
Create a vserver with 64bits:
LANG=C newvserver --hostname template64 --domain teuwen.org --ip <new_ip>/24 --dist etch
Create a vserver with 32bits emulation:
LANG=C newvserver --hostname template32 --domain teuwen.org --ip <new_ip>/24 --dist etch --arch i386 --mirror "http://<i386_debian_mirror>"
Tuning:
* take care of the config duplication!
* enter the vserver and run tzconfig to choose the proper timezone
* fix /etc/apt/sources.list
* delete rcX.d links to umountroot
* Warning! If you use newvserver as such, it will overwrite the host /etc/motd due to a symlink
* See [Vserver tools] for a patch for newvserver
Removing unnecessary progs (check if you really don't need them!!):
* aptitude apt-utils base-config cpio dselect tasksel libncursesw5 libsigc++-1.2-5c2 libsigc++-2.0-0c2a
* dmidecode laptop-detect module-init-tools
* bsdmainutils ed nano nvi
* groff-base man-db manpages info libgdbm3
* netcat traceroute wget libssl0.9.8
* gettext-base libconsole libgnutls11 liblzo2-2 libtasn1-2-bin

===Automatic start at bootup===
echo default > /etc/vservers/<my_vserver>/apps/init/mark
Note that at shotdown all vservers will be stopped
===Delete a vserver===
Remove dirs /home/vservers/<my_vserver> (depends on the setting of vdirbase, cf. above), /etc/vservers/<my_vserver> and /var/run/vservers/<my_vserver> and the corresponding symlink in /var/run/vservers.rev
===Config of a vserver===
''TODO''
?? /etc/vservers/<my_vserver>.conf
?? S_CAPS
see [http://www.nongnu.org/util-vserver/doc/conf/configuration.html Detailed config page (better choosing boring CSS...)]
===Run a vserver===
vserver <my_vserver> start
vserver <my_vserver> enter
If you get "mesg: /dev/pts/1: Operation not permitted", be root on the host with "su -"
vserver <my_vserver> stop
===Other tools===
vserver <my_vserver> status
vserver-stat
vtop, vps, vpstree, vkill
/etc/rc.d/init.d/rebootmgr is a daemon which can be called from vservers via vreboot and vhalt to stop/restart the vserver from inside

See also [http://www.nongnu.org/util-vserver/doc/conf/compatibility.html compatibility of util-vserver alpha branch]

See [[Vserver tools]] for my own/modified scripts

===Duplicate a vserver===
vserver <my_vserver1> stop
dupvserver --from <my_vserver1> --to <my_vserver2> --ip <new_ip>
dupvserver is broken with the new configuration structure /etc/vservers/<my_vserver>/
<br>See [[Vserver tools]] for a patch for dupvserver
===Move/copy a vserver===
Basically stop the vserver and copy /etc/vservers/<my_vserver> and /home/vservers/<my_vserver>
<br>E.g. rsync -e ssh -avHl /vservers/XX new-server:/vserver/XX
==Share directories==
To mount a directory from one vserver into another from the host:
vnamespace -e <vserver> mount --rbind /directory/to/mount/somewhere /where/to/mount/it
vnamespace -e <vserver> umount /where/it/was/mounted

or
mount --bind /home /var/lib/vservers/vserver1/home
mount --bind /home /var/lib/vservers/vserver2/home
The second method had the disavantage to require a reboot of the vserver
==Apt-get==
LANG=C vapt-get <my_vserver1> <my_vserver2> <...> -- install <pkg1> <pkg2>
==Unify==
cf immutable-linkage-invert flag

Preparation:
mkdir /etc/vservers/template64/apps/vunify
mkdir /etc/vservers/<my_vserver>/apps/vunify
ln -s /etc/vservers/template64 /etc/vservers/<my_vserver>/apps/vunify/refserver.template64
Unification:
<br>Be sure both vservers are running
vserver <my_vserver> unify [-n] [-R]
-n for dry run, no change
<br>-R for de-unifying

When using tar, add option -U to unlink & recreate files instead of overwriting.
<br>Manual set/unset of the immutable-linkage-invert flag:
setattr --iunlink /my/file
setattr --~iunlink /my/file
==Disk limits==
cf http://linux-vserver.org/Disk+Limits

* Assign static contexts for the vservers (i.e. have a value between 2 and 49151 in /etc/vservers/<name>/context)
* Mount the filesystem holding the vserver(s) with the tagxid option
** Check if this is mounted properly: use cat /proc/mounts<br>Ex.: /dev/mapper/Zeus-home /home reiserfs rw,tagxid 0 0
** WARNING: if the filesystem is already in use with vservers, nothing prevent you to umount the filesystem while the vservers are still running, which is VERY BAD! Be careful.
** I could only get the tagxid taken properly into account after a reboot
* Change the xid of already existing files:
chxid -c <my_vserver> -R /home/vservers/<my_vserver>
* Set limits, first method: here limit to 5Gb, 100000 inodes and 5% for the root user<br>For info as I could not get it working properly yet
mkdir /var/cache/vservers
ln -s /var/cache/vservers /etc/vservers/.defaults/cachebase
mkdir /etc/vservers/.defaults/cachebase/<my_server>
ln -s /etc/vservers/.defaults/cachebase/<my_server> /etc/vservers/<my_server>/cache
mkdir -p /etc/vservers/<my_vserver>/dlimits/0
echo /home/vservers/<my_vserver> > /etc/vservers/<my_vserver>/dlimits/0/directory
echo $(( 5 * 1024 * 1024 )) > /etc/vservers/<my_vserver>/dlimits/0/space_total
echo 100000 > /etc/vservers/<my_vserver>/dlimits/0/inodes_total
echo 5 > /etc/vservers/<my_vserver>/dlimits/0/reserved
* Set limits, second method:
** Install my vdlimit_ script in /usr/local/sbin: [[Vserver tools]]
ln -s /usr/local/sbin/vdlimit_ /etc/vservers/<my_vserver>/scripts/post-start.d/vdlimit_$((5*1024))
==Network==
===Intern network===
For pure loopback, use dummy interface, cf http://mirabellug.org/wikini/wakka.php?wiki=VServers

For usable dummy interface, us permanent taps as the uml tools allow:
apt-get install uml-utilities
* Create a pseudo-interface:
<pre>
auto tap0
iface tap0 inet static
address 192.168.2.1
netmask 255.255.255.0
tunctl_user uml-net
</pre>
And configure vservers with the same dev=tap0

Update: to check but actually all traffic with private or public IP will anyway be done through lo so this is probably not required
===Configure daemons to listen only to the IP-address of the mothersystem===
* ''openbsd-inetd:'' (not netkit-inetd) in file /etc/inetd.conf:<br>Prepend the service with <IP pub>:<br>Example
<IP pub>:cvspserver stream tcp nowait root /usr/sbin/tcpd /usr/sbin/cvs-pserver
* ''xinetd:'' (not inetd) in file /etc/xinetd.conf:
defaults
{ bind = <IP pub> }

/etc/init.d/xinetd restart
* ''sshd:'' in file /etc/ssh/sshd_config:
ListenAddress <IP pub>

/etc/init.d/ssh restart
* ''exim4:'' in file /etc/exim4/update-exim4.conf.conf:
dc_local_interfaces='<IP pub>'

/etc/init.d/exim4 restart
Better to do it through debconf to avoid surprises at update time: dpkg-reconfigure exim4-config
* ''courier-imap:'' in file /etc/courier/imapd:
ADDRESS=<IP pub>

/etc/init.d/courier-imap restart
* ''courier-imap-ssl:'' in file /etc/courier-ssl/imapd:
ADDRESS=<IP pub>

/etc/init.d/courier-imap-ssl restart
* ''imapproxy:'' in file /etc/imapproxy.conf:
listen_address <IP pub>
Within a vserver, you'll probably hav to reduce the cache_size or give capability to the vserver to raise the setrlimit.
* ''mysql:'' in file /etc/mysql/my.cnf:
bind-address = <IP pub>
* ''vsFtpd:'' in file /etc/vsftpd.conf:
listen_address=<IP pub>
* ''postgresql:'' in file /etc/postgresql/postgresql.conf:
virtual_host = '<IP pub>'
* ''apache2:'' in file /etc/apache2/ports.conf:
Listen <IP pub>:80
* ''zope2.9:'' in file /etc/zope2.9/<instance>/zope.conf:
ip-address <IP pub>
* ''portmap:'' in file /etc/default/portmap:
OPTIONS="-i <IP pub/loopback>"
* ''dnsmasq:'' in file /etc/dnsmasq.conf:
listen-address=<IP pub>
bind-interfaces
* netstat -lp -> other greedy daemons?
* Seems that this is possible via another method, here it will bind the daemon to the first IP of the interface:<br>exec /usr/sbin/chbind --ip eth0 /path/to/daemon
===Add an interface without rebooting the vserver===
* add the ip to the host (ip addr add ...)
* add the ip to the guest's network context
# naddress --add --nid <nid> --ip <ip>/<mask>
* enter the guest (best via ssh)
* restart the services if required<br>(most services will automatically start using the new addresses)
* update the config to reflect the changes for the next guest restart (if desired)
Thanks Herbert!
==Understanding vservers==
===Security contextes===
* Find security context of process N:
chcontext --ctx 1 cat /proc/N/status|grep s_context
* Be in the same context:
chcontext --ctx X /bin/sh
* Master context: 1, example to get all listening ports:
chcontext --ctx 1 netstat -lpn
See also [http://www.solucorp.qc.ca/miscprj/s_context.hc Virtual private servers and security contexts]
===Ceiling capabilities===
* As non-root, check capBset:
cat /proc/self/status
* Reduce ceiling caps:
reducecap --secure /bin/sh
* Now capBset is reduced:
cat /proc/self/status
su
* capEff raised a bit but not enough to do for example /sbin/ifconfig eth0 down
* See also [Capabilities in Linux|http://www.lids.org/lids-howto/node34.html]

==Security==
Not necessarily related to vserver but always useful to consider :-)
*ssh
**Use the AllowUsers option to give ssh rights only to those who need it.
**Brute-force protection: apt-get install denyhosts
*iptables (on the host)
**cf --uid-owner and other --XXX-owner options<br>on OUTPUT table to avoid download of malicious code<br>on INPUT table to avoid bindshells
*resource limits
** cpu/mem


===GrSec===
===GrSec===
* http://people.linux-vserver.org/~harry/_README_
* http://people.linux-vserver.org/~harry/_README_
===Iptables Proxy===
==Iptables Proxy==
* http://www.virtuaserver.com.br/forum/viewtopic.php?t=130
* http://www.virtuaserver.com.br/forum/viewtopic.php?t=130

===Other tricks===
* For other tweaks, see http://deb.riseup.net/vserver/usage/ :
** What if I accidentally removed a vserver while it was running?
** Howto convert legacy vservers to the new format
** Howto add an IP to a running vserver, without restarting it?
** Howto make the host interface and IP available in a vserver
** Howto impose disk limits in each vserver
* http://www.paul.sladen.org/vserver/faq
* [http://linux-vserver.org/ProblematicPrograms Problematic programs]

==TODO==
* http://www.nongnu.org/util-vserver/doc/conf/compatibility.html
* http://linux-vserver.derjohn.de/
* [VServer wiki|http://vserver.strahlungsfrei.de/tiki-index.php]
* [Administrator Guide|http://linux-vserver.org/linux-vserver_administrators_gide]
* [Debian newvserver|http://www.paul.sladen.org/vserver/debian/]
* [Howto Debian vserver|http://www.howtoforge.com/linux_vserver_debian]
* ?? apt-get install vlan
* ?? ipac-ng
* With grsecurity?
** http://linux-vserver.org/grsecurityHowto
** http://team.lea-linux.org/bgigon/vserver/mirror/ChangeLog
** http://pax.grsecurity.net/ apparently not yet stable for 2.6 and not yet available for amd64
** Interessant: http://ludit.kuleuven.be/software/vserver/_README_
* CPU limit
** http://linux-vserver.org/Linux-VServer-Paper-06
** http://list.linux-vserver.org/archive/vserver/msg08134.html
* BW limit
** http://lartc.org/howto/
* http://linux-vserver.org/HowTo+Read+ProcFS
* http://linux-vserver.org/HistoryList?full=1
* Publish Munin scripts
* http://linux-vserver.org/VServer+installation+Fedora+Core+5
* http://vserver.13thfloor.at/Experimental/
* http://www.archivesat.com/Linux-VServer/
* http://www.solucorp.qc.ca/miscprj/s_context.hc?s1=1&s2=0&s3=0&s4=0&full=0&prjstate=1&nodoc=0
* (fr) http://fr.wikibooks.org/wiki/Vserver

Revision as of 20:24, 17 February 2007

Introduction

Official homepage: Linux VServer Project

Good introduction:

Debian support: 

apt-cache search vserver
  kernel-patch-vserver - context switching virtual private servers - kernel patch
  util-vserver - tools for Virtual private servers and context switching
  vserver-debiantools - Tools to manage debian virtual servers

Misc:

Kernel compilation

The Debian way

I followed instructions given in 

apt-get install kernel-patch-vserver linux-source-2.6.16 kernel-package fakeroot
cd /usr/src
tar xjf linux-source-2.6.16.tar.bz2
cd /usr/src/linux-source-2.6.16
cp config-2.6.16-1-amd64-k8 .config
export PATCH_THE_KERNEL=YES
make-kpkg --rootcmd fakeroot \
        --revision custom01 \
        --added-patches vserver \
        --append-to-version +vserver \
        --initrd \
        binary-arch
"Virtual root device support" -> **y**
"Legacy kernel API" -> y
"Show a Legacy Version ID" -> n
"Disable Legacy Networking Kernel API" -> n
"Enable Proc Security" -> y
"Enable Hard CPU Limits" -> y
"Limit the IDLE task" -> n
"Persistent Inode Context Tagging" -> UID24/GID24 (32/32 probably not yet supported on Reiserfs)
"Tag NFSD User Auth and Files" -> n
"VServer Debugging Code" -> n

Install kernel and reboot

Vanilla with GrSec, still the Debian way

I used linux-2.6.17.14.tar.bz2 + patch-2.6.17.14-vs2.0.2.1-grsec2.1.9.diff
and the config of the Debian kernel config-2.6.17-2-vserver-amd64 

make oldconfig

I activated HARDCPU limits and misc PAX & GRSEC stuff (this page can help): 

CONFIG_VSERVER_HARDCPU=y
CONFIG_VSERVER_HARDCPU_IDLE=y
CONFIG_PAX=y
CONFIG_PAX_SOFTMODE=y
CONFIG_PAX_PT_PAX_FLAGS=y
CONFIG_PAX_HAVE_ACL_FLAGS=y
CONFIG_PAX_NOEXEC=y
CONFIG_PAX_PAGEEXEC=y
CONFIG_PAX_MPROTECT=y
CONFIG_PAX_ASLR=y
CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y
CONFIG_PAX_MEMORY_SANITIZE=y
CONFIG_GRKERNSEC=y
CONFIG_GRKERNSEC_CUSTOM=y
CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_IO=y
CONFIG_GRKERNSEC_PROC_MEMMAP=y
CONFIG_GRKERNSEC_BRUTE=y
CONFIG_GRKERNSEC_MODSTOP=y
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=3
CONFIG_GRKERNSEC_ACL_TIMEOUT=30
CONFIG_GRKERNSEC_PROC=y
CONFIG_GRKERNSEC_PROC_USER=y
CONFIG_GRKERNSEC_PROC_ADD=y
CONFIG_GRKERNSEC_LINK=y
CONFIG_GRKERNSEC_FIFO=y
CONFIG_GRKERNSEC_CHROOT=y
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
CONFIG_GRKERNSEC_CHROOT_UNIX=y
CONFIG_GRKERNSEC_CHROOT_NICE=y
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
CONFIG_GRKERNSEC_RESLOG=y
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y
CONFIG_GRKERNSEC_PROC_IPADDR=y
CONFIG_GRKERNSEC_EXECVE=y
CONFIG_GRKERNSEC_SHM=y
CONFIG_GRKERNSEC_DMESG=y
CONFIG_GRKERNSEC_RANDPID=y
CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_SYSCTL=y
CONFIG_GRKERNSEC_FLOODTIME=10
CONFIG_GRKERNSEC_FLOODBURST=4

make-kpkg --rootcmd fakeroot --us --uc --initrd kernel-image

And I got a linux-image-2.6.17.14-grsec2.1.9-vs2.0.2.1_2.6.17.14-grsec2.1.9-vs2.0.2.1-10.00.Custom_amd64.deb

Host preparation

apt-get install util-vserver vserver-debiantools
wget http://vserver.13thfloor.at/Stuff/SCRIPT/testme.sh
chmod +x testme.sh
./testme.sh
dd bs=1024k count=1024 if=/dev/zero of=1gb.test
modprobe loop
losetup /dev/loop0 ./1gb.test
./testfs.sh [ -F reiser ] -D /dev/loop0 -M /mnt
losetup -d /dev/loop0
modprobe -r loop

There is no error at this point but as I'm using Reiserfs, I have to activate manually the extended attributes (for lsattr/chattr) by adding the following option to /etc/fstab lines: "attrs" (?? also option acl ??)
Test: lsattr <mount point of a Reiserfs>

Change the vserver base path

  • /etc/vservers/.defaults/vdirbase -> /var/lib/vservers
  • I change it to /home/vservers, fix the above symlink
  • Re-create the "chroot barrier": %%%setattr --barrier /home/vservers%%%showattr /home -> B for vservers
  • Some tools could have /var/lib/vservers hardcoded, for safety I create a symlink /var/lib/vservers pointing to /home/vservers

Manipulating vservers

Create a vserver

Edit /etc/vservers/newvserver-vars: 

# cf http://amd64.debian.net/README.mirrors.html
MIRROR="http://ftp.belnet.be/debian-amd64/debian"
INTERFACE="<my_if>"
ARCH="amd64"

Create a vserver with 64bits: 

LANG=C newvserver --hostname template64 --domain teuwen.org --ip <new_ip>/24 --dist etch

Create a vserver with 32bits emulation: 

LANG=C newvserver --hostname template32 --domain teuwen.org --ip <new_ip>/24 --dist etch --arch i386 --mirror "http://<i386_debian_mirror>"

Tuning:

  • take care of the config duplication!
  • enter the vserver and run tzconfig to choose the proper timezone
  • fix /etc/apt/sources.list
  • delete rcX.d links to umountroot
  • Warning! If you use newvserver as such, it will overwrite the host /etc/motd due to a symlink
  • See [Vserver tools] for a patch for newvserver

Removing unnecessary progs (check if you really don't need them!!):

  • aptitude apt-utils base-config cpio dselect tasksel libncursesw5 libsigc++-1.2-5c2 libsigc++-2.0-0c2a
  • dmidecode laptop-detect module-init-tools
  • bsdmainutils ed nano nvi
  • groff-base man-db manpages info libgdbm3
  • netcat traceroute wget libssl0.9.8
  • gettext-base libconsole libgnutls11 liblzo2-2 libtasn1-2-bin

Automatic start at bootup

echo default > /etc/vservers/<my_vserver>/apps/init/mark

Note that at shotdown all vservers will be stopped

Delete a vserver

Remove dirs /home/vservers/<my_vserver> (depends on the setting of vdirbase, cf. above), /etc/vservers/<my_vserver> and /var/run/vservers/<my_vserver> and the corresponding symlink in /var/run/vservers.rev

Config of a vserver

TODO 

?? /etc/vservers/<my_vserver>.conf
?? S_CAPS

see Detailed config page (better choosing boring CSS...)

Run a vserver

vserver <my_vserver> start
vserver <my_vserver> enter

If you get "mesg: /dev/pts/1: Operation not permitted", be root on the host with "su -" 

vserver <my_vserver> stop

Other tools

vserver <my_vserver> status
vserver-stat
vtop, vps, vpstree, vkill

/etc/rc.d/init.d/rebootmgr is a daemon which can be called from vservers via vreboot and vhalt to stop/restart the vserver from inside

See also compatibility of util-vserver alpha branch

See Vserver tools for my own/modified scripts

Duplicate a vserver

vserver <my_vserver1> stop
dupvserver --from <my_vserver1> --to <my_vserver2> --ip <new_ip>

dupvserver is broken with the new configuration structure /etc/vservers/<my_vserver>/
See Vserver tools for a patch for dupvserver

Move/copy a vserver

Basically stop the vserver and copy /etc/vservers/<my_vserver> and /home/vservers/<my_vserver>
E.g. rsync -e ssh -avHl /vservers/XX new-server:/vserver/XX

Share directories

To mount a directory from one vserver into another from the host: 

vnamespace -e <vserver> mount --rbind /directory/to/mount/somewhere /where/to/mount/it
vnamespace -e <vserver> umount /where/it/was/mounted

or 

mount --bind /home /var/lib/vservers/vserver1/home
mount --bind /home /var/lib/vservers/vserver2/home

The second method had the disavantage to require a reboot of the vserver

Apt-get

LANG=C vapt-get <my_vserver1> <my_vserver2> <...> -- install <pkg1> <pkg2>

Unify

cf immutable-linkage-invert flag

Preparation: 

mkdir /etc/vservers/template64/apps/vunify
mkdir /etc/vservers/<my_vserver>/apps/vunify
ln -s /etc/vservers/template64 /etc/vservers/<my_vserver>/apps/vunify/refserver.template64

Unification:
Be sure both vservers are running 

vserver <my_vserver> unify [-n] [-R]

-n for dry run, no change
-R for de-unifying

When using tar, add option -U to unlink & recreate files instead of overwriting.
Manual set/unset of the immutable-linkage-invert flag: 

setattr --iunlink /my/file
setattr --~iunlink /my/file

Disk limits

cf http://linux-vserver.org/Disk+Limits

  • Assign static contexts for the vservers (i.e. have a value between 2 and 49151 in /etc/vservers/<name>/context)
  • Mount the filesystem holding the vserver(s) with the tagxid option
    • Check if this is mounted properly: use cat /proc/mounts
      Ex.: /dev/mapper/Zeus-home /home reiserfs rw,tagxid 0 0
    • WARNING: if the filesystem is already in use with vservers, nothing prevent you to umount the filesystem while the vservers are still running, which is VERY BAD! Be careful.
    • I could only get the tagxid taken properly into account after a reboot
  • Change the xid of already existing files:
chxid -c <my_vserver> -R /home/vservers/<my_vserver>
  • Set limits, first method: here limit to 5Gb, 100000 inodes and 5% for the root user
    For info as I could not get it working properly yet
mkdir /var/cache/vservers
ln -s /var/cache/vservers /etc/vservers/.defaults/cachebase
mkdir /etc/vservers/.defaults/cachebase/<my_server>
ln -s /etc/vservers/.defaults/cachebase/<my_server> /etc/vservers/<my_server>/cache
mkdir -p /etc/vservers/<my_vserver>/dlimits/0
echo /home/vservers/<my_vserver> > /etc/vservers/<my_vserver>/dlimits/0/directory
echo $(( 5 * 1024 * 1024 )) > /etc/vservers/<my_vserver>/dlimits/0/space_total
echo 100000 > /etc/vservers/<my_vserver>/dlimits/0/inodes_total
echo 5 > /etc/vservers/<my_vserver>/dlimits/0/reserved
  • Set limits, second method: 
ln -s /usr/local/sbin/vdlimit_ /etc/vservers/<my_vserver>/scripts/post-start.d/vdlimit_$((5*1024))

Network

Intern network

For pure loopback, use dummy interface, cf http://mirabellug.org/wikini/wakka.php?wiki=VServers

For usable dummy interface, us permanent taps as the uml tools allow: 

apt-get install uml-utilities
  • Create a pseudo-interface:
auto tap0
iface tap0 inet static
    address 192.168.2.1
    netmask 255.255.255.0
    tunctl_user uml-net

And configure vservers with the same dev=tap0

Update: to check but actually all traffic with private or public IP will anyway be done through lo so this is probably not required

Configure daemons to listen only to the IP-address of the mothersystem

  • openbsd-inetd: (not netkit-inetd) in file /etc/inetd.conf:
    Prepend the service with <IP pub>:
    Example
<IP pub>:cvspserver       stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/sbin/cvs-pserver
  • xinetd: (not inetd) in file /etc/xinetd.conf:
defaults
{ bind = <IP pub> }

/etc/init.d/xinetd restart
  • sshd: in file /etc/ssh/sshd_config:
ListenAddress <IP pub>

/etc/init.d/ssh restart
  • exim4: in file /etc/exim4/update-exim4.conf.conf:
dc_local_interfaces='<IP pub>'

/etc/init.d/exim4 restart

Better to do it through debconf to avoid surprises at update time: dpkg-reconfigure exim4-config

  • courier-imap: in file /etc/courier/imapd:
ADDRESS=<IP pub>

/etc/init.d/courier-imap restart
  • courier-imap-ssl: in file /etc/courier-ssl/imapd:
ADDRESS=<IP pub>

/etc/init.d/courier-imap-ssl restart
  • imapproxy: in file /etc/imapproxy.conf:
listen_address <IP pub>

Within a vserver, you'll probably hav to reduce the cache_size or give capability to the vserver to raise the setrlimit.

  • mysql: in file /etc/mysql/my.cnf:
bind-address = <IP pub>
  • vsFtpd: in file /etc/vsftpd.conf:
listen_address=<IP pub>
  • postgresql: in file /etc/postgresql/postgresql.conf:
virtual_host = '<IP pub>'
  • apache2: in file /etc/apache2/ports.conf:
Listen <IP pub>:80
  • zope2.9: in file /etc/zope2.9/<instance>/zope.conf:
ip-address <IP pub>
  • portmap: in file /etc/default/portmap:
OPTIONS="-i <IP pub/loopback>"
  • dnsmasq: in file /etc/dnsmasq.conf:
listen-address=<IP pub>
bind-interfaces
  • netstat -lp -> other greedy daemons?
  • Seems that this is possible via another method, here it will bind the daemon to the first IP of the interface:
    exec /usr/sbin/chbind --ip eth0 /path/to/daemon

Add an interface without rebooting the vserver

  • add the ip to the host (ip addr add ...)
  • add the ip to the guest's network context
# naddress --add --nid <nid> --ip <ip>/<mask>
  • enter the guest (best via ssh)
  • restart the services if required
    (most services will automatically start using the new addresses)
  • update the config to reflect the changes for the next guest restart (if desired)

Thanks Herbert!

Understanding vservers

Security contextes

  • Find security context of process N:
chcontext --ctx 1 cat /proc/N/status|grep s_context
  • Be in the same context:
chcontext --ctx X /bin/sh
  • Master context: 1, example to get all listening ports:
chcontext --ctx 1 netstat -lpn

See also Virtual private servers and security contexts

Ceiling capabilities

  • As non-root, check capBset:
cat /proc/self/status
  • Reduce ceiling caps:
reducecap --secure /bin/sh
  • Now capBset is reduced:
cat /proc/self/status
su

Security

Not necessarily related to vserver but always useful to consider :-)

  • ssh
    • Use the AllowUsers option to give ssh rights only to those who need it.
    • Brute-force protection: apt-get install denyhosts
  • iptables (on the host)
    • cf --uid-owner and other --XXX-owner options
      on OUTPUT table to avoid download of malicious code
      on INPUT table to avoid bindshells
  • resource limits
    • cpu/mem

GrSec

Iptables Proxy

Other tricks

TODO

Retrieved from "https://wiki.yobi.be/index.php?title=Vserver_administration&oldid=1805"