Difference between revisions of "Vserver administration"

From YobiWiki
Jump to navigation Jump to search
m
Line 1: Line 1:
  +
==Introduction==
  +
Official homepage: [http://linux-vserver.org/ Linux VServer Project]
  +
  +
Good introduction:
  +
* [http://linux-vserver.org/index.php?page=Linux-VServer-Paper Linux-VServer Technology]
  +
* [http://linux-vserver.org/index.php?page=Linux-VServer-Paper-French La Technologie Linux-VServer]
  +
  +
Debian support:
  +
apt-cache search vserver
  +
kernel-patch-vserver - context switching virtual private servers - kernel patch
  +
[http://www.nongnu.org/util-vserver/ util-vserver] - tools for Virtual private servers and context switching
  +
vserver-debiantools - Tools to manage debian virtual servers
  +
  +
Misc:
  +
* [http://www.lri.fr/~fragile/IMG/pdf/Quetier.pdf Benchmark Comparisons between UML, VMWare, vserver and Xen (pdf)]
  +
  +
==Kernel compilation==
  +
===The Debian way===
  +
I followed instructions given in
  +
* /usr/share/doc/kernel-patch-vserver/README.Debian
  +
* [http://linux-vserver.org/Step-by-Step+Guide+2.6 Step-by-step 2.6]
  +
* [http://deb.riseup.net/vserver/preparing/ Debian vservers]
  +
* [http://arnofear.free.fr/linux/vserver-1.php Debian and vserver, french howto]
  +
* [http://lena.franken.de/linux/debian_and_vserver/ Debian and vserver]
  +
<pre>
  +
apt-get install kernel-patch-vserver linux-source-2.6.16 kernel-package fakeroot
  +
cd /usr/src
  +
tar xjf linux-source-2.6.16.tar.bz2
  +
cd /usr/src/linux-source-2.6.16
  +
cp config-2.6.16-1-amd64-k8 .config
  +
export PATCH_THE_KERNEL=YES
  +
make-kpkg --rootcmd fakeroot \
  +
--revision custom01 \
  +
--added-patches vserver \
  +
--append-to-version +vserver \
  +
--initrd \
  +
binary-arch
  +
"Virtual root device support" -> **y**
  +
"Legacy kernel API" -> y
  +
"Show a Legacy Version ID" -> n
  +
"Disable Legacy Networking Kernel API" -> n
  +
"Enable Proc Security" -> y
  +
"Enable Hard CPU Limits" -> y
  +
"Limit the IDLE task" -> n
  +
"Persistent Inode Context Tagging" -> UID24/GID24 (32/32 probably not yet supported on Reiserfs)
  +
"Tag NFSD User Auth and Files" -> n
  +
"VServer Debugging Code" -> n
  +
</pre>
  +
Install kernel and reboot
  +
===Vanilla with GrSec, still the Debian way===
  +
I used linux-2.6.17.14.tar.bz2 + patch-2.6.17.14-vs2.0.2.1-grsec2.1.9.diff
  +
<br>and the config of the Debian kernel config-2.6.17-2-vserver-amd64
  +
make oldconfig
  +
I activated HARDCPU limits and misc PAX & GRSEC stuff ([http://people.linux-vserver.org/~harry/_README_ this page] can help):
  +
<pre>
  +
CONFIG_VSERVER_HARDCPU=y
  +
CONFIG_VSERVER_HARDCPU_IDLE=y
  +
CONFIG_PAX=y
  +
CONFIG_PAX_SOFTMODE=y
  +
CONFIG_PAX_PT_PAX_FLAGS=y
  +
CONFIG_PAX_HAVE_ACL_FLAGS=y
  +
CONFIG_PAX_NOEXEC=y
  +
CONFIG_PAX_PAGEEXEC=y
  +
CONFIG_PAX_MPROTECT=y
  +
CONFIG_PAX_ASLR=y
  +
CONFIG_PAX_RANDUSTACK=y
  +
CONFIG_PAX_RANDMMAP=y
  +
CONFIG_PAX_MEMORY_SANITIZE=y
  +
CONFIG_GRKERNSEC=y
  +
CONFIG_GRKERNSEC_CUSTOM=y
  +
CONFIG_GRKERNSEC_KMEM=y
  +
CONFIG_GRKERNSEC_IO=y
  +
CONFIG_GRKERNSEC_PROC_MEMMAP=y
  +
CONFIG_GRKERNSEC_BRUTE=y
  +
CONFIG_GRKERNSEC_MODSTOP=y
  +
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
  +
CONFIG_GRKERNSEC_ACL_MAXTRIES=3
  +
CONFIG_GRKERNSEC_ACL_TIMEOUT=30
  +
CONFIG_GRKERNSEC_PROC=y
  +
CONFIG_GRKERNSEC_PROC_USER=y
  +
CONFIG_GRKERNSEC_PROC_ADD=y
  +
CONFIG_GRKERNSEC_LINK=y
  +
CONFIG_GRKERNSEC_FIFO=y
  +
CONFIG_GRKERNSEC_CHROOT=y
  +
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
  +
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
  +
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
  +
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
  +
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
  +
CONFIG_GRKERNSEC_CHROOT_UNIX=y
  +
CONFIG_GRKERNSEC_CHROOT_NICE=y
  +
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
  +
CONFIG_GRKERNSEC_RESLOG=y
  +
CONFIG_GRKERNSEC_SIGNAL=y
  +
CONFIG_GRKERNSEC_FORKFAIL=y
  +
CONFIG_GRKERNSEC_PROC_IPADDR=y
  +
CONFIG_GRKERNSEC_EXECVE=y
  +
CONFIG_GRKERNSEC_SHM=y
  +
CONFIG_GRKERNSEC_DMESG=y
  +
CONFIG_GRKERNSEC_RANDPID=y
  +
CONFIG_GRKERNSEC_RANDNET=y
  +
CONFIG_GRKERNSEC_SYSCTL=y
  +
CONFIG_GRKERNSEC_FLOODTIME=10
  +
CONFIG_GRKERNSEC_FLOODBURST=4
  +
</pre>
  +
make-kpkg --rootcmd fakeroot --us --uc --initrd kernel-image
  +
And I got a linux-image-2.6.17.14-grsec2.1.9-vs2.0.2.1_2.6.17.14-grsec2.1.9-vs2.0.2.1-10.00.Custom_amd64.deb
  +
==Host preparation==
  +
<pre>
  +
apt-get install util-vserver vserver-debiantools
  +
wget http://vserver.13thfloor.at/Stuff/SCRIPT/testme.sh
  +
chmod +x testme.sh
  +
./testme.sh
  +
dd bs=1024k count=1024 if=/dev/zero of=1gb.test
  +
modprobe loop
  +
losetup /dev/loop0 ./1gb.test
  +
./testfs.sh [ -F reiser ] -D /dev/loop0 -M /mnt
  +
losetup -d /dev/loop0
  +
modprobe -r loop
  +
</pre>
  +
There is no error at this point but as I'm using Reiserfs, I have to activate manually the extended attributes (for lsattr/chattr) by adding the following option to /etc/fstab lines: "attrs" (?? also option acl ??)
  +
<br>Test: lsattr <mount point of a Reiserfs>
  +
===Change the vserver base path===
  +
* /etc/vservers/.defaults/vdirbase -> /var/lib/vservers
  +
* I change it to /home/vservers, fix the above symlink
  +
* Re-create the "chroot barrier": %%%setattr --barrier /home/vservers%%%showattr /home -> B for vservers
  +
* Some tools could have /var/lib/vservers hardcoded, for safety I create a symlink /var/lib/vservers pointing to /home/vservers
  +
==Manipulating vservers==
  +
===Create a vserver===
  +
Edit /etc/vservers/newvserver-vars:
  +
<pre>
  +
# cf http://amd64.debian.net/README.mirrors.html
  +
MIRROR="http://ftp.belnet.be/debian-amd64/debian"
  +
INTERFACE="<my_if>"
  +
ARCH="amd64"
  +
</pre>
  +
Create a vserver with 64bits:
  +
LANG=C newvserver --hostname template64 --domain teuwen.org --ip <new_ip>/24 --dist etch
  +
Create a vserver with 32bits emulation:
  +
LANG=C newvserver --hostname template32 --domain teuwen.org --ip <new_ip>/24 --dist etch --arch i386 --mirror "http://<i386_debian_mirror>"
  +
Tuning:
  +
* take care of the config duplication!
  +
* enter the vserver and run tzconfig to choose the proper timezone
  +
* fix /etc/apt/sources.list
  +
* delete rcX.d links to umountroot
  +
* Warning! If you use newvserver as such, it will overwrite the host /etc/motd due to a symlink
  +
* See [Vserver tools] for a patch for newvserver
  +
Removing unnecessary progs (check if you really don't need them!!):
  +
* aptitude apt-utils base-config cpio dselect tasksel libncursesw5 libsigc++-1.2-5c2 libsigc++-2.0-0c2a
  +
* dmidecode laptop-detect module-init-tools
  +
* bsdmainutils ed nano nvi
  +
* groff-base man-db manpages info libgdbm3
  +
* netcat traceroute wget libssl0.9.8
  +
* gettext-base libconsole libgnutls11 liblzo2-2 libtasn1-2-bin
  +
  +
===Automatic start at bootup===
  +
echo default > /etc/vservers/<my_vserver>/apps/init/mark
  +
Note that at shotdown all vservers will be stopped
  +
===Delete a vserver===
  +
Remove dirs /home/vservers/<my_vserver> (depends on the setting of vdirbase, cf. above), /etc/vservers/<my_vserver> and /var/run/vservers/<my_vserver> and the corresponding symlink in /var/run/vservers.rev
  +
===Config of a vserver===
  +
''TODO''
  +
?? /etc/vservers/<my_vserver>.conf
  +
?? S_CAPS
  +
see [http://www.nongnu.org/util-vserver/doc/conf/configuration.html Detailed config page (better choosing boring CSS...)]
  +
===Run a vserver===
  +
vserver <my_vserver> start
  +
vserver <my_vserver> enter
  +
If you get "mesg: /dev/pts/1: Operation not permitted", be root on the host with "su -"
  +
vserver <my_vserver> stop
  +
===Other tools===
  +
vserver <my_vserver> status
  +
vserver-stat
  +
vtop, vps, vpstree, vkill
  +
/etc/rc.d/init.d/rebootmgr is a daemon which can be called from vservers via vreboot and vhalt to stop/restart the vserver from inside
  +
  +
See also [http://www.nongnu.org/util-vserver/doc/conf/compatibility.html compatibility of util-vserver alpha branch]
  +
  +
See [[Vserver tools]] for my own/modified scripts
  +
  +
===Duplicate a vserver===
  +
vserver <my_vserver1> stop
  +
dupvserver --from <my_vserver1> --to <my_vserver2> --ip <new_ip>
  +
dupvserver is broken with the new configuration structure /etc/vservers/<my_vserver>/
  +
<br>See [[Vserver tools]] for a patch for dupvserver
  +
===Move/copy a vserver===
  +
Basically stop the vserver and copy /etc/vservers/<my_vserver> and /home/vservers/<my_vserver>
  +
<br>E.g. rsync -e ssh -avHl /vservers/XX new-server:/vserver/XX
  +
==Share directories==
  +
To mount a directory from one vserver into another from the host:
  +
vnamespace -e <vserver> mount --rbind /directory/to/mount/somewhere /where/to/mount/it
  +
vnamespace -e <vserver> umount /where/it/was/mounted
  +
  +
or
  +
mount --bind /home /var/lib/vservers/vserver1/home
  +
mount --bind /home /var/lib/vservers/vserver2/home
  +
The second method had the disavantage to require a reboot of the vserver
  +
==Apt-get==
  +
LANG=C vapt-get <my_vserver1> <my_vserver2> <...> -- install <pkg1> <pkg2>
  +
==Unify==
  +
cf immutable-linkage-invert flag
  +
  +
Preparation:
  +
mkdir /etc/vservers/template64/apps/vunify
  +
mkdir /etc/vservers/<my_vserver>/apps/vunify
  +
ln -s /etc/vservers/template64 /etc/vservers/<my_vserver>/apps/vunify/refserver.template64
  +
Unification:
  +
<br>Be sure both vservers are running
  +
vserver <my_vserver> unify [-n] [-R]
  +
-n for dry run, no change
  +
<br>-R for de-unifying
  +
  +
When using tar, add option -U to unlink & recreate files instead of overwriting.
  +
<br>Manual set/unset of the immutable-linkage-invert flag:
  +
setattr --iunlink /my/file
  +
setattr --~iunlink /my/file
  +
==Disk limits==
  +
cf http://linux-vserver.org/Disk+Limits
  +
  +
* Assign static contexts for the vservers (i.e. have a value between 2 and 49151 in /etc/vservers/<name>/context)
  +
* Mount the filesystem holding the vserver(s) with the tagxid option
  +
** Check if this is mounted properly: use cat /proc/mounts<br>Ex.: /dev/mapper/Zeus-home /home reiserfs rw,tagxid 0 0
  +
** WARNING: if the filesystem is already in use with vservers, nothing prevent you to umount the filesystem while the vservers are still running, which is VERY BAD! Be careful.
  +
** I could only get the tagxid taken properly into account after a reboot
  +
* Change the xid of already existing files:
  +
chxid -c <my_vserver> -R /home/vservers/<my_vserver>
  +
* Set limits, first method: here limit to 5Gb, 100000 inodes and 5% for the root user<br>For info as I could not get it working properly yet
  +
mkdir /var/cache/vservers
  +
ln -s /var/cache/vservers /etc/vservers/.defaults/cachebase
  +
mkdir /etc/vservers/.defaults/cachebase/<my_server>
  +
ln -s /etc/vservers/.defaults/cachebase/<my_server> /etc/vservers/<my_server>/cache
  +
mkdir -p /etc/vservers/<my_vserver>/dlimits/0
  +
echo /home/vservers/<my_vserver> > /etc/vservers/<my_vserver>/dlimits/0/directory
  +
echo $(( 5 * 1024 * 1024 )) > /etc/vservers/<my_vserver>/dlimits/0/space_total
  +
echo 100000 > /etc/vservers/<my_vserver>/dlimits/0/inodes_total
  +
echo 5 > /etc/vservers/<my_vserver>/dlimits/0/reserved
  +
* Set limits, second method:
  +
** Install my vdlimit_ script in /usr/local/sbin: [[Vserver tools]]
  +
ln -s /usr/local/sbin/vdlimit_ /etc/vservers/<my_vserver>/scripts/post-start.d/vdlimit_$((5*1024))
  +
==Network==
  +
===Intern network===
  +
For pure loopback, use dummy interface, cf http://mirabellug.org/wikini/wakka.php?wiki=VServers
  +
  +
For usable dummy interface, us permanent taps as the uml tools allow:
  +
apt-get install uml-utilities
  +
* Create a pseudo-interface:
  +
<pre>
  +
auto tap0
  +
iface tap0 inet static
  +
address 192.168.2.1
  +
netmask 255.255.255.0
  +
tunctl_user uml-net
  +
</pre>
  +
And configure vservers with the same dev=tap0
  +
  +
Update: to check but actually all traffic with private or public IP will anyway be done through lo so this is probably not required
  +
===Configure daemons to listen only to the IP-address of the mothersystem===
  +
* ''openbsd-inetd:'' (not netkit-inetd) in file /etc/inetd.conf:<br>Prepend the service with <IP pub>:<br>Example
  +
<IP pub>:cvspserver stream tcp nowait root /usr/sbin/tcpd /usr/sbin/cvs-pserver
  +
* ''xinetd:'' (not inetd) in file /etc/xinetd.conf:
  +
defaults
  +
{ bind = <IP pub> }
  +
  +
/etc/init.d/xinetd restart
  +
* ''sshd:'' in file /etc/ssh/sshd_config:
  +
ListenAddress <IP pub>
  +
  +
/etc/init.d/ssh restart
  +
* ''exim4:'' in file /etc/exim4/update-exim4.conf.conf:
  +
dc_local_interfaces='<IP pub>'
  +
  +
/etc/init.d/exim4 restart
  +
Better to do it through debconf to avoid surprises at update time: dpkg-reconfigure exim4-config
  +
* ''courier-imap:'' in file /etc/courier/imapd:
  +
ADDRESS=<IP pub>
  +
  +
/etc/init.d/courier-imap restart
  +
* ''courier-imap-ssl:'' in file /etc/courier-ssl/imapd:
  +
ADDRESS=<IP pub>
  +
  +
/etc/init.d/courier-imap-ssl restart
  +
* ''imapproxy:'' in file /etc/imapproxy.conf:
  +
listen_address <IP pub>
  +
Within a vserver, you'll probably hav to reduce the cache_size or give capability to the vserver to raise the setrlimit.
  +
* ''mysql:'' in file /etc/mysql/my.cnf:
  +
bind-address = <IP pub>
  +
* ''vsFtpd:'' in file /etc/vsftpd.conf:
  +
listen_address=<IP pub>
  +
* ''postgresql:'' in file /etc/postgresql/postgresql.conf:
  +
virtual_host = '<IP pub>'
  +
* ''apache2:'' in file /etc/apache2/ports.conf:
  +
Listen <IP pub>:80
  +
* ''zope2.9:'' in file /etc/zope2.9/<instance>/zope.conf:
  +
ip-address <IP pub>
  +
* ''portmap:'' in file /etc/default/portmap:
  +
OPTIONS="-i <IP pub/loopback>"
  +
* ''dnsmasq:'' in file /etc/dnsmasq.conf:
  +
listen-address=<IP pub>
  +
bind-interfaces
  +
* netstat -lp -> other greedy daemons?
  +
* Seems that this is possible via another method, here it will bind the daemon to the first IP of the interface:<br>exec /usr/sbin/chbind --ip eth0 /path/to/daemon
  +
===Add an interface without rebooting the vserver===
  +
* add the ip to the host (ip addr add ...)
  +
* add the ip to the guest's network context
  +
# naddress --add --nid <nid> --ip <ip>/<mask>
  +
* enter the guest (best via ssh)
  +
* restart the services if required<br>(most services will automatically start using the new addresses)
  +
* update the config to reflect the changes for the next guest restart (if desired)
  +
Thanks Herbert!
  +
==Understanding vservers==
  +
===Security contextes===
  +
* Find security context of process N:
  +
chcontext --ctx 1 cat /proc/N/status|grep s_context
  +
* Be in the same context:
  +
chcontext --ctx X /bin/sh
  +
* Master context: 1, example to get all listening ports:
  +
chcontext --ctx 1 netstat -lpn
  +
See also [http://www.solucorp.qc.ca/miscprj/s_context.hc Virtual private servers and security contexts]
  +
===Ceiling capabilities===
  +
* As non-root, check capBset:
  +
cat /proc/self/status
  +
* Reduce ceiling caps:
  +
reducecap --secure /bin/sh
  +
* Now capBset is reduced:
  +
cat /proc/self/status
  +
su
  +
* capEff raised a bit but not enough to do for example /sbin/ifconfig eth0 down
  +
* See also [Capabilities in Linux|http://www.lids.org/lids-howto/node34.html]
  +
  +
==Security==
  +
Not necessarily related to vserver but always useful to consider :-)
  +
*ssh
  +
**Use the AllowUsers option to give ssh rights only to those who need it.
  +
**Brute-force protection: apt-get install denyhosts
  +
*iptables (on the host)
  +
**cf --uid-owner and other --XXX-owner options<br>on OUTPUT table to avoid download of malicious code<br>on INPUT table to avoid bindshells
  +
*resource limits
  +
** cpu/mem
   
 
===GrSec===
  
===GrSec===
 
* http://people.linux-vserver.org/~harry/_README_
  
* http://people.linux-vserver.org/~harry/_README_
−
===Iptables Proxy===
 +
==Iptables Proxy==
 
* http://www.virtuaserver.com.br/forum/viewtopic.php?t=130
  
* http://www.virtuaserver.com.br/forum/viewtopic.php?t=130
  +
  +
===Other tricks===
  +
* For other tweaks, see http://deb.riseup.net/vserver/usage/ :
  +
** What if I accidentally removed a vserver while it was running?
  +
** Howto convert legacy vservers to the new format
  +
** Howto add an IP to a running vserver, without restarting it?
  +
** Howto make the host interface and IP available in a vserver
  +
** Howto impose disk limits in each vserver
  +
* http://www.paul.sladen.org/vserver/faq
  +
* [http://linux-vserver.org/ProblematicPrograms Problematic programs]
  +
  +
==TODO==
  +
* http://www.nongnu.org/util-vserver/doc/conf/compatibility.html
  +
* http://linux-vserver.derjohn.de/
  +
* [VServer wiki|http://vserver.strahlungsfrei.de/tiki-index.php]
  +
* [Administrator Guide|http://linux-vserver.org/linux-vserver_administrators_gide]
  +
* [Debian newvserver|http://www.paul.sladen.org/vserver/debian/]
  +
* [Howto Debian vserver|http://www.howtoforge.com/linux_vserver_debian]
  +
* ?? apt-get install vlan
  +
* ?? ipac-ng
  +
* With grsecurity?
  +
** http://linux-vserver.org/grsecurityHowto
  +
** http://team.lea-linux.org/bgigon/vserver/mirror/ChangeLog
  +
** http://pax.grsecurity.net/ apparently not yet stable for 2.6 and not yet available for amd64
  +
** Interessant: http://ludit.kuleuven.be/software/vserver/_README_
  +
* CPU limit
  +
** http://linux-vserver.org/Linux-VServer-Paper-06
  +
** http://list.linux-vserver.org/archive/vserver/msg08134.html
  +
* BW limit
  +
** http://lartc.org/howto/
  +
* http://linux-vserver.org/HowTo+Read+ProcFS
  +
* http://linux-vserver.org/HistoryList?full=1
  +
* Publish Munin scripts
  +
* http://linux-vserver.org/VServer+installation+Fedora+Core+5
  +
* http://vserver.13thfloor.at/Experimental/
  +
* http://www.archivesat.com/Linux-VServer/
  +
* http://www.solucorp.qc.ca/miscprj/s_context.hc?s1=1&s2=0&s3=0&s4=0&full=0&prjstate=1&nodoc=0
  +
* (fr) http://fr.wikibooks.org/wiki/Vserver

Revision as of 22:24, 17 February 2007

Introduction

Official homepage: Linux VServer Project

Good introduction:

Debian support: 

apt-cache search vserver
  kernel-patch-vserver - context switching virtual private servers - kernel patch
  util-vserver - tools for Virtual private servers and context switching
  vserver-debiantools - Tools to manage debian virtual servers

Misc:

Kernel compilation

The Debian way

I followed instructions given in 

apt-get install kernel-patch-vserver linux-source-2.6.16 kernel-package fakeroot
cd /usr/src
tar xjf linux-source-2.6.16.tar.bz2
cd /usr/src/linux-source-2.6.16
cp config-2.6.16-1-amd64-k8 .config
export PATCH_THE_KERNEL=YES
make-kpkg --rootcmd fakeroot \
        --revision custom01 \
        --added-patches vserver \
        --append-to-version +vserver \
        --initrd \
        binary-arch
"Virtual root device support" -> **y**
"Legacy kernel API" -> y
"Show a Legacy Version ID" -> n
"Disable Legacy Networking Kernel API" -> n
"Enable Proc Security" -> y
"Enable Hard CPU Limits" -> y
"Limit the IDLE task" -> n
"Persistent Inode Context Tagging" -> UID24/GID24 (32/32 probably not yet supported on Reiserfs)
"Tag NFSD User Auth and Files" -> n
"VServer Debugging Code" -> n

Install kernel and reboot

Vanilla with GrSec, still the Debian way

I used linux-2.6.17.14.tar.bz2 + patch-2.6.17.14-vs2.0.2.1-grsec2.1.9.diff
and the config of the Debian kernel config-2.6.17-2-vserver-amd64 

make oldconfig

I activated HARDCPU limits and misc PAX & GRSEC stuff (this page can help): 

CONFIG_VSERVER_HARDCPU=y
CONFIG_VSERVER_HARDCPU_IDLE=y
CONFIG_PAX=y
CONFIG_PAX_SOFTMODE=y
CONFIG_PAX_PT_PAX_FLAGS=y
CONFIG_PAX_HAVE_ACL_FLAGS=y
CONFIG_PAX_NOEXEC=y
CONFIG_PAX_PAGEEXEC=y
CONFIG_PAX_MPROTECT=y
CONFIG_PAX_ASLR=y
CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y
CONFIG_PAX_MEMORY_SANITIZE=y
CONFIG_GRKERNSEC=y
CONFIG_GRKERNSEC_CUSTOM=y
CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_IO=y
CONFIG_GRKERNSEC_PROC_MEMMAP=y
CONFIG_GRKERNSEC_BRUTE=y
CONFIG_GRKERNSEC_MODSTOP=y
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=3
CONFIG_GRKERNSEC_ACL_TIMEOUT=30
CONFIG_GRKERNSEC_PROC=y
CONFIG_GRKERNSEC_PROC_USER=y
CONFIG_GRKERNSEC_PROC_ADD=y
CONFIG_GRKERNSEC_LINK=y
CONFIG_GRKERNSEC_FIFO=y
CONFIG_GRKERNSEC_CHROOT=y
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
CONFIG_GRKERNSEC_CHROOT_UNIX=y
CONFIG_GRKERNSEC_CHROOT_NICE=y
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
CONFIG_GRKERNSEC_RESLOG=y
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y
CONFIG_GRKERNSEC_PROC_IPADDR=y
CONFIG_GRKERNSEC_EXECVE=y
CONFIG_GRKERNSEC_SHM=y
CONFIG_GRKERNSEC_DMESG=y
CONFIG_GRKERNSEC_RANDPID=y
CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_SYSCTL=y
CONFIG_GRKERNSEC_FLOODTIME=10
CONFIG_GRKERNSEC_FLOODBURST=4

make-kpkg --rootcmd fakeroot --us --uc --initrd kernel-image

And I got a linux-image-2.6.17.14-grsec2.1.9-vs2.0.2.1_2.6.17.14-grsec2.1.9-vs2.0.2.1-10.00.Custom_amd64.deb

Host preparation

apt-get install util-vserver vserver-debiantools
wget http://vserver.13thfloor.at/Stuff/SCRIPT/testme.sh
chmod +x testme.sh
./testme.sh
dd bs=1024k count=1024 if=/dev/zero of=1gb.test
modprobe loop
losetup /dev/loop0 ./1gb.test
./testfs.sh [ -F reiser ] -D /dev/loop0 -M /mnt
losetup -d /dev/loop0
modprobe -r loop

There is no error at this point but as I'm using Reiserfs, I have to activate manually the extended attributes (for lsattr/chattr) by adding the following option to /etc/fstab lines: "attrs" (?? also option acl ??)
Test: lsattr <mount point of a Reiserfs>

Change the vserver base path

  • /etc/vservers/.defaults/vdirbase -> /var/lib/vservers
  • I change it to /home/vservers, fix the above symlink
  • Re-create the "chroot barrier": %%%setattr --barrier /home/vservers%%%showattr /home -> B for vservers
  • Some tools could have /var/lib/vservers hardcoded, for safety I create a symlink /var/lib/vservers pointing to /home/vservers

Manipulating vservers

Create a vserver

Edit /etc/vservers/newvserver-vars: 

# cf http://amd64.debian.net/README.mirrors.html
MIRROR="http://ftp.belnet.be/debian-amd64/debian"
INTERFACE="<my_if>"
ARCH="amd64"

Create a vserver with 64bits: 

LANG=C newvserver --hostname template64 --domain teuwen.org --ip <new_ip>/24 --dist etch

Create a vserver with 32bits emulation: 

LANG=C newvserver --hostname template32 --domain teuwen.org --ip <new_ip>/24 --dist etch --arch i386 --mirror "http://<i386_debian_mirror>"

Tuning:

  • take care of the config duplication!
  • enter the vserver and run tzconfig to choose the proper timezone
  • fix /etc/apt/sources.list
  • delete rcX.d links to umountroot
  • Warning! If you use newvserver as such, it will overwrite the host /etc/motd due to a symlink
  • See [Vserver tools] for a patch for newvserver

Removing unnecessary progs (check if you really don't need them!!):

  • aptitude apt-utils base-config cpio dselect tasksel libncursesw5 libsigc++-1.2-5c2 libsigc++-2.0-0c2a
  • dmidecode laptop-detect module-init-tools
  • bsdmainutils ed nano nvi
  • groff-base man-db manpages info libgdbm3
  • netcat traceroute wget libssl0.9.8
  • gettext-base libconsole libgnutls11 liblzo2-2 libtasn1-2-bin

Automatic start at bootup

echo default > /etc/vservers/<my_vserver>/apps/init/mark

Note that at shotdown all vservers will be stopped

Delete a vserver

Remove dirs /home/vservers/<my_vserver> (depends on the setting of vdirbase, cf. above), /etc/vservers/<my_vserver> and /var/run/vservers/<my_vserver> and the corresponding symlink in /var/run/vservers.rev

Config of a vserver

TODO 

?? /etc/vservers/<my_vserver>.conf
?? S_CAPS

see Detailed config page (better choosing boring CSS...)

Run a vserver

vserver <my_vserver> start
vserver <my_vserver> enter

If you get "mesg: /dev/pts/1: Operation not permitted", be root on the host with "su -" 

vserver <my_vserver> stop

Other tools

vserver <my_vserver> status
vserver-stat
vtop, vps, vpstree, vkill

/etc/rc.d/init.d/rebootmgr is a daemon which can be called from vservers via vreboot and vhalt to stop/restart the vserver from inside

See also compatibility of util-vserver alpha branch

See Vserver tools for my own/modified scripts

Duplicate a vserver

vserver <my_vserver1> stop
dupvserver --from <my_vserver1> --to <my_vserver2> --ip <new_ip>

dupvserver is broken with the new configuration structure /etc/vservers/<my_vserver>/
See Vserver tools for a patch for dupvserver

Move/copy a vserver

Basically stop the vserver and copy /etc/vservers/<my_vserver> and /home/vservers/<my_vserver>
E.g. rsync -e ssh -avHl /vservers/XX new-server:/vserver/XX

Share directories

To mount a directory from one vserver into another from the host: 

vnamespace -e <vserver> mount --rbind /directory/to/mount/somewhere /where/to/mount/it
vnamespace -e <vserver> umount /where/it/was/mounted

or 

mount --bind /home /var/lib/vservers/vserver1/home
mount --bind /home /var/lib/vservers/vserver2/home

The second method had the disavantage to require a reboot of the vserver

Apt-get

LANG=C vapt-get <my_vserver1> <my_vserver2> <...> -- install <pkg1> <pkg2>

Unify

cf immutable-linkage-invert flag

Preparation: 

mkdir /etc/vservers/template64/apps/vunify
mkdir /etc/vservers/<my_vserver>/apps/vunify
ln -s /etc/vservers/template64 /etc/vservers/<my_vserver>/apps/vunify/refserver.template64

Unification:
Be sure both vservers are running 

vserver <my_vserver> unify [-n] [-R]

-n for dry run, no change
-R for de-unifying

When using tar, add option -U to unlink & recreate files instead of overwriting.
Manual set/unset of the immutable-linkage-invert flag: 

setattr --iunlink /my/file
setattr --~iunlink /my/file

Disk limits

cf http://linux-vserver.org/Disk+Limits

  • Assign static contexts for the vservers (i.e. have a value between 2 and 49151 in /etc/vservers/<name>/context)
  • Mount the filesystem holding the vserver(s) with the tagxid option
    • Check if this is mounted properly: use cat /proc/mounts
      Ex.: /dev/mapper/Zeus-home /home reiserfs rw,tagxid 0 0
    • WARNING: if the filesystem is already in use with vservers, nothing prevent you to umount the filesystem while the vservers are still running, which is VERY BAD! Be careful.
    • I could only get the tagxid taken properly into account after a reboot
  • Change the xid of already existing files:
chxid -c <my_vserver> -R /home/vservers/<my_vserver>
  • Set limits, first method: here limit to 5Gb, 100000 inodes and 5% for the root user
    For info as I could not get it working properly yet
mkdir /var/cache/vservers
ln -s /var/cache/vservers /etc/vservers/.defaults/cachebase
mkdir /etc/vservers/.defaults/cachebase/<my_server>
ln -s /etc/vservers/.defaults/cachebase/<my_server> /etc/vservers/<my_server>/cache
mkdir -p /etc/vservers/<my_vserver>/dlimits/0
echo /home/vservers/<my_vserver> > /etc/vservers/<my_vserver>/dlimits/0/directory
echo $(( 5 * 1024 * 1024 )) > /etc/vservers/<my_vserver>/dlimits/0/space_total
echo 100000 > /etc/vservers/<my_vserver>/dlimits/0/inodes_total
echo 5 > /etc/vservers/<my_vserver>/dlimits/0/reserved
  • Set limits, second method: 
ln -s /usr/local/sbin/vdlimit_ /etc/vservers/<my_vserver>/scripts/post-start.d/vdlimit_$((5*1024))

Network

Intern network

For pure loopback, use dummy interface, cf http://mirabellug.org/wikini/wakka.php?wiki=VServers

For usable dummy interface, us permanent taps as the uml tools allow: 

apt-get install uml-utilities
  • Create a pseudo-interface:
auto tap0
iface tap0 inet static
    address 192.168.2.1
    netmask 255.255.255.0
    tunctl_user uml-net

And configure vservers with the same dev=tap0

Update: to check but actually all traffic with private or public IP will anyway be done through lo so this is probably not required

Configure daemons to listen only to the IP-address of the mothersystem

  • openbsd-inetd: (not netkit-inetd) in file /etc/inetd.conf:
    Prepend the service with <IP pub>:
    Example
<IP pub>:cvspserver       stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/sbin/cvs-pserver
  • xinetd: (not inetd) in file /etc/xinetd.conf:
defaults
{ bind = <IP pub> }

/etc/init.d/xinetd restart
  • sshd: in file /etc/ssh/sshd_config:
ListenAddress <IP pub>

/etc/init.d/ssh restart
  • exim4: in file /etc/exim4/update-exim4.conf.conf:
dc_local_interfaces='<IP pub>'

/etc/init.d/exim4 restart

Better to do it through debconf to avoid surprises at update time: dpkg-reconfigure exim4-config

  • courier-imap: in file /etc/courier/imapd:
ADDRESS=<IP pub>

/etc/init.d/courier-imap restart
  • courier-imap-ssl: in file /etc/courier-ssl/imapd:
ADDRESS=<IP pub>

/etc/init.d/courier-imap-ssl restart
  • imapproxy: in file /etc/imapproxy.conf:
listen_address <IP pub>

Within a vserver, you'll probably hav to reduce the cache_size or give capability to the vserver to raise the setrlimit.

  • mysql: in file /etc/mysql/my.cnf:
bind-address = <IP pub>
  • vsFtpd: in file /etc/vsftpd.conf:
listen_address=<IP pub>
  • postgresql: in file /etc/postgresql/postgresql.conf:
virtual_host = '<IP pub>'
  • apache2: in file /etc/apache2/ports.conf:
Listen <IP pub>:80
  • zope2.9: in file /etc/zope2.9/<instance>/zope.conf:
ip-address <IP pub>
  • portmap: in file /etc/default/portmap:
OPTIONS="-i <IP pub/loopback>"
  • dnsmasq: in file /etc/dnsmasq.conf:
listen-address=<IP pub>
bind-interfaces
  • netstat -lp -> other greedy daemons?
  • Seems that this is possible via another method, here it will bind the daemon to the first IP of the interface:
    exec /usr/sbin/chbind --ip eth0 /path/to/daemon

Add an interface without rebooting the vserver

  • add the ip to the host (ip addr add ...)
  • add the ip to the guest's network context
# naddress --add --nid <nid> --ip <ip>/<mask>
  • enter the guest (best via ssh)
  • restart the services if required
    (most services will automatically start using the new addresses)
  • update the config to reflect the changes for the next guest restart (if desired)

Thanks Herbert!

Understanding vservers

Security contextes

  • Find security context of process N:
chcontext --ctx 1 cat /proc/N/status|grep s_context
  • Be in the same context:
chcontext --ctx X /bin/sh
  • Master context: 1, example to get all listening ports:
chcontext --ctx 1 netstat -lpn

See also Virtual private servers and security contexts

Ceiling capabilities

  • As non-root, check capBset:
cat /proc/self/status
  • Reduce ceiling caps:
reducecap --secure /bin/sh
  • Now capBset is reduced:
cat /proc/self/status
su

Security

Not necessarily related to vserver but always useful to consider :-)

  • ssh
    • Use the AllowUsers option to give ssh rights only to those who need it.
    • Brute-force protection: apt-get install denyhosts
  • iptables (on the host)
    • cf --uid-owner and other --XXX-owner options
      on OUTPUT table to avoid download of malicious code
      on INPUT table to avoid bindshells
  • resource limits
    • cpu/mem

GrSec

Iptables Proxy

Other tricks

TODO

Retrieved from "https://wiki.yobi.be/index.php?title=Vserver_administration&oldid=1805"