ptrace¶

CERN has seen a number of compromised Linux machines during the last weeks. The previous Linux kernel had a vulnerability that allowed local users to run commands as "root". Once running with sysadmin privileges, attackers then have installed so called "root kits" that allow them to take over them machine again in the future, and to hide their presence.

Please check whether your machine has been compromised, then either upgrade the kernel to a secure version or reinstall with a secure version. Until you upgrade, please apply a hotfix to make sure that your machine does not get exploited in the mean time.

A new interim release (CERN Linux 7.3.2) is being prepared that contains the fixed kernel and all accumulated security updates against 7.3.1. If you re-install a machine, please use 7.3.2 on it.

1. How to check whether your machine has been compromised:

if you have AFS:

# cd /afs/cern.ch/project/security/cern/public/tools/check-rootkit
# ./crk

(see the README file there and "crk -h" for more details)

if you do not have AFS:
- the above check-rootkit directory can be copied to a local directory and the script run from there, please see the README file for details.
```
# scp -r username@lxplus:/afs/cern.ch/project/security/cern/public/tools/check-rootkit   /tmp/check-rootkit
# cd /tmp/check-rootkit
# ./crk
```
- Alternatively, a simple check for the rootkit ("suckit") seen lately at CERN is
```
# ls -li /sbin/init /sbin/telinit
```
  On a "good" machine, this should give /sbin/telinit as a symbolic link to init. With the rootkit, both appear as regular files, with the same inode number (1st number in output) and reference count (2nd number)=1.

If your machine has been compromised, please inform computer.security@cern.ch immediately, and pull out the network cable from the machine while leaving it running. A followup will be done to identify compromised user accounts.

2. How to upgrade your kernel

The base distribution from which CERN Linux is derived can be found by doing cat /etc/redhat-release. The currently running version of the Linux kernel can be found by running uname -r, and the processor type with uname -p.

on CERN Linux 7.3.1:

Base distribution: Red Hat Linux release 7.3 (Valhalla)
Previous kernel(s): 2.4.18-17.7.x, 2.4.18-18.7.x

Please follow the recipe published at http://linux.web.cern.ch/linux/

When upgrading OpenAFS, you may receive messages about editing files (for setting cache size) and turning on servers. Please ignore these, your old settings will continue to be valid.

on CERN Linux 7.2.1:

Base distribution: Red Hat Linux release 7.2 (Enigma)
Previous kernel(s): 2.4.9-31.1.cern

Please consider upgrading to 7.3.2. If you cannot, please try using the above recipe for 7.3.1. If this does not work in your configuration (it has received only very limited testing), you will need to move to 7.3.2.

Some additional packages that you may need to upgrade from the 7.3.2 repository, before being able to do the kernel upgrade itself (thanks to T.Wildish):

# cd /afs/cern.ch/project/linux/redhat/cern/7.3.2/i386/RedHat/RPMS/
# rpm -Fvh modutils-*rpm dev-*rpm iptables-*rpm xfsprogs-*rpm

on CERN Linux 6.1.1:

Base distribution: Red Hat Linux release 6.1 (Cartman)
Previous kernel(s): 2.2.19-6.2.1.1, 2.2.12-20

Please upgrade to 7.3.2 as soon as possible, since 6.1.1 will no longer be supported by CERN IT from end of June 2003 onwards (Red Hat has already stopped support for the 6.X series). The following recipe is provided for machines which are vitally important for an existing service, and which are regularly monitored for abnormal activity. Please do not use this to extend the lifetime of a 6.1.1-based desktop machine, this will create more problems for you in the future when no more updates will be made available:

# cd /afs/cern.ch/project/linux/redhat/cern/6.1.1/updates/

Upgrade the kernel RPMs that you have already installed. Please note that the kernel RPMs are provided for i386, i586, and i686, both for Multiprocessor (".smp" postfix) and Uniprocessor machines.

(you may want to test the installation first, in which case you should install the kernel in parallel to the one you already have, i.e. rpm -i instead of rpm -F)

# rpm -Fvh kernel-2.2.24-6.2.3.1.i686.rpm \
           kernel-headers-2.2.24-6.2.3.1.i386.rpm \
           kernel-utils-2.2.24-6.2.3.1.i386.rpm

In case you get errors about broken dependencies (e.g for mount), you should first upgrade the packages concerned from the main 6.1.1 package repository (/afs/cern.ch/project/linux/redhat/cern/6.1.1/RedHat/RPMS/). Most likely your system was not installed with CERN 6.1.1, but with some older distribution.

Edit /etc/lilo.conf to use the newly installed kernel. In case you need additional modules to boot (e.g. with a SCSI or RAID system disk), you can prepare an initial RAM disk with the following command:

# /sbin/mkinitrd /boot/initrd-2.2.24-6.2.3.1.img 2.2.24-6.2.3.1

Run LILO to update the boot information:

# /sbin/lilo

Upgrade AFS to a version that has modules for the new kernel:

# /usr/sue/etc/sue.install afs

Reboot and test.

3. Hotfix

This "hotfix" will disallow automatic module loading, which could break functionality. This "hotfix" needs re-applying at each reboot, before regular users get access to the machine. It is recommended that you first check which modules are actually needed in your configuration (use /sbin/lsmod to identify kernel modules loaded during operation), and preload these via explicit /sbin/modprobe commands.

# echo "/nomoremodules" > /proc/sys/kernel/modprobe

From now on, failed attempts to load modules will show up in /var/log/messages as e.g kernel: kmod: failed to exec /nomoredynamicmodules -s -k net-pf-10, errno = 2

You can get the same effect by placing the keyword nomodules on the kernel command line. To do this permanently, you will need to change the /etc/lilo.conf file (and re-run /sbin/lilo), or change /etc/grub.conf.