Debian stable Web Server step-by-step

This is a series of specific steps for setting up Debian Etch 5.0 as a nice PHP web server. You are welcome to correct any mistakes or errors, improve the content, or add anything that seems to be missing.

Run memtest overnight

First run Memtest86+ on the machine overnight, to test that the CPU and memory are error-free. Also run a CPU stress test for 5 to 10 minutes to check CPU is reliable and adequately cooled. The easiest way to download and run these memory and CPU testing tools is with the Ultimate Boot CD.

Also use System Rescue CD to check bad blocks on the hard drive, before trusting it with any data: badblocks -v -s -w -c 4096 /dev/sda

If the machine does not pass memtest and CPU stress and HDD tests without any errors or warnings at all, then do NOT proceed any further until this is rectified.

Install Debian Etch 4.0 base on an HP netserver 1000R with a NetRAID card.

• Boot from Debian 4.0r0 i386 Netinst CD
• Enter (default installation)
• Lang: English [default]
• Country: Australia
• Keymap: American English [default]
• Primary network interface: Eth0 [default]
• "Go back" after network autodetection
• Configure network manually
• 192.168.0.100 as IP address, and then take the defaults for netmask, gateway, and DNS.
• Hostname: netserver
• Domain name: YourHostName.com
• Partitioning method: Guided - yes entire disk [default]
• Disk to partition: "SCSI3 (1,0,0) (sda) - 18.2 Gb MegaRAID LD 0 RAID1 17G" [default, only choice]
• Partitioning scheme: All files in one partition [default]
• Finish partitioning, and write changes to disk [default]
• Yes, write changes to disk. [default]
• City: Sydney [default]
• Root password: standard
• Extra Username: nickj
• Extra Username Password: standard
• Use a network mirror: Yes [default]
• Australian mirror [default]
• use: ftp.au.debian.org [default]
• Proxy: leave blank [default]
• Participate in package user survey: No [default]
• Choose software to install: Web server + File server + Mail server + SQL database + Standard system
• Continue installing libc-client without maildir support: Yes [default]
• Workgroup name: WORKGROUP
• Modify smb.conf to use WINS settings for DHCP: Yes
• Install the GRUB boot loader to the master boot record: Yes [default]
• remove CD
• Continue [default]
• Machine reboots
• Check boots okay, login as root, check networking okay, check SSH running.
• SSH into the box, and can now do remaining stuff below remotely via SSH.

Update /etc/apt/sources.list.

"nano /etc/apt/sources.list", and comment out the "deb cdrom:" line or lines.

This makes all data come off the Internet, which is what we want.

Get the latest and greatest packages.

Get the latest and greatest packages (currently there are no updates, but later there probably will be).

aptitude update
aptitude upgrade
aptitude dist-upgrade

Remove unneeded or unwanted packages.

Remove unneeded or unwanted packages for a headless webserver, most especially those that talk to the network.

Note: if you are using the system as a Linux desktop, and especially if you are running Ubuntu, then either skip this step, or take care to make sure that the following doesn't remove packages needed for the desktop to work. For example, on Ubuntu, do NOT remove anything where it will also remove the "ubuntu-desktop" package (use 'q' to quit if aptitude says it is going to remove this).

# remove NFS
aptitude --purge remove nfs-common nfs-kernel-server 

# Remove appletalk compat:
aptitude --purge remove netatalk

# Remove PHP4 (we'll install PHP5 later)
aptitude --purge remove php4-common libapache2-mod-php4

# RPC daemon
aptitude --purge remove portmap

# Remove postgres (we'll install MySQL later)
aptitude --purge remove postgresql postgresql-client postgresql-common \
 postgresql-doc postgresql-contrib

# remove email related stuff :
aptitude --purge remove qpopper sa-exim uw-imapd

# remove ident:
aptitude --purge remove pidentd

# remove analog
aptitude --purge remove analog

# Remove unneeded packages:
aptitude --purge remove spamassassin spamc python-newt mutt libapache2-mod-perl2 \
                        libapache2-mod-python

These are steps that make sense on a webserver, but you should probably skip these on a laptop or Linux Desktop:

# DHCP and related functionality
aptitude --purge remove winbind dhcp3-client dhcp3-common 

# remove samba:
aptitude --purge remove samba samba-common samba-doc smbfs

Then purge the old RC files from uninstalled packages:

dpkg --purge $(COLUMNS=132 dpkg -l | grep ^rc | awk '{ print $2; }')

Need to close down open ports. Check what ports are open with these 3 commands, should be stripped back now:

nmap localhost
lsof -i
netstat -l

Install LAMP.

# Install LAMP:
aptitude install php5 php5-gd php5-cgi php5-mysql mysql-server mysql-client php5-curl lynx php-pear \
ca-certificates xml-core apache2 libapache2-mod-php5

Install extra useful software.

Installing extra useful software:

aptitude install nmap

# This will install the Exim V4 mail server - when prompted, select the "internet site; mail is sent and received directly using SMTP" configuration option :
aptitude install logcheck

aptitude install diffstat
aptitude install zip unzip
aptitude install subversion subversion-tools patch

# A java compiler:
aptitude install gcj make autoconf automake1.9

# Tidying web pages.
aptitude install tidy

# package info:
aptitude install grep-dctrl

# for debugging apache crashes:
aptitude install gdb

# useful utility
aptitude install curl

# For building PHP modules:
aptitude install php5-dev

# For converting HTML to text in a console-only environment
aptitude install html2text

# To be able to compile PHP from snaps.php.net
aptitude install flex libxml2-dev

# To get the "ip" command
aptitude install iproute

# Get the "dig" command:
aptitude install dnsutils

# Get the "cruft" command:
aptitude install cruft

# Get the "sloccount" command:
aptitude install sloccount

# Get the p7zip command:
aptitude install p7zip

# Install mailer for sending attachments
aptitude install mutt

# Other useful packages:
aptitude install bzip2 deborphan p7zip-full fakeroot indent fdutils xsltproc

# Get the "uprecords" command:
aptitude install uptimed

# S.M.A.R.T. monitoring tools (skip if running inside a virtual machine/box - cannot monitor disks from inside a guest - needs to be done by the host O/S).
aptitude install smartmontools
# Then enable smartd:
nano /etc/default/smartmontools
# ... and uncomment the start smartd line.

# Get pinfo, a better GUI for viewing info and manual pages.
aptitude install pinfo

# Get ddrescue, useful for recovering data from failing hard disks:
aptitude install dd_rescue

# Get the "lsb_release -a" command (not installed by default).
aptitude install lsb-release

Disable IPv6.

Disabling IPv6:

nano /etc/modprobe.d/aliases

and change this line:

- #alias net-pf-10 ipv6
+ alias net-pf-10 off

Then have to reboot for this change to take effect:

reboot

Can check has been successfully disabled with:

ip a | grep inet6

(should give no output when IPv6 disabled)

... and:

/usr/bin/lsof -i | grep LISTEN

Should only list "IPv4" entries.

... and:

netstat -l

Should not show "tcp6" and "udp6" after the reboot if it is working.

Stop console screen from blanking.

Stop console screen from blanking, due to power-saving:

nano /etc/console-tools/config

Change this line:

BLANK_TIME=30

To:

BLANK_TIME=0

Exim 4 configuration.

Exim 4 configuration:

Need to configure exim rather than remove it because the "at" package depends on a mail-transport-agent package.

dpkg-reconfigure exim4-config

Then choose these options, for a server in a data centre:

General type of mail configuration:  internet site; mail is sent and received directly using SMTP
System mail name: YourHostName.com   <Note: this should be a valid domain to reply to, so use this instead of the machine's name>
IP-addresses to listen on for incoming SMTP connections: 127.0.0.1
Other destinations for which mail is accepted:  <blank>
Domains to relay mail for: YourOtherHostName.com
machines to relay for:  <blank>
Keep number of DNS-queries minimal (Dial-on-Demand)?  No
Delivery method for local mail:  mbox format in /var/mail/
Split configuration into small files? No [default]

[this allows mail to be sent locally, but not by other hosts connecting to this one]

For a development or test server, on a broadband connection, have to use smarthosts, as most ISPs block port 25 for outgoing mail. For example, the Exim4 config for Telstra BigPond is the same as above, except for the following:

General type of mail configuration:  mail sent by smarthost; received via SMTP or fetchmail 
IP address or host name of the outgoing smarthost: mail-hub.bigpond.net.au
Hide local name in outgoing mail? No

If sending mail is not working:
To see the queue, and try to clear it out, do this:

exim4 -v -v -v -qff

To view the log messages, do this:

tail -f /var/log/exim4/mainlog

Customizing Apache configuration.

Customizing Apache:

nano /etc/apache2/apache2.conf

Make these two changes:

- ServerTokens Full
+ ServerTokens Prod

- ServerSignature On
+ ServerSignature Off

Comment this out:

#    Alias /icons/ "/usr/share/apache2/icons/"
#
#    <Directory "/usr/share/apache2/icons">
#        Options Indexes MultiViews
#        AllowOverride None
#        Order allow,deny
#        Allow from all
#    </Directory>

Save, and test config:

apache2ctl -t

Restart the web server:

/etc/init.d/apache2 force-reload

Then to test from the command line that these changes have taken effect:

telnet localhost 80
GET / HTTP/1.1
(press enter twice).

... and the response header should contain "Server: Apache", without the version number.

Removing some more unwanted things:

nano /etc/apache2/sites-available/default

Then change this line:

  <Directory /var/www/>
- Options Indexes FollowSymLinks MultiViews
+ Options -Indexes FollowSymLinks MultiViews

... and delete this section (special directory we don't use):

-        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
-        <Directory "/usr/lib/cgi-bin">
-                AllowOverride None
-                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
-                Order allow,deny
-                Allow from all
-        </Directory>

... and delete this section (another special directory we don't use):

-    Alias /doc/ "/usr/share/doc/"
-    <Directory "/usr/share/doc/">
-        Options Indexes MultiViews FollowSymLinks
-        AllowOverride None
-        Order deny,allow
-        Deny from all
-        Allow from 127.0.0.0/255.0.0.0 ::1/128
-    </Directory>

Turn server signature off on forbidden pages:

-        ServerSignature On
+        ServerSignature Off

Also: remove the bit about "apache2-default", and the 3 line comment above it.

Test config:

apache2ctl -t

Then reload apache so that it gets these changes:

/etc/init.d/apache2 reload

Note: if HTTPS has already been setup, then need to repeat the above steps for the SSL site configuration files too.

Then remove mod_autoindex, not needed:

a2dismod autoindex

Then reload apache so that it gets this change:

/etc/init.d/apache2 force-reload

To see a list of which Apache2 modules are enabled, do:

ls /etc/apache2/mods-enabled/

Can then remove unwanted modules like so:

a2dismod perl
a2dismod mod_python

Reload apache:

/etc/init.d/apache2 force-reload 

Enable the expires module:

a2enmod expires

Reload apache:

/etc/init.d/apache2 force-reload

Few more apache tweaks - allow index.php3 as index file - "nano /etc/apache2/mods-enabled/dir.conf", and change:

- DirectoryIndex index.html index.cgi index.pl index.php index.xhtml
+ DirectoryIndex index.html index.cgi index.pl index.php index.xhtml index.php3

OPTIONAL, MOST PEOPLE SHOULD SKIP THIS ITEM - Then "nano /etc/apache2/apache2.conf" , and below the ErrorDocument information, add this for handling 404 requests (e.g. http://bling/broken.php3444 )

ErrorDocument 404 /broken.php3

Then "nano /etc/apache2/mods-enabled/php5.conf", and change to prevent PHP .inc file source code from being visible (e.g.: http://IP-address/lib-functions.inc )

- AddType application/x-httpd-php .php .phtml .php3
+ AddType application/x-httpd-php .php .phtml .php3 .inc

Then check above syntax is OK:

apache2ctl -t

Then restart apache:

/etc/init.d/apache2 force-reload

Set up the HTTP / Apache auth / passwords. Needed to be prompted for a password when accessing directories with .htaccess and .htpasswd files.

nano /etc/apache2/sites-available/default

... and replace all occurrences like this, with the line below:

- AllowOverride None
+ AllowOverride AuthConfig

Test config:

apache2ctl -t

Then reload apache so that it gets these changes:

/etc/init.d/apache2 reload

Note: if HTTPS has already been setup, then need to repeat the above steps for the SSL site configuration files too. ( nano /etc/apache2/sites-available/ssl )

Set up Apache virtual hosting.

Setting up Apache virtual hosting:

Load the mass virtual hosting module:

a2enmod vhost_alias

Then "nano /etc/apache2/apache2.conf", and add this to the end of the file to enable mass virtual hosting:

# ---------------------------------------------------
# ---------------- Virtual Hosting ------------------
<IfModule mod_vhost_alias.c>

# Get the server name from the Host: header
UseCanonicalName Off

# The virtual document root is under /var/www/__name_of_whatever_was_requested__
VirtualDocumentRoot /var/www/%0

# Note: Regrettably, cannot set up separate SSL certs for each virtual host here (need different IPs or ports)
# see: http://www.mail-archive.com/modssl-users@modssl.org/msg15648.html

</IfModule>
# ---------------------------------------------------

Then check above syntax is OK:

apache2ctl -t

Then restart apache:

/etc/init.d/apache2 force-reload

Then try to access a page by the hostname (e.g. "http://IP-address"). Should get an error like "The requested URL / was not found on this server." Also a "tail -f /var/log/apache2/error.log" should show an entry like: "File does not exist: /var/www/IP-address". This indicates that virtual hosting is working.

Apache content compression.

Setting up Apache content compression:

Source.

To enable the required modules; mod_deflate is now used instead of libapache-mod-gzip:

a2enmod deflate

Load the headers module:

a2enmod headers

Then "nano /etc/apache2/sites-available/default", and add this to the end of the file (but inside the <VirtualHost> block) to enable mod_deflate, and turn on logging for it.

# ---------------------------------------------------
# -------------- Enabling Mod_Deflate ---------------
<IfModule mod_deflate.c>
  # Compress all content, manually excluding specified file types
  # place filter 'DEFLATE' on all outgoing content
  SetOutputFilter DEFLATE
  # exclude uncompressible content via file type
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|rar|zip)$ no-gzip

  # properly handle requests coming from behind proxies
  <IfModule mod_headers.c>
      Header append Vary User-Agent env=!dont-vary
  </IfModule>

  # Keep a log of compression ratio on each request
  DeflateFilterNote Input instream
  DeflateFilterNote Output outstream
  DeflateFilterNote Ratio ratio
  LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate
  CustomLog /var/log/apache2/deflate.log deflate

  # Properly handle old browsers that do not support compression
  BrowserMatch ^Mozilla/4 gzip-only-text/html
  BrowserMatch ^Mozilla/4\.0[678] no-gzip
  BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
</IfModule>
# ---------------------------------------------------

Then check above syntax is OK:

apache2ctl -t

Then restart apache:

/etc/init.d/apache2 force-reload

Can then check it is working by accessing some pages, and doing:

tail -f /var/log/apache2/deflate.log

Note: Have to paste this into /etc/apache2/sites-available/ssl too (if have created it), or otherwise just do HTTPS / SSL setup at the end, which makes things easier.

Configure php.ini.

Configuring PHP.ini (note: for Ubuntu rather than debian, the path is probably /etc/php5/cgi rather than /etc/php5/apache2)

cp /etc/php5/apache2/php.ini /etc/php5/apache2/orig-php.ini
nano /etc/php5/apache2/php.ini

... and make these changes:

- ;date.timezone =
+ date.timezone = Australia/Sydney

# change this line, otherwise get errors about assigning a global using a session value.
# ("PHP versions 4.2.3 and lower have an undocumented feature/bug that allows you to initialize
# a session variable in the global scope, albeit register_globals is disabled")
- session.bug_compat_warn = 1
+ session.bug_compat_warn = Off

# hide PHP in the headers
- expose_php = On
+ expose_php = Off

- max_execution_time = 30
+ max_execution_time = 300 ; Equals 5 mins

- memory_limit = 16M
+ memory_limit = 32M

- upload_max_filesize = 2M
+ upload_max_filesize = 10M

- allow_url_fopen = On
+ allow_url_fopen = 0

Then need to restart apache for changes to take effect:

/etc/init.d/apache2 force-reload

Then soft-link the command line php.ini to the apache php.ini, so that they can have a shared configuration:

# diff -u --ignore-all-space /etc/php5/cli/php.ini /etc/php5/apache2/php.ini
mv /etc/php5/cli/php.ini /etc/php5/cli/php.ini-orig
ln -s /etc/php5/apache2/php.ini /etc/php5/cli/php.ini
ls -al /etc/php5/cli/php.ini

Enable SSL in Apache 2.

Enabling SSL in Apache 2:

a2enmod ssl
echo "Listen 443" >> /etc/apache2/ports.conf
mkdir /etc/apache2/ssl
export RANDFILE=/dev/random
openssl req $@ -new -x509 -days 365 -nodes -out \
 /etc/apache2/ssl/apache.pem -keyout /etc/apache2/ssl/apache.pem

When prompts, will ask a series of questions. Some fictional answers:

Country: AU
State: NSW
City: Sydney
Organisation Name: FooBar Pty Ltd
Dept: YourHostName
YOUR name: www.YourHostName.com   [must be the host's fully qualified name, or will get a msg that the certificate doesn't match the site name]
Email address: webmaster@YourHostName.com

Then

chmod 600 /etc/apache2/ssl/apache.pem

This should come after the default sites-available config, in the previous sections, to avoid having to repeat steps for both non-SSL and for SSL.

cp /etc/apache2/sites-available/default /etc/apache2/sites-available/ssl

"nano /etc/apache2/sites-available/ssl", and make a few updates (add ":443" twice, and the two SSL lines) :

NameVirtualHost *:443
<virtualhost *:443>
   ServerAdmin webmaster@YourHostName.com

   SSLEngine On
   SSLCertificateFile /etc/apache2/ssl/apache.pem

   DocumentRoot .... (etc, rest of this file as per normal the non HTTPS sites)

Or, if have already bought a signed certificate from a CA (with the .CRT server cerificate file, and a .KEY private key file), then use this instead of the "SSLCertificateFile" line above:

   #   Server Certificate:
   #   Point SSLCertificateFile at a PEM encoded certificate.  If
   #   the certificate is encrypted, then you will be prompted for a
   #   pass phrase.  Note that a kill -HUP will prompt again. A test
   #   certificate can be generated with `make certificate' under
   #   built time.
   SSLCertificateFile /etc/apache2/ssl/server.crt

   #   Server Private Key:
   #   If the key is not combined with the certificate, use this
   #   directive to point at the key file.
   SSLCertificateKeyFile /etc/apache2/ssl/server.key

Then enable this site:

a2ensite ssl

Then reload apache with:

/etc/init.d/apache2 force-reload

Change the date.

To change the time and date, if it's wrong:

date --set 0:42:05
hwclock --systohc --utc

See the NTP step as the long-term solution to keeping time accurate.

Installing and configuring tripwire.

Installing and configuring tripwire:

aptitude install tripwire

Setup questions:

Do you wish to create/use your local key pass-phrase / site pass during installation? --> Yes to both
Rebuild files? --> Yes to both
Will probably also need to specify a local and a site passphrase. Record both of these, will need them later.

Initialize tripwire baseline with:

/usr/sbin/tripwire --init

This will generate a lot of errors. Now comment out the files / entries that are not on this system, or that generated errors:

nano /etc/tripwire/twpol.txt

Then run this command to apply the policy:

twadmin -m P /etc/tripwire/twpol.txt

Then reinitialize tripwire with this policy:

/usr/sbin/tripwire --init

Then repeat the above 3 steps until there are no errors. Can also add stop points to the twpol.txt file to exclude files, like so:

!/lib/init/rw;   # exclude this file - is on a different file system.

Then do a test system check (this should be completely or mostly empty):

/usr/sbin/tripwire --check

Then to update the tripwire database, such as when packages are upgraded or installed, do this:

/usr/sbin/tripwire -m c -I

Tweak MySQL configuration.

Edit configuration file:

nano /etc/mysql/my.cnf

... and add these settings to increase performance a bit:

# change from listed default of 16 to 64  :
- key_buffer              = 16M
+ key_buffer              = 64M
# uncomment, and change from default of 64 to 256:
- # table_cache             = 64
+ table_cache             = 256
# Add this on the line after the above
+ sort_buffer             = 4M

Also increase query cache a bit:

query_cache_limit       = 2M
query_cache_size        = 50M

Then to make changes take effect:

/etc/init.d/mysql restart

Installing and configuring portsentry.

installing and configuring portsentry:

aptitude install portsentry

Then:

nano /etc/portsentry/portsentry.conf

... and change to add some ignored ports:

- ADVANCED_EXCLUDE_TCP="113,139"
+ ADVANCED_EXCLUDE_TCP="113,139,25,445,135"

# Enable blocking:
- BLOCK_UDP="0"
- BLOCK_TCP="0"
+ BLOCK_UDP="1"
+ BLOCK_TCP="1"

# Make a little less likely to react:
- SCAN_TRIGGER="0"
+ SCAN_TRIGGER="2"

Then:

nano /etc/portsentry/portsentry.ignore.static

... and add:

# Put hosts in here you never want blocked. This includes the IP addresses
# of all local interfaces on the protected host (i.e virtual host, mult-home)
# Keep 127.0.0.1 and 0.0.0.0 to keep people from playing games.
127.0.0.1
0.0.0.0
# Add the local IP address, the gateway address, DNS addresses,
# addresses of hosts you know you will be connecting from, etc etc.
192.168.0.0/24

Then:

nano /etc/default/portsentry

... (will be an empty or non-existent file), and add / change the two lines to ATCP and AUDP modes (these are the inverse modes) :

TCP_MODE="atcp"
UDP_MODE="audp"

Can then reload portsentry by doing:

/etc/init.d/portsentry restart

Can then test with:

nmap 127.0.0.1

Note: when not using atcp and audp, portsentry will open lots of ports, which will show up with the above command.

Locking down the IP functionality.

Locking down the IP functionality to make the system behave sensibly: [from http://www.debian.org/doc/manuals/securing-debian-howto/ch4.en.html , section 4.17.3 ]:

nano /etc/sysctl.conf

... and add the following to the end of that file:

# Additional settings - adapted from the script contributed
# by Dariusz Puchala (see below)
# Ignore ICMP broadcasts
net/ipv4/icmp_echo_ignore_broadcasts = 1
#
# Ignore bogus ICMP errors
net/ipv4/icmp_ignore_bogus_error_responses = 1
# 
# Do not accept ICMP redirects (prevent MITM attacks)
net/ipv4/conf/all/accept_redirects = 0
# _or_
# Accept ICMP redirects only for gateways listed in our default
# gateway list (enabled by default)
# net/ipv4/conf/all/secure_redirects = 1
#
# Do not send ICMP redirects (we are not a router)
net/ipv4/conf/all/send_redirects = 0
#
# Do not forward IP packets (we are not a router)
# Note: Make sure that /etc/network/options has 'ip_forward=no'
net/ipv4/conf/all/forwarding = 0
#
# Enable TCP Syn Cookies
# Note: Make sure that /etc/network/options has 'syncookies=yes'
net/ipv4/tcp_syncookies = 1
#
# Log Martian Packets
# Commented out as this can create heavy load on server flooded with information:
# net/ipv4/conf/all/log_martians = 1
#
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
# Note: Make sure that /etc/network/options has 'spoofprotect=yes'
net/ipv4/conf/all/rp_filter = 1
#
# Do not accept IP source route packets (we are not a router)
net/ipv4/conf/all/accept_source_route = 0

... then do:

cat /proc/sys/net/ipv4/tcp_syncookies

... should say "0". Then do:

reboot

... then do again:

cat /proc/sys/net/ipv4/tcp_syncookies

... and if it worked, it should say "1".

limits.conf.

nano /etc/security/limits.conf

... and add these two lines:

*                hard    core            0
*                hard    nproc           150

This makes DOS attacks marginally harder - sets the core size to 0 K, and the maximum number of processes to 150.

Install bastille.

aptitude install bastille

On Debian 5 there are two files that need to be modified after installing the bastille package, so that it can run (otherwise get an error about not being a supported OS) -

• /usr/lib/Bastille/API.pm
• /usr/lib/Bastille/IOLoader.pm

Search for DB4.0 and you will see it grouped with the OS compatibility listings. Just add DB5.0 right after the DB4.0 and you're set. At least, it worked fine for me.

Then run:

InteractiveBastille

... to configure it. Some possible answers to questions:

accept terms
restrictive permissions on admin utils: no  [default]
disable SUID status for mount/unmount: yes [default]
disable SUID status for ping: yes [default]
disable SUID status for at: no   [differs from default, needed for my setup]
Should Bastille disable clear-text r-protocols that use IP-based authentication? [Y]  [default]
enforce password aging:  no [differs from default, don't want to have to do this]
restrict use to cron to admin accounts:  Yes. [default]
set default umask:  Yes. [default]
Umask: 002 - 
disallow root login on all ttys: n [default]
password-protect the GRUB prompt? [N] [default]
disable CTRL-ALT-DELETE rebooting? [N] [default]
password protect single user mode [Y] [default]
default-deny on TCP Wrappers and xinetd? [N]    [default]
ensure the telnet service does not run on this system? [y]  [default]
ensure inetd's FTP service does not run on this system? [y] [default]
display "Authorized Use" messages at log-in time? [Y]   [default]
disable the gcc compiler? [N] [default]
put limits on system resource usage? [N]  [default]   [would say yes to this next time, since did this manually above].
restrict console access to a small group of user accounts? [N] [default]
add additional logging? [Y]     [default]
remote logging host? [N]   [default]
install TMPDIR/TMP scripts? [N]    [default]
run the packet filtering script? [N]    [default] [asks all sorts of over-detailed Qs]
make changes? [y]        [default]

Install the latest SUN JDK using pinning.

Installing the latest SUN JDK using pinning:

There is a good explanation of pinning here: http://jaqque.sbih.org/kplug/apt-pinning.html

nano /etc/apt/sources.list

... and make like so:

# From Lenny (the next release) :
deb http://ftp.au.debian.org/debian/ lenny main contrib non-free
deb-src http://ftp.au.debian.org/debian/ lenny main
deb http://security.debian.org/ lenny/updates main contrib
deb-src http://security.debian.org/ lenny/updates main contrib

# From Etch (this release) :
deb http://ftp.au.debian.org/debian/ etch main contrib non-free
deb-src http://ftp.au.debian.org/debian/ etch main
deb http://security.debian.org/ etch/updates main contrib
deb-src http://security.debian.org/ etch/updates main contrib

Then:

nano /etc/apt/preferences

... and make like so (this says "always use stable in preference to testing, but if testing is the only source of a package, then use that"). :

Package: *
Pin: release a=stable
Pin-Priority: 700

Package: *
Pin: release a=testing
Pin-Priority: 650

Update the info about what's in the repository:

aptitude update

This should not offer to upgrade anything:

aptitude upgrade

Now install Sun's JDK 6. This will come from testing, because it is not in stable.

aptitude install sun-java6-jdk sun-java6-jre

... and say "yes, accept" when prompted about the terms and conditions.

To show the available JVMs & JDKs :

update-alternatives --display java

Check current default java version with:

java -version

Change the system default from GCJ to sun's JDK:

update-java-alternatives --verbose --set java-6-sun

Check version has been updated with:

java -version

If ever need to restore to GCJ, can do so with this:update-java-alternatives --verbose --set java-gcj

Install some useful java libraries:

aptitude install libmysql-java junit junit-doc

Install the NTP daemon to keep the time current.

Install NTP daemon to keep the time current:

aptitude install ntp ntp-doc

Then

nano /etc/ntp.conf

... and set up as follows:

# /etc/ntp.conf, configuration for ntpd

driftfile /var/lib/ntp/ntp.drift
statsdir /var/log/ntpstats/

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

# deny-by-default policy
restrict default ignore

# You do need to talk to an NTP server or two (or three).
server clock.psu.edu       iburst
server ntp0.cornell.edu    iburst
server ntp0.nl.net         iburst
server ntp2a.mcc.ac.uk     iburst
server salmon.maths.tcd.ie iburst

# Have to grant access to the above servers - however the servers are not allowed
# to modify the run-time configuration or query the NTP server.
restrict clock.psu.edu       nomodify nopeer notrap noquery
restrict ntp0.cornell.edu    nomodify nopeer notrap noquery
restrict ntp0.nl.net         nomodify nopeer notrap noquery
restrict ntp2a.mcc.ac.uk     nomodify nopeer notrap noquery
restrict salmon.maths.tcd.ie nomodify nopeer notrap noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1           nomodify nopeer notrap

Then restart with:

/etc/init.d/ntp restart

Check that there were no errors in the system log with:

tail -f /var/log/syslog

Should get output like this:

Jul  7 17:17:31 ludo ntpd[3473]: ntpd 4.2.2p4@1.1585-o Sun Mar  4 13:21:35 UTC 2007 (1)
Jul  7 17:17:31 ludo ntpd[3474]: precision = 2.000 usec
Jul  7 17:17:31 ludo modprobe: WARNING: Not loading blacklisted module ipv6
Jul  7 17:17:31 ludo ntpd[3474]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Jul  7 17:17:31 ludo ntpd[3474]: Listening on interface lo, 127.0.0.1#123 Enabled
Jul  7 17:17:31 ludo ntpd[3474]: Listening on interface eth0, 192.168.0.5#123 Enabled
Jul  7 17:17:31 ludo ntpd[3474]: kernel time sync status 0040
Jul  7 17:17:31 ludo ntpd[3474]: frequency initialized -70.588 PPM from /var/lib/ntp/ntp.drift

Can then check if it is working with:

ntpq -p

Should get output like this:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*otc2.psu.edu    128.118.25.12    2 u    1   64    1  256.700   -0.417   1.318
 cudns.cit.corne 192.5.41.209     2 u    2   64    1  263.631   -4.051   0.580
 ntp0.nl.uu.net  .GPS.            1 u    1   64    1  350.644    0.044   1.723
 maverick.mcc.ac 193.62.22.98     2 u    2   64    1  325.238   -3.433   0.123
 salmon.maths.tc 134.226.1.114    3 u    1   64    1  335.144   -0.182   0.198

Can then check for open UDP connections, like so:

netstat -l

Should get output which includes something like this:

udp        0      0 ludo.yourhostname.c:ntp *:*
udp        0      0 localhost:ntp           *:*
udp        0      0 *:ntp                   *:*

(There does not seem to be any way to turn these udp connections off, which is why we use the deny-by-default policy)

Automatic daily downloading and email notification of pending security updates.

Automatic daily downloading and email notification of pending security updates. source. Note: this step is probably only appropriate for a headless web server, and a laptop or desktop-system has synaptic, which provides a nicer interface to package updating. Install:

aptitude install cron-apt

Configure:

nano /etc/cron-apt/config

Change this line:

MAILON="always"

To test, can run with:

/usr/sbin/cron-apt

This will then automatically download any security updates each night, and send a daily email to root about what updates need to be applied. Then install apt-listchanges for a printout and emailed list of package changes when actually installing:

aptitude install apt-listchanges

Configure apt-listchanges to show information about what an update contains, before we actually install it:

dpkg-reconfigure apt-listchanges
Method for changes display: text
skip changes that have already been seen? No.
E-mail Address(es) which will receive changes: root
Changes displayed with apt: both
Prompt for confirmation after displaying changes? Yes
What would you like to do about listchanges.conf? Install the package maintainer's version

Then check the above configuration applied successfully:

cat /etc/apt/listchanges.conf

When ready to apply package updates, can then manually install the updates with:

aptitude update
aptitude upgrade

This won't apply any kernel updates, however. For these, need to do:

aptitude dist-upgrade

... and then "reboot". Note: a log of package upgrades, downloads, and installations is also kept locally, and can be viewed with:

less /var/log/aptitude

Test if all CPUs are being used.

Q: On an SMP kernel, how to test if both CPUs are being used? How to see the load on each CPU?

aptitude install sysstat

Then to see the load on each CPU, do:

mpstat -P ALL

Output will be like so for 2 CPUs, or more entries if more are being detected and used:

www:~# mpstat -P ALL
Linux 2.6.18-4-686 (www)        04/07/07

21:06:12     CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
21:06:12     all    0.40    7.09    0.24    0.12    0.00    0.00    0.00   92.15    253.26
21:06:12       0    0.40    6.99    0.23    0.15    0.00    0.00    0.00   92.23    252.79
21:06:12       1    0.39    7.20    0.24    0.10    0.00    0.00    0.00   92.07      0.48
www:~#

Move Cronjobs forward a few hours.

Start the overnight cron jobs earlier - seem to be ending around 7:00 am, and this is too late. By default, crontab runs some jobs at 6:52 AM - this is way too late, want things to run from 1:55 AM instead, so that they are well and truly finished by the early morning, so as to not impact early-morning users.

nano /etc/crontab

Change as follows:

- 25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
- 47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
- 52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
+ 55 1    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
+ 01 4    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
+ 02 5    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )

Alias the poweroff command.

Sooner or later, probably at 5 PM on a Friday afternoon as you're rushing to get out of the office and down to the pub, you're going to stuff up and do something silly. For me, that silly stuff-up was mistaking a shell on an important production server for a shell on a testing machine, and powering it off, requiring an embarrassed call to a data centre tech to get it turned back on again ASAP. Because of this, I now like to alias the poweroff command, because on a production server, it's a command that you will hopefully only ever want to use once - and that's when it's about to be removed from the data centre. To do this:

nano ~/.bashrc

Uncomment the lines for a coloured ls, and then add these 2 lines:

# Warning on "poweroff" command.
alias poweroff='echo This is a production machine! If you REALLY want to poweroff, do /sbin/poweroff'

Then reload your bashrc with:

source ~/.bashrc

Permission changes.

A few permission changes recommended by Tiger (a security auditing program) -

Fix the permissions in one manually created directory:

chmod g-w /usr/local/lib
chmod g-w /usr/local

Some other changes:

chmod 660 /var/log/btmp
chmod 600 /boot/grub/menu.lst

Install APC as the PHP opcode cache.

Installing APC as the opcode cache:

Benchmark before installing, on some PHP page on your site:

ab -kc 10 -t 30 http://yourhostname.com/some_php_page.php

Install required dependencies:

aptitude install apache2-dev
aptitude install build-essential

Install APC (choose "yes" when prompted) :

pecl install apc

Then:

nano /etc/php5/apache2/php.ini

... and add these 3 lines to the end:

; APC opcode cache:
extension=apc.so
apc.mmap_file_mask=/tmp/apc.XXXXXX

Then reload apache:

/etc/init.d/apache2 restart

Check that there are no errors in the error log:

tail -20 /var/log/apache2/error.log

Benchmark after installing:

ab -kc 10 -t 30 http://yourhostname.com/some_php_page.php

• For me this showed an increase from 895 completed requests to 3728 completed requests on a test box.
• For me this showed an increase from 2889 completed requests to 12023 completed requests on a production box.

That's around a 4.1 times speed increase on two systems - not bad.