Mammoth Networks Blog

WordPress has evolved from a blogging platform into a development framework. It is becoming increasingly common to see it used as a basis for a variety of software projects, including e-commerce stores. Its ease of use and low barrier to entry means it has rapidly become a favourite.

Unfortunately, as with every popular software system, it has become a target for hackers, spammers and other nefarious types on the Internet. WordPress sites are responsible for a large number of the phishing and malware pages online.

If you're running a website, your site will be a target eventually, no matter how big or small. Malicious users run automated web crawlers (like what Google uses to discover content) to discover and scan zillions of websites looking for vulnerabilities. WordPress is relatively easy to detect, so it's not if your site is targeted by hackers - it's when.

Is WordPress Insecure?

Why is this the case? Is WordPress fundamentally broken or insecure?

The short answer is "no". The longer answer is "no, but… sometimes".

There seem to be two major factors that influence WordPress security: plugins and

passwords.

The Risk of Plugins

One of the best features of WordPress is that it is easily extensible: it can be modified and adapted easily by means of powerful plugin and theming systems.

If you run a WordPress site, you almost certainly have some plugins installed to extend the functionality - maybe a contact form so people can send you email, maybe a spam filter. There is simply a staggering number of plugins, made available because WordPress is easy to develop for, and they're dead easy for anyone to install.

Unfortunately, as with all great power, it comes with great responsibility. It is very important to note that any random WordPress plugin can wield enormous power on your site. The very features that make them so useful also make them so risky. A malicious plugin could track user or admin activity and report it to remote sites.

Genuinely malicious plugins, however, are (probably) pretty rare. Much more common - and certainly responsible for the majority of websites getting hacked or otherwise compromised - are plain old buggy plugins. A plugins with a simple coding error - a character in the wrong spot, or the wrong function call used, or one not used - can mean your entire site is an open book to malicious users. They can read all your data, and - possibly worse - write over it.

Many plugins simply increase the attack surface of a WordPress site - it creates more opportunities for a malicious user to compromise your website.

(It should also be noted that a lot of the problems that apply to plugins also apply to WordPress themes. A theme is basically just a bunch of code. Many themes include plugins, but don't forget that a theme with a bug in it can also cause a lot of problems.)

The Risk of Passwords

Everyone knows this one, so I won't spend too much time on it. Passwords are hard. Everyone wants a password that's easy to remember, but these are typically very weak. Longer passwords are better, but harder to remember. There's the correct horse battery staple method, but then you've got people saying that's no good either. Argh!

Putting that aside for a minute, let's take a look at a real-world example of what is going on when people try to hack your WordPress site by brute forcing your password.

The below is a (small!) excerpt from a authentication log file showcasing a concerted attempt to brute force the passwords on a (small, non-descript, personal) WordPress blog. The sheer volume of requests is interesting - the excerpt doesn't show them all, but there were thousands over the course of a 24 hour period - as is the fact that they come from a large number of different IP addresses (presumably some sort of botnet).

The really scary thing about this is that owners of many WordPress blogs will probably never know that these attempts are taking place. Unless you have some sort of logging (like in the above example), the vast majority of these attempts will happen without you noticing. There may be no effect on your site at all, although if the logins are very frequent it may cause your site's performance to suffer.

But remember: attackers (basically) have unlimited time and resources to continue these attempts. If they finally get the password right, your site has been hacked, and is now compromised.

How do you tell if your site has been compromised?

Many of these attacks are trying to add your site into a network that serves spam pages trying to sell CH3AP PH4RMACEUT1CALS, or to make it so you serve malware to users. Or they might simply get added to a botnet that tries to find more vulnerable sites.

In the case of spamming or malware, unless you keep a really close eye on the contents of every part of your site and the log files, it's often pretty hard to notice. The first you'll hear about it will probably be something like an email from your site host saying your site is doing weird stuff. Your customers might tell you that they get a warning in Firefox or Chrome when they visit your site (like in the below images).

The worst case scenario is if you have a big site with a lot of interesting data - names, email addresses, phone numbers, payment information - anything that real criminals can use for identity theft or for other scary purposes.

Unfortunately, in this case, if your site contains data that is more valuable than the opportunity presented by simply using it to serve spam, it might be very hard to ever notice that it's been compromised. A clever malicious user might simply choose to just sit behind the scenes in your website taking advantage of the information that they now have access to. (This is probably a pretty low risk scenario for most users.)

What happens if I've been hacked?

This is a big topic and it is dependent on precise details. There are two basic things that I'd recommend doing:

1. Find out how the hack occurred. Was it a brute force? Was it a vulnerable plugin? If you don't know how it happened, stopping it from happening again can be very difficult. Unfortunately, finding how it happened can be difficult - you'll need to end up spending a lot of time poring over log files looking for clues.

2. Reinstall WordPress from scratch. This is painful. You need to import your old database - but first you need to make sure there's nothing in the database that might lead to another compromise (e.g., an attacker may add themselves their own user account, which you might not notice if you have a lot of different users).

You can reinstall WordPress from a "last known good" backup - but you need to really know that it's good (i.e., not already compromised), and you really need to have closed whatever hole that let them in.

What can I do to lower the risks?

Here are some simple things you can do to help maximise WordPress security. Before we get into it though, two important notes:

1. Software, and thus security of software, is a moving target. Things are changing all the time - software is modified creating new exploits, more research is done on old code revealing new weaknesses. There is no silver bullet.

2. Prevention is better than the cure. There is always a strong temptation to postpone security measures. Don't. Do what you can right from the start.

The easy stuff:

1. Keep the WordPress core up-to-date
   While very few vulnerabilities have been found in WordPress core recently, it is still very important to ensure you're running the latest release. This has been somewhat simplified by a recent change that offers automatic updating, but the onus is on you to keep it updated. If you're not logging into your administration section regularly (weekly at the very least), the WordPress Development Blog is a great way to stay up-to-date via RSS, or if you prefer email notifications they also have an announcement mailing list.

2. Keep your plugins and themes up-to-date.
   Plugins are updated regularly, but they won't auto-update in WordPress. Many attackers are out there looking for holes in WordPress plugins, so it's vitally important that these are kept as current as possible.

3. Use as few plugins as possible.
   Reduce the attack surface as much as you can. Use plugins that have high ratings. If you can, read the plugin source code to see what it does. Look for plugins that just do the one thing you need - the larger and more complex the plugin, the bigger the attack surface, and the more risk associated.

4. Use strong passwords on your WordPress accounts.
   Come up with a strong password that is long and has a lot of funny characters. Use your browser's password system to remember it. Get another password management system like PasswordSafe and start using it to create and store strong passwords.

5. Use strong passwords on your FTP accounts.
   FTP brute force attempts are also pretty common. If you can, disable FTP altogether.

6. Sign up for Google Webmaster Tools.
   Google's Webmaster Tools has a bunch of great features that give you interesting insights into how your site is working. One of these tools is a malware scanner, and it will notify you if they detect anything scary on your site.

Harder stuff:

1. Add a separate .htaccess-based password on your /wp-admin directory.
   If an attacker gets into your /wp-admin directory, they can basically do anything - edit your site, look at your users, read your database. Putting an additional layer of security by password protecting this directory via a .htaccess will significantly cut down risk. Setting this up only takes a few minutes, and in most cases you'll be able to save the password in your browser, so it will barely be an inconvenience.

2. Remove write permissions.
   As noted in WordPress's own " Hardening WordPress" guide, many of the neat features in WordPress exist because it can write to the disk, allowing you to easily install plugins and themes with a single click. Unfortunately this is a massive attack vector and probably the most commonly exploited part of WordPress as a whole - one bug in the wrong spot and a malicious user can spray whatever files they want all over your site.
   
   In many cases, disabling write permissions will flat-out stop this from happening - in the event of an exploit in a plugin or theme, the web server will simply be unable to write to the disk, which will stop many common attacks.
   
   This is a tricky one though, as it comes with a big downside: it will stop a lot of WordPress functionality from working. You won't be able to install plugins and themes from the administration interface. You won't be able to update WordPress. Some plugins that need to write to the disk (e.g., WP Super Cache) might not be able to function. You won't be able to upload images.
   
   In short, anything that needs to write to the disk won't be able to. The site will generally operate fine though - as long as your code doesn't need to write to the disk, users will still be able to access everything and make comments and place orders and all of that stuff.
   
   You can, of course, simply change the permissions back if you need to do any of these things. But with the loss of convenience comes improved security.

For VPS users:

If you're running your WordPress setup on your own VPS (as opposed to shared hosting), there's a few more things you can do to reduce your risk:

1. Install Fail2ban. Now!
   Fail2ban is a neat piece of software that can scan log files and perform actions based on things it sees. One very useful practical application of this is that you can catch failed login attempts to your WordPress site and then ban the IP addresses involved - massively reducing the effectiveness of brute force attacks.
   
   Fail2ban is good to have on your VPS in general (it can stop other kinds of brute force attempts, including SSH and FTP). It takes a few minutes to install and set up by itself, and there's already a good WordPress plugin called WP fail2ban that is dead easy to get working.
   
   As a bonus, in addition to increasing security it's possible you'll see some performance improvements as well as large-volume brute forces are cut off early and stop hammering your site.

2. Install Tripwire.
   Tripwire monitors changes to files on your disk. It's a useful security tool for WordPress - you can check to see if any of your files have been edited, or if new files have been added, without you doing something.
   
   It's a little more fiddly, but there are plenty of guides online that can walk you through the process.

WordPress is no longer just a great platform for blogging - it's a flexible piece of software that can be used as the basis for many projects. When handled with care it can be just as reliable and secure as anything else, but it's vitally important to remember that - as with any popular software that is Internet-connected - it will be a target for hackers.

We don't regulary post about securities issues here (because, frankly, we'd be doing it all day), but every now and then there's a Big One that makes everyone sit up and take notice.

Last week, a vulnerability in the Bourne Again Shell - more colloquially known as Bash - was announced. Bash is widely deployed in most Linux-based operating systems (and can be found on Windows in cygwin).

The exploit - initially tagged CVE-2014-6271 but now widely known as 'shellshock' - allows remote attackers to execute arbitrary code via a crafted environment.

Since the initial bug was discovered, many more eyes have been on the Bash source code, resulting in several other updates. At the time of writing, another seemingly significant new exploit has been discovered ( CVE-2014-6278 has been reserved).

We're not going to go into too much detail about the issue - it has been covered very extensively elsewhere, and we encourage users interested in more details to read up on the technical roots.

This is a significant vulnerability and we advise all customers to update Bash immediately, and to continue to closely monitor any advisors relating to Bash in the coming weeks.

A new vulnerability in the OpenSSL cryptographic software library was revealed yesterday. It has quickly gathered worldwide attention as an extremely serious problem as it can allow remote access to critical information on servers.

The vulnerability - nicknamed "Heartbleed" - can allow anyone on the Internet to directly access the memory of systems that are using the vulnerable version of the OpenSSL library.

Heartbleed has been assigned CVE-2014-0160. We encourage our customers to check their VPSs to ensure that their OpenSSL has been updated.

Some good references on the subject include:

- http://heartbleed.com/ - a great summary of the issue, including background, details of affected versions, and more information about is exposed.

- Attack of the week: OpenSSL Heartbleed - a more technical overview of the issue, including analysis of the vulnerable code, as well as some testing tools to check if your server is vulnerable.

One of the first metrics encountered when learning to administer a Linux system is the system load average. The load average is available directly through the /proc/loadavg file and included in the output of commands such as 'w', 'uptime', and 'top'. For example:

$ uptime
 12:36:23 up 23 days, 19:58,  1 user,  load average: 0.38, 0.38, 0.43

A quick check of the manual will tell you that the three numbers are the system load averages for the past 1, 5, and 15 minutes - however, this does not explain what "system load" is actually a measure of. As it turns out, the system load is easy to understand but often not clearly explained.

The system load is the number of processes that are in one of two states: running (actively executing code) or ready to run (not actively executing, but will do so as soon as possible). Processes that are ready to run - but not yet running - can be in this state for a number of reasons, but the two most common are:

• waiting for CPU time: all the processors are busy running some other process
• waiting for I/O request to complete: for example, the process has requested some data from the hard disk, and is waiting for the data to be read

This second point is crucial to understanding why an overwhelmed system may show a load average of 30, 50, or even over 100: in such cases, it is rare that the problem is lack of CPU time. Instead, the high load average is typically caused by an I/O subsystem that is unable to keep up with the request rate.

System Load and Xen

When applied to a privileged domain ("Dom0") in Xen, we now have a problem. The Linux system load average counts processes that are running or ready to run, but Xen guest domains are not Linux processes and so do not factor into the calculation.

On a Xen host, we can use the command "xm list" to see a list of the guest domains and their current state. For example:

$ sudo xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  1536     2     r----- 11690263.8
dom000040                                  339  1536     4     r----- 192239.1
dom000048                                   15   128     4     ------ 332418.0
dom000067                                    5   512     1     r----- 10394745.1
dom000089                                  323   256     4     -b----  51310.3
dom000149                                  348  2048     4     -b----  25867.2
dom000173                                   10   128     4     -b---- 117079.9
dom000193                                   11   128     1     -b---- 128118.4
dom000514                                  123   192     4     -b---- 767836.0
dom000576                                  340   512     4     -b----  75573.7
dom000657                                  357  2048     4     -b----  94863.6
dom000701                                  185   256     4     -b---- 208035.7
dom000720                                  341  1024     4     -b---- 127005.0
dom000727                                  358  2048     2     -b----   7916.4

Here we have 3 guests in the "running" state (r-----), one in the ready to run state (------) and the rest are all "blocked" (-b----) which under Xen means they are currently sleeping.

The "xm list" command gives us all the information we need to calculate the system load for Xen: what is needed is a program to average this information.

Introducing xenload

xenload ( download it from our github account) is a short python script that runs as a daemon, calculating the Xen system load by parsing the "xm list" output every five seconds and storing the result.

To install xenload, download it to /usr/local/sbin and start the daemon:

wget -q https://raw.github.com/MammothMedia/XenExtras/master/xenload \
-O /usr/local/sbin/xenload
/usr/local/sbin/xenload --daemon 

Wait a while for it to gather some data, and then check the Xen system load:

$ /srv/xen/xenload
xen load average: 4.80, 4.67, 4.81

Or if you need the data in a more easily parsable for software like Cacti:

$ /srv/xen/xenload --cacti
load_1min:4.40 load_5min:5.00 load_15min:4.80

(To make sure xenload is always available, place "/usr/local/sbin/xenload --daemon" into /etc/rc.local or an init script to load the daemon at system boot. )

This is a simple addition to your system administrator toolbox to provide an "at a glance" metric for system health, and is easy to graph due to its similarity to the Linux system load average.

PV-GRUB (or PvGrub) is an under-advertised feature of Xen for booting para-virtualized Linux guests. It performs the same role as the more prominent PyGrub, reading /boot/grub/menu.lst from the guest's disk and then booting with the appropriate kernel and initrd.

Where they differ is that PyGrub reads this data from within a Dom0 process before the guest has started, providing a potential window for exploit by a malicious guest. PvGrub on the other hand, is utilised as the guest's kernel directly; once the guest boots PvGrub then chainloads into the guest's kernel in a similar mechanism to a regular Grub installation. As such, PvGrub is an excellent tool for commercial Xen providers who must protect their hosts against potential attacks from clients.

Unfortunately Ubuntu (and Debian) do not distribute PvGrub as part of their xen installation; as this was first reported as a bug over two years ago its safe to assume that its not likely to be fixed any time soon. While it is possible to build from source, because PvGrub is a bootable kernel it has no dependencies and can be copied as-is from another distribution.

A good source for this is Gitco, a Xen repository for CentOS 5.x . The process is simple enough - download the .rpm , extract its contents, and copy the PvGrub files into place. In the example below I have used Xen 4.1.2, the version that shipped with Ubuntu 12.04 LTS:

# Create a working area
mkdir /root/pvgrub ; cd /root/pvgrub
# Download Xen from gitco
wget http://www.gitco.de/repo/xen4.1.2/xen-4.1.2-2.el5.x86_64.rpm
# Install required tool to extract the .rpm contents
apt-get install -y rpm2cpio </dev/null
rpm2cpio xen-*.rpm | cpio -idm
# Copy PvGrub into place
mv usr/lib/xen/boot/pv* /usr/lib/xen-4.1/boot
# Cleanup
cd ; rm -rf /root/pvgrub

With PvGrub in place, its just a matter of updating the guest's Xen configuration appropriately - for example:

kernel = "/usr/lib/xen-4.1/boot/pv-grub-x86_32.gz"
extra = "(hd0,0)/grub/menu.lst"

Linux guests can now control the kernel they boot with, at no risk to the host.

In a typical Xen installation, guest networking is provided via a bridge created on the host (the other common options being NAT or route). Traffic is received by the host from its ethernet port/s before being passed to the bridge; and the bridge then passes that traffic to the appropriate guest based on the destination MAC address.

This setup creates a scenario where each guest will act as though its physically plugged into a switch. While this is convenient, it also means guests will be able to both receive and send any type of layer 2 traffic such as Spanning Tree Protocol or ATA over ethernet.

If the guests are untrusted, then this level of network access is likely to be undesirable: instead, guests should be restricted to IPv4, IPv6, and ARP. Linux includes a tool called ebtables ("ethernet bridge tables") that can perform exactly this type of filtering. To do so, create a startup script containing the following:

ebtables -N PROTOCOLS
ebtables -A PROTOCOLS -p ip -j RETURN
ebtables -A PROTOCOLS -p ip6 -j RETURN
ebtables -A PROTOCOLS -p arp -j RETURN
ebtables -A PROTOCOLS -j DROP
ebtables -I FORWARD -j PROTOCOLS

Now each packet that is forwarded through the host's bridge will have its protocol checked; if it is not one of IPv4, IPv6, or ARP than the packet (ethernet frame) is dropped. Guests will be unable to emit undesired traffic, and will not receive unnecessary ethernet frames for other protocols.

The excellent web development blog Smashing Magazine has a great new article up for anyone looking to convert an existing website into a WordPress one.

If you're stuck trying to maintain an old, dated website - maybe you're still manually editing HTML files, or using a really painful and slow content management system to update pages - this might be a good read.

Switching over to WordPress will open up new worlds of functionality and accessibility for your users.