What is buffer overflow, an old vulnerability that’s causing new problems?

In mid-May, a nasty security bug was discovered in popular messaging app WhatsApp that allowed hackers to install powerful malware on iPhone and Android devices. Interestingly, to exploit the bug, attackers didn’t need to gain physical access to the victim’s phone. They didn’t even need to run a phishing scam and trick their target to download a malicious file or visit a boobytrapped website.

All they had to do was to send a specially crafted WhatsApp voice call to the victim’s phone. And the victim didn’t even need to answer the call. Leaving their phone on and letting it ring would be enough to set the bug in action.

Aside from the fact that an Israeli vulnerability broker has allegedly used the bug to try to break into the phone of a U.K.-based lawyer, we know very little about the scope of the damage the vulnerability has caused.

But what we do know, according to a Facebook advisory, is that the culprit was a “buffer overflow” vulnerability.

Buffer overflow, a very dangerous kind of security vulnerability, has been haunting software developers and security experts for decades. At its core, buffer overflow is a very simple bug, but despite advances to security software and computer code security tools, it remains a source of concern.

Here’s what you need to know about how buffer overflow errors happen and why they can be so destructive.

A primer on computer code and memory

Before we dig into buffer overflows, you need to know how programs work. Every time you fire up an application on your PC or mobile device, the program is loaded into the device’s memory. For security reasons, most modern computer architectures give every app its own memory space, which means programs can’t access or manipulate the memory allocated to other running programs.

Also, depending on the architecture of your device, the app’s memory space might be split between physical memory (RAM) and your hard disk. But for the sake of simplicity, let’s consider every application has a contiguous piece of memory (in reality, this is called “virtual memory”) that spans across several gigabytes. Each byte of memory in your application has an address, say 1000 or 8192, based on how far it stands from the beginning of the memory space.

The memory space of each application is composed of two main segments: code and data (there is more to it than that, but we’ll stick to this categorization for now). The code segment contains instructions, the commands that perform the different functions of the application and is read-only, which means its contents do not change. The data segment contains “variables,” changing bits of information that control the app’s behavior.

Variables can be things like the number unread emails in your inbox, the display name on your Facebook account, or an image displayed on your Twitter feed, or a message you’re typing in WhatsApp. These variables are loaded into the app’s memory space when it is starts and become modified as you interact with the app.

Every variable has an address that marks where it is located in the memory space. So, for instance, when your Facebook app wants to display your name, it will have to go to read it from the address where the variable is stored.

Variables also have a size, the number of bytes they occupy in the memory space. For some types of data, such as numbers, the size of variables is fixed (usually 4 or 8 bytes), while for others, such as text or images, the size can vary depending on the purpose they serve.

This is where things get tricky.

How does buffer overflow happen?

Buffer overflow happens when data that is supposed to be stored in one part of memory exceeds its boundaries and spills into neighboring memory addresses.

Say you’re running a video conferencing application. Like most other video conferencing applications, the program prompts for your display name when you’re joining a conference. The application has a 50-byte-sized variable where it stores your name. Considering every character takes up one byte of space, you can have a name that is up to 50 characters long (it’s actually 49 characters because the system reserves one byte to put a “0” to mark the end of the string).

What happens if you type in a name that is 60 characters long? The extra 10 bytes will go beyond the space allocated to the user name and will overwrite whatever variable was located after. For instance, if that space was reserved for some other text, such as the title of the conference, then you’ll probably see the trailing part of your name appear on top of your window.

Why is buffer overflow dangerous?

This was a very simple example of buffer overflow. Such errors almost never happen these days. Even novice programmers learn to avoid these stupid mistakes, and most programming tools have built-in features that detect simple buffer overflow errors. Also, many popular programming languages (e.g. Javascript and Python) automatically prevent text input from causing buffer overflows.

The example is also very harmless and only results in a minor nuisance to the app’s users. In other cases, buffer overflow errors may crash the application, or cause it to behave in erratic ways.

Crafty hackers find subtle ways to cause buffer overflow errors and use them to manipulate the behavior of applications for malicious purposes. For instance, a hacker might be able to exploit a buffer overflow vulnerability to alter a sensitive piece of memory, such as user privileges or the address of the recipient of a payment.

Stack buffer overflow attacks

In the case of WhatsApp, according to Facebook’s notice, “A buffer overflow vulnerability in WhatsApp VOIP stack allowed remote code execution via specially crafted series of SRTCP packets sent to a target phone number.”

First, VoIP (short for voice over IP) is the term use for applications that run voice communication over the internet as opposed to regular phone lines. SRTCP is a protocol that enables secure, encrypted communications for VoIP apps. But what’s important here are two terms: “stack” and “remote code execution.”

Stack buffer overflow is a very popular and dangerous kind of buffer overflow. The memory space of every running application contains a “stack segment” (something we omitted earlier in this article for the sake of simplicity). Stack is like the data segment, with the difference is that its contents are very temporary. The stack mainly contains “function argument data.” These can be input received from users, such as the user name example we discussed above.

But the stack also contains “return addresses,” sensitive bits of information that contain the numerical address of the instruction that the applications must execute after finishing its current function. Return addresses are like bookmarks that allow the program to remember its location as it hops from one part of the code to another.

A well-placed stack buffer overflow attack can alter return addresses, effectively enabling the attacker to force a program to jump to an arbitrary sequence of instructions, such as a set of commands that download and install a malicious app on the device.

This is the “remote code execution” part mentioned in the Facebook notice.

Buffer overflow has a long history

The WhatsApp security disaster is just the latest of a long list of buffer overflow vulnerabilities discovered by security researchers in the past decades. In fact, the Morris Worm, the first internet worm, which dates back to 1988, exploited a buffer overflow vulnerability to spread across Unix machines.

In 2001, a nasty virus called Code Red exploited a buffer overflow vulnerability in Microsoft Internet Information Services, a very popular service for hosting web applications, to stage a distributed denial of service (DDoS) attack on the White House’s servers.

In 2003, a buffer overflow in another popular Microsoft service, SQL Server 2000, enabled the SQL Slammer worm to spread across more than 250,000 computers and cause a massive internet outage.

More recently, Intel patched a critical buffer overflow vulnerability that existed in many of its processors.

Over the years, computer security experts and architects of operating systems have devised various methods to help mitigate the threat of buffer overflow exploits. Most notable among them is address space layout randomization (ASLR), a technique that scrambles the memory space of an application every time it runs. So even if hackers find a buffer overflow vulnerability, they will still have a hard time guessing the locations of instructions and manipulating the behavior of the targeted application to their will.

However, as WhatsApp’s latest security bug reminds us, while buffer overflow exploits have become harder, they haven’t gone away.

READ MORE: 

Ben Dickson

Ben Dickson

Ben Dickson is a software engineer and founder of TechTalks. His work has been published by TechCrunch, VentureBeat, the Next Web, PC Magazine, Huffington Post, and Motherboard, among others.