Buffer Overflow Exploits And Computer Security Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Buffer overflow exploits have been and continue to remain to be a major threat to computer security. In order to have better understand on this, this paper discuss about the topics that related to buffer overflow. The paper begins with the explanation what is buffer overflow all about and then a little bit about the types of buffer overflow. Next is briefly explains about how buffer overflow happened. This paper also discussing a little bit about buffer overflow exploit and finally in depth about buffer overflow prevention techniques.


Buffer overflow attacks continue to be among the most prevalent form of attacks on computer system. A buffer overflow occurs when data is written outside of the bounds of its allocated memory buffer. A buffer overflow is a technique that exploits vulnerabilities in languages that lack bounds checking on the memory accesses, allowing an attacker to write malicious content past the end of a reserves memory region. A buffer overflow proofs that the computer science or software programming community still does not have a better understanding or good knowledge of how to design, create and implement source code. All buffer overflows are a product of poorly constructed software program. This program may have multiple deficiencies like stack overflows, heap corruption, format string bugs and race conditions. Buffer overflows can be as small as one misplaced character in a million line program or as complex as multiple character arrays that are inappropriately handled.

Some buffer overflows can be found in local programs like calendar application, calculators, games and Microsoft Office application. Moreover, buffer overflows also exist in remote software like e-mail servers, FTP, DNS and Internet Web servers (Kuperman et al, 2005).


There are two primary types of buffer overflow situation which are stack based and heap based. The most common type of buffer overflow attacks is stack based. The stack based involves overwriting and thus changing security sensitive variables or control flags stored in memory adjacent to the unchecked buffer. The stack is an actual data structure in memory, accessed in LIFO which is last in first out order. This memory segment, located above and adjacent to heap data, grows down from a higher memory address to a lower memory address. Like heap data, stack data is also allocated at runtime. The stack is like a "scratch pad" that temporarily holds a function's parameters and local variables as well as the return address is of prime importance as it presents executable code sitting on the stack waiting for its turn to execute. While, the heap based attacks flood the memory space reserved for a program, but the difficulty involved with performing such an attack makes them rare (Foster et al, 2005).


In order to better understand how a buffer overflow occurs, a simple example can be demonstrating using strings. Before go any further, we will look at how the strings work first. In programming languages like C and C++ which are the most commonly used languages in application today, strings are stored in a unique manner. While there are actual string variables in both languages, a string is actually represented by an array of bytes, which are represented as characters. In most stacks, the smallest unit of memory allowed is a word, which is four bytes. Thus, all buffers must be in multiples of four (4, 8, 12, etc.). If a string wants to allocate memory for 17 characters, it must allocated 20 because of using words. When referring a string, a pointer to the first byte of the array is used so that the program knows where to begin. After the actual character input, zero or null bytes are stored in the remaining bytes in the array. When a zero byte is reached, the program knows that it has reached the end of the string.

Now, we look at the example of buffer overflow string. When a string is declared, the memory sets aside a character array with a length equal to the maximum length of the string. Let assume that this particular program declares two strings which are FirstName and LastName, each with a maximum length of 6. Upon declaration, FirstName is left unassigned (all zero bytes) and LastName is assigned the name of Bryan.

Thus, each character array representing the strings would look as follows:















Figure 1: Initialized character arrays.

As stated above, the FirstName string is undeclared and contains only zero bytes, while the LastName string contains the characters B, R, Y, A, N and a zero byte to indicate the end of the string. so, in this example of a six character array, the maximum amount of characters that can be used is five, due to the inclusion of the zero byte that is need to identify the end of the string. Then, let assume that the program does not check that the user enters an amount of characters within the bounds of the array and the user enters the name "MICHEAL" into the FirstName string, "CHRISTINE" has eight total characters (seven in the name and one for the extra zero byte) but the system memory has only allocated six characters for the input. What then happens is the remaining four characters overflow, rewriting the data adjacent to the FirstName buffer on the stack.















Figure 2: Character overflow into adjacent memory.

This is how a buffer overflow occurs in the simplest concept. We can see the data that exceeds the allocated memory buffer overflows into adjacent memory and overwrites it with the excess data. In the above example, the only effect the overflow will have is altering the value of the LastName string. While this is an undesired result, there is nothing malicious or devastating about this occurrence (Kuperman et al, 2005).


Exploiting a buffer overflow is to make use of the existing vulnerability of a running program including the operating system and injecting a set of computer codes that can be placed when the buffer's capacity in query has been exceeded. These codes can finally lead to the running of the attackers' codes. Attackers' codes can carry virus, worm, rootkit or any malicious code that can perform a harmful action such as crushing the application, crushing the system, spreading viruses and others.

Exploiting a buffer overflow has been an attractive and effective technique to attackers' society in order to gain remote access to victims' computers. Exploit consists of two parts which are injection vector and payload. Payload can carry code to cause remote shell, install rootkit and any other malicious actions. SQL Slammer and Witty worm are the very famous examples of worms that are based on buffer overflow exploitation (Donaldson, 2002).


In order to prevent these buffer overflow vulnerabilities, the extensive research has focused on tools and techniques. One of the best ways to prevent the exploitation of buffer overflow vulnerabilities is to detect and eliminate them from the source code before the software is put to use, usually by performing some sort of static analysis on either the source code or on the compiled binaries. A proven technique for uncovering flaws in software is source code review, also known as source code auditing. Among the various efforts along these lines, the best known is the OpenBSD project. Tools designed for automatic source code analysis complement manual audits by identifying potential security violations, including functions that perform unbounded string copying. Some of the best-known tools are ITS4 and LCLint.

Most buffer overflow vulnerabilities are due to the presence of unbounded copying function or unchecked buffer lengths in programming languages like C. One way to prevent programs from having such vulnerabilities is to write the using a language like Java or Pascal that performs bound checking. However, such languages often lack the low level data manipulation needed by some applications. Therefore, researchers have produced more secure versions of C that are mostly compatible with existing programs but add additional security features (Leontie et al, 2005).

According to Microsoft resources, preventing buffer overflow is a matter of writing a robust code. Buffer overflow is mainly related to low level programming language and especially C and C++. It is also related in some cases to the type of the Micro ship and its memory management techniques. For instance, Latest Intel Micro-ship has implemented some security mechanism to protect against buffer overflow. Since buffer overflow resulted from poor coding, it is important to have certain measures during and even after the development phases to avoid these programs' issues. There are two categories of countermeasure against buffer overflow which are traditional and proactive.


For traditional countermeasure, it can be summarize as follows:

One of the important steps to make sure that the input to the program is checked and does not exceed the memory assigned to it is never trust user's input. Boundary checks to the input functions that are available in the standard library are not done by default. Therefore, it is imperative to do the proper checking. In order to perform this task, many tools are available to help automate this process by using fault injection and static analysis technique. Many static analysis tools have been developed, some of them are open source like SPLINT, BOON, UNO, ARCHER and others are commercials such as Verifier and Polyspace.

Using Fault Injection Security Tools (FIST). Fault injection tools are very helpful to minimize the risk of having buffer overflow. They simulate the buffer overflow situation by entering inputs of different sizes or by injecting errors into the source code of the product in question before production that, presumably lead to buffer overflow, analyze the effect of these errors as if it would happen when the product has been released and used by the public.

Use high level languages that have safe library whenever it is possible.

Consider upgrading to a new Intel ship that implements security against buffer overflow whenever it is possible.

Using programming tools to safeguard the return function from being overwritten to avoid transfer the program execution to attacker's code such as StackShield and StackGuard.


There are many tools and techniques used as proactive measures. The techniques used are:

Make the stack section of the memory execution as non executable. In most cases, program does not need to execute code in the stack section. Disabling the stack execution of an operating system makes the system safer against buffer overflow. However, the solution is not easy to implement. Although possible in Linux, some compilers use trampoline functions to implement taking the address of a nested function that works on the system stack being executable. A trampoline is a small piece of code created at run time when the address of a nested function is taken. It normally resides in the stack, in the stack frame of the containing function and thus requires the stack to be executable.

Using a robust and safe C and C++ Library. For instance, C library functions are strcpy(), strcat(), sprintf() and vsprintf(). It providing a better version of library that includes functions that provide thorough checks and robust coding would be definite assets toward safer security application. gets() is another function that reads user input into a buffer from stdin until a terminating newline or EOF is found. The scanf() family of functions also may result in buffer overflows. Hence, the best way to deal with buffer overflow is to not allow them to occur in the first place. Developers should be educated about how to minimize the use of these vulnerable functions.

Protecting the return address by the Compiler. This is the solution to protect the return address has been proposed at the compiler level. The concept is based on inserting "canary" in the return section of the function assuming that if this has been compromised, the whole section will be corrupted.

Dynamic run time check. In this scheme, an application has restricted access in order to prevent attacks. This method primarily relies on the safety code being preloaded before an application is executed. This preloaded component can either provide safer versions of the standard unsafe functions or it can ensure that return addresses are not overwritten. One example of such tool is libsafe. The libsafe library provides away to secure calls to these functions, even if the function is not available. It makes use of the fact that stack frames are linked together by frame pointers. When a buffer is passed as an argument to any of the unsafe functions, libsafe follows the frame pointers to the correct stack frame. It then checks the distance to the nearest return address and when the function executes, it makes sure that address is not overwritten (Leontie et al, 2005).


No tools can solve completely the problem of buffer overflow, but they surely can decrease the probability of stack smashing attack. The best way to avoid buffer overflow is to hire the best programmers and to write the best programs. Since the ideal is not easily achievable, doable or practical, measures and testing should be performed to avoid buffer overflow attacks (Refai, 2006).