CMU-CS-21-113
Computer Science Department
School of Computer Science, Carnegie Mellon University



CMU-CS-21-113

Mitigating Memory-Safety Bugs with
Efficient Out-of-Process Integrity Checking

Daming Dominic Chen

Ph.D. Thesis

June 2021

CMU-CS-21-113.pdf


Keywords: Memory safety, inter-process communication, control-flow integrity, pointer integrity, data-flow integrity, compiler, kernel, shared memory, web browser

Computer programs written in low-level languages with manual memory management, like C and C++, can contain unintentional memory-safety bugs[175] due to developer error. Examples of these bugs include spatial buffer overflows, as well as temporal use-after-frees and double frees, which can be leveraged by attackers to exploit programs by altering their runtime behavior. Indeed, statistics from both Google Chrome[1] and Microsoft[129] show that ~70% of all security vulnerabilities in their codebases involve memory-safety bugs.

Past work, as discussed in Chapter 2, has proposed various strategies to eagerly detect or lazily mitigate such bugs. Eager approaches detect memory-safety bugs by checking pointer operations (§2.1), whereas lazy mitigations prevent exploitation by validating program data (§2.4,§2.5). To improve accuracy, mitigations may need to maintain internal state (metadata) about program execution, which must also be protected from corruption. This has been achieved using different techniques, including software-based address space partitioning (§2.2), and hardware-based fine-grained instruction monitoring (§2.3). Nevertheless, these approaches suffer from significant complexity, brittleness, or incompatibility, which reduces their efficiency and effectiveness.

In this thesis, we observe that existing mitigations are limited by their decision to maintain internal metadata within the same process. We show that augmenting hardware with a small, secure, and efficient AppendWrite inter-process communication (IPC) primitive allows metadata storage and policy checking to be performed in a separate isolated process, which improves both security and performance. We implement this design in our HerQules [42] and data-flow integrity that are more precise than past work. We evaluate our designs on a variety of real-world programs, including multiple benchmark suites, the NGINX web server, and the Google Chromium web browser.

97 pages

Thesis Committee:
Phillip B. Gibbons (Chair)
James C. Hoe
Bryan Parno
Taesoo Kim (Georgia Institute of Technology)

Srinivasan Seshan, Head, Computer Science Department
Martial Hebert, Dean, School of Computer Science


Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by [email protected]