Vulnerability Note VU#162289

C compilers may silently discard some wraparound checks

Original Release date: 04 Apr 2008 | Last revised: 08 Oct 2008

Overview

Some C compilers optimize away pointer arithmetic overflow tests that depend on undefined behavior without providing a diagnostic (a warning). Applications containing these tests may be vulnerable to buffer overflows if compiled with these compilers.

Description

In the C language, given the following types:

        char *buf;
        int len;

some C compilers will assume that buf+len >= buf. As a result, code that performs wrapping checks similar to the following:

    len = 1<<30;
    [...]
    if(buf+len < buf)  /* wrap check */
      [...overflow occurred...]

are optimized out by these compilers; no object code to perform the check will appear in the resulting executable program. In the case where the wrap test expression is optimized out, a subsequent manipulation of len could cause an overflow. As a result, applications that perform such checks may be vulnerable to buffer overflows.

Wrapping checks that use methods similar to the one described above depend on undefined behavior. Conforming implementations are permitted to perform the optimization by the ISO/IEC 9899:1999 C specification (§6.5.6p8) as undefined behavior. Even if a conforming implementation currently generates object code for an undefined behavior, future versions of the compiler are not obligated to do the same; this behavior may be viewed as an opportunity for further optimization. To ensure that such changes to the compiler do not invalidate assumptions, developers should follow the recommendations described in CERT C Secure Coding recommendation MSC15-C and rule ARR38-C to avoid this error. Furthermore, compilers are not required to issue diagnostics for undefined behavior, so there is frequently no easy way to identify undefined behavior in code, particularly during manual code audits.

Note that this issue does not strictly constitute a vulnerability in the compilers themselves. Rather, this behavior may introduce vulnerabilities in applications that include similar code and are compiled with affected compiler implementations. Existing code that relies on the undefined behavior in the wrapping check is particularly susceptible to this behavior.

Multiple implementations are known to perform this optimization. This optimization may be affected by the setting of the optimization level as well as other flags. Additional information about affected implementations can be found in the Systems Affected section of this document.

Impact

An application that performs wrapping checks based on an expression such as the one described above may be vulnerable to buffer overflow if compiled with affected compiler implementations. The nature of the resulting vulnerability would be specific to the application and depends on how the affected code is used.

Solution

Use casts
Cast objects of type char* to uintptr_t before comparison. The faulty wrapping check listed above would be written

    #include <stdint.h>
    [...]
    if((uintptr_t)buf+len < (uintptr_t)buf)
       [...]
Alternatively, developers can use size_t on platforms that do not provide the uintptr_t type. Developers should also follow the recommendations described in CERT C Secure Coding rule ARR38-C.

Avoid affected compiler implementations
Application developers and vendors of large codebases that cannot be audited for use of the defective wrapping checks are urged to avoid using compiler implementations that perform the offending optimization. Vendors and developers should carefully evaluate the conditions under which their compiler may perform the offending optimization. In some cases, downgrading the version of the compiler in use or sticking with versions of the compiler that do not perform the offending optimization may mitigate resulting vulnerabilities in applications.

Systems Affected (Learn More)

VendorStatusDate NotifiedDate Updated
GNU Compiler CollectionAffected-29 Jul 2008
Intel CorporationNot Affected17 Apr 200825 Apr 2008
Sun Microsystems, Inc.Not Affected17 Apr 200826 Aug 2008
Cray Inc.Unknown23 Apr 200823 Apr 2008
Hewlett-Packard CompanyUnknown17 Apr 200817 Apr 2008
IBM CorporationUnknown17 Apr 200817 Apr 2008
Microsoft CorporationUnknown17 Apr 200817 Apr 2008
Silicon Graphics, Inc.Unknown17 Apr 200817 Apr 2008
SybaseUnknown17 Apr 200817 Apr 2008
Wind River Systems, Inc.Unknown17 Apr 200817 Apr 2008
If you are a vendor and your product is affected, let us know.

CVSS Metrics (Learn More)

Group Score Vector
Base N/A N/A
Temporal N/A N/A
Environmental N/A N/A

References

Credit

Felix von Leitner originally published information about this behavior in the gcc compiler. Russ Cox later noticed this behavior as well and provided additional information and assistance.

This document was written by Chad R Dougherty and Robert C Seacord.

Other Information

  • CVE IDs: Unknown
  • Date Public: 17 Apr 2006
  • Date First Published: 04 Apr 2008
  • Date Last Updated: 08 Oct 2008
  • Document Revision: 61

Feedback

If you have feedback, comments, or additional information about this vulnerability, please send us email.