Loading
Excel.Tips.Net ExcelTips (Menu Interface)

Errors When Subtracting

Please Note: This article is written for users of the following Microsoft Excel versions: 97, 2000, 2002, and 2003. If you are using a later version (Excel 2007 or later), this tip may not work for you. For a version of this tip written specifically for later versions of Excel, click here: Errors When Subtracting.

Fred expressed some confusion and concern about the results he received when subtracting certain numbers. For instance, when he subtracts 3809717.98 from 3799904.94, he should get -9813.04, but instead gets -9813.04000000003.

What Fred is witnessing is an artifact of Excel's limitations. The problem is ultimately related to how Excel works with floating-point numbers. Computers must store numbers internally as binary values, not as the decimal values we see displayed on the screen. Whole numbers can be stored as binary values relatively easily. When you throw a decimal point into the mix, then storing very large or very small numbers becomes more problematic—Excel just isn't able to store them with absolute precision. Instead, Excel provides an "approximate" result, out to 15 digits (the limits of its precision). Thus, you end up with something like -9813.04000000003, which contain the full 15 digits of precision possible in Excel.

A full discussion of how floating-point numbers are maintained in Excel and computers in general can very quickly get extremely technical. For those who want more information on the topic, here are two places you can start your research:

http://support.microsoft.com/kb/78113
http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html

The bottom line is that the only way to get the "exact" results you want, to the number of decimal places you want, is to use the ROUND function in your formulas, as shown here:

=ROUND(3799904.94 - 3809717.98, 2)

You can also, if desired, change the precision used to within all formulas by following these steps:

  1. Choose Options from the Tools menu. Excel displays the Options dialog box.
  2. Make sure the Calculation tab is displayed. (See Figure 1.)
  3. Figure 1. The Calculation tab of the Options dialog box.

  4. Ensure that the Precision As Displayed check box is selected.
  5. Click OK.

Now, Excel uses the precision shown on the screen in all of its calculations, instead of doing calculations at the full 15-digit precision it normally maintains.

ExcelTips is your source for cost-effective Microsoft Excel training. This tip (3354) applies to Microsoft Excel 97, 2000, 2002, and 2003. You can find a version of this tip for the ribbon interface of Excel (Excel 2007 and later) here: Errors When Subtracting.

Related Tips:

Program Successfully in Excel! John Walkenbach's name is synonymous with excellence in deciphering complex technical topics. With this comprehensive guide, "Mr. Spreadsheet" shows how to maximize your Excel experience using professional spreadsheet application development tips from his own personal bookshelf. Check out Excel 2013 Power Programming with VBA today!

 

Leave your own comment:

*Name:
Email:
  Notify me about new comments ONLY FOR THIS TIP
Notify me about new comments ANYWHERE ON THIS SITE
Hide my email address
*Text:
*What is 5+3 (To prevent automated submissions and spam.)
 
 
           Commenting Terms

Comments for this tip:

There are currently no comments for this tip. (Be the first to leave your comment—just use the simple form above!)
 
 

Our Company

Sharon Parq Associates, Inc.

About Tips.Net

Contact Us

 

Advertise with Us

Our Privacy Policy

Our Sites

Tips.Net

Beauty and Style

Cars

Cleaning

Cooking

DriveTips (Google Drive)

ExcelTips (Excel 97–2003)

ExcelTips (Excel 2007–2016)

Gardening

Health

Home Improvement

Money and Finances

Organizing

Pests and Bugs

Pets and Animals

WindowsTips (Microsoft Windows)

WordTips (Word 97–2003)

WordTips (Word 2007–2016)

Our Products

Helpful E-books

Newsletter Archives

 

Excel Products

Word Products

Our Authors

Author Index

Write for Tips.Net

Copyright © 2016 Sharon Parq Associates, Inc.