Precision and Number Display

In the previous tip you learned how Excel handles the internal precision of numbers you enter. With that understanding firmly in mind, you should also understand that the internal precision is entirely different from what Excel displays. While this may sound confusing, it is analogous to how Word deals with text: Text and the display of text (formatting) are intrinsically different. In Excel, numbers and the display of numbers are also intrinsically different.

For instance, let's say you enter the number 4.760546328934 in an Excel worksheet cell. This number is automatically converted to fifteen-digit precision by Excel. This means that it is stored as 4.76054632893400 internally. How you display that number, however, is entirely up to you. You must remember, as well, that regardless of how the number is displayed, Excel will use the entire fifteen digits in performing calculations.

When you first enter the number, Excel displays exactly what you entered (to fifteen digits) in the Formula bar at the top of the screen. What is displayed in the cell is determined by the formatting you have applied to the cell. If the format is set to General, then Excel displays only eleven digits, and counts the decimal point as one of those digits. Thus, what you see displayed would be 4.760546329. Notice that this is a rounded value, which is consistent with how Excel displays shorter numbers, as well.

If you want to change the formatting applied to a number in a cell, you can do so by choosing Cells from the Format menu, and then clicking on the Number tab. (Formatting cells is discussed in other ExcelTips issues.)