Precision and Number Display

Written by Allen Wyatt (last updated September 19, 2022)
This tip applies to Excel 97, 2000, 2002, and 2003


In a different ExcelTip you learned how Excel handles the internal precision of numbers you enter. With that understanding firmly in mind, you should also understand that the internal precision is entirely different from what Excel displays. While this may sound confusing, it is analogous to how Word deals with text: Text and the display of text (formatting) are intrinsically different. In Excel, numbers and the display of numbers are also intrinsically different.

For instance, let's say you enter the number 4.760546328934 in an Excel worksheet cell. This number is automatically converted to fifteen-digit precision by Excel. This means that it is stored as 4.76054632893400 internally. How you display that number, however, is entirely up to you. You must remember, as well, that regardless of how the number is displayed, Excel will use the entire fifteen digits in performing calculations.

When you first enter the number, Excel displays exactly what you entered (to fifteen digits) in the Formula bar at the top of the screen. What is displayed in the cell is determined by the formatting you have applied to the cell. If the format is set to General, then Excel displays only eleven digits, and counts the decimal point as one of those digits. Thus, what you see displayed would be 4.760546329. Notice that this is a rounded value, which is consistent with how Excel displays shorter numbers, as well.

If you want to change the formatting applied to a number in a cell, you can do so by choosing Cells from the Format menu, and then clicking on the Number tab. (Formatting cells is discussed in other ExcelTips issues.)

ExcelTips is your source for cost-effective Microsoft Excel training. This tip (1982) applies to Microsoft Excel 97, 2000, 2002, and 2003.

Author Bio

Allen Wyatt

With more than 50 non-fiction books and numerous magazine articles to his credit, Allen Wyatt is an internationally recognized author. He is president of Sharon Parq Associates, a computer and publishing services company. ...

MORE FROM ALLEN

Editing AutoText Entries Directly

Editing AutoText entries, and particularly deleting them, can be cumbersome if you have a lot of changes to make. There ...

Discover More

Counting the Instances of a Text String

Sometimes it is helpful to know how often a particular phrase appears within a document. If you need to know such a ...

Discover More

Using the GotoButton Field

Need to jump from one place in your document to another? One way to do this is through the user of the GotoButton field, ...

Discover More

Save Time and Supercharge Excel! Automate virtually any routine task and save yourself hours, days, maybe even weeks. Then, learn how to make Excel do things you thought were simply impossible! Mastering advanced Excel macros has never been easier. Check out Excel 2010 VBA and Macros today!

More ExcelTips (menu)

Getting Rid of the Startup Spreadsheet

When you start Excel, you normally see a blank worksheet displayed. Here's how to get rid of it.

Discover More

Precision in Excel

The best way to understand why Excel provides the answers it does is to examine how it handles precision. This tip ...

Discover More

Changing the Color of Worksheet Gridlines

Want the gridlines in your worksheet to be a different color? You aren't limited to stodgy black; Excel lets you make ...

Discover More
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

View most recent newsletter.

Comments

If you would like to add an image to your comment (not an avatar, but an image to help in making the point of your comment), include the characters [{fig}] (all 7 characters, in the sequence shown) in your comment text. You’ll be prompted to upload your image when you submit the comment. Maximum image size is 6Mpixels. Images larger than 600px wide or 1000px tall will be reduced. Up to three images may be included in a comment. All images are subject to review. Commenting privileges may be curtailed if inappropriate images are posted.

What is one less than 9?

There are currently no comments for this tip. (Be the first to leave your comment—just use the simple form above!)


This Site

Got a version of Excel that uses the menu interface (Excel 97, Excel 2000, Excel 2002, or Excel 2003)? This site is for you! If you use a later version of Excel, visit our ExcelTips site focusing on the ribbon interface.

Newest Tips
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

(Your e-mail address is not shared with anyone, ever.)

View the most recent newsletter.