Precision and Number Display

by Allen Wyatt
(last updated November 8, 2017)

In a different ExcelTip you learned how Excel handles the internal precision of numbers you enter. With that understanding firmly in mind, you should also understand that the internal precision is entirely different from what Excel displays. While this may sound confusing, it is analogous to how Word deals with text: Text and the display of text (formatting) are intrinsically different. In Excel, numbers and the display of numbers are also intrinsically different.

For instance, let's say you enter the number 4.760546328934 in an Excel worksheet cell. This number is automatically converted to fifteen-digit precision by Excel. This means that it is stored as 4.76054632893400 internally. How you display that number, however, is entirely up to you. You must remember, as well, that regardless of how the number is displayed, Excel will use the entire fifteen digits in performing calculations.

When you first enter the number, Excel displays exactly what you entered (to fifteen digits) in the Formula bar at the top of the screen. What is displayed in the cell is determined by the formatting you have applied to the cell. If the format is set to General, then Excel displays only eleven digits, and counts the decimal point as one of those digits. Thus, what you see displayed would be 4.760546329. Notice that this is a rounded value, which is consistent with how Excel displays shorter numbers, as well.

If you want to change the formatting applied to a number in a cell, you can do so by choosing Cells from the Format menu, and then clicking on the Number tab. (Formatting cells is discussed in other ExcelTips issues.)

ExcelTips is your source for cost-effective Microsoft Excel training. This tip (1982) applies to Microsoft Excel 97, 2000, 2002, and 2003.

Author Bio

Allen Wyatt

With more than 50 non-fiction books and numerous magazine articles to his credit, Allen Wyatt is an internationally recognized author. He  is president of Sharon Parq Associates, a computer and publishing services company. ...

MORE FROM ALLEN

Saving and Closing All Open Documents

Want to close or save all your documents at the same time? This trick does it for you.

Discover More

Continuing Macro Lines

Sometimes a macro command line can get very, very long. This can make it hard to understand when you look at it a month or so ...

Discover More

Conditionally Displaying a Message Box

You can, from within your macros, easily display a message box containing a message of your choice. If you want to display ...

Discover More

Create Custom Apps with VBA! Discover how to extend the capabilities of Office 2013 (Word, Excel, PowerPoint, Outlook, and Access) with VBA programming, using it for writing macros, automating Office applications, and creating custom applications. Check out Mastering VBA for Office 2013 today!

More ExcelTips (menu)

Changing the Color of Worksheet Gridlines

Want the gridlines in your worksheet to be a different color? You aren't limited to stodgy black; Excel lets you make them ...

Discover More

Precision in Excel

The best way to understand why Excel provides the answers it does is to examine how it handles precision. This tip provides a ...

Discover More

Getting Rid of the Startup Spreadsheet

When you start Excel, you normally see a blank worksheet displayed. Here's how to get rid of it.

Discover More
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

View most recent newsletter.

Comments

If you would like to add an image to your comment (not an avatar, but an image to help in making the point of your comment), include the characters [{fig}] in your comment text. You’ll be prompted to upload your image when you submit the comment. Maximum image size is 6Mpixels. Images larger than 600px wide or 1000px tall will be reduced. Up to three images may be included in a comment. All images are subject to review. Commenting privileges may be curtailed if inappropriate images are posted.

What is 6 - 0?

There are currently no comments for this tip. (Be the first to leave your comment—just use the simple form above!)


This Site

Got a version of Excel that uses the menu interface (Excel 97, Excel 2000, Excel 2002, or Excel 2003)? This site is for you! If you use a later version of Excel, visit our ExcelTips site focusing on the ribbon interface.

Newest Tips
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

(Your e-mail address is not shared with anyone, ever.)

View the most recent newsletter.