Precision and Number Display

Written by Allen Wyatt (last updated September 19, 2022)
This tip applies to Excel 97, 2000, 2002, and 2003


In a different ExcelTip you learned how Excel handles the internal precision of numbers you enter. With that understanding firmly in mind, you should also understand that the internal precision is entirely different from what Excel displays. While this may sound confusing, it is analogous to how Word deals with text: Text and the display of text (formatting) are intrinsically different. In Excel, numbers and the display of numbers are also intrinsically different.

For instance, let's say you enter the number 4.760546328934 in an Excel worksheet cell. This number is automatically converted to fifteen-digit precision by Excel. This means that it is stored as 4.76054632893400 internally. How you display that number, however, is entirely up to you. You must remember, as well, that regardless of how the number is displayed, Excel will use the entire fifteen digits in performing calculations.

When you first enter the number, Excel displays exactly what you entered (to fifteen digits) in the Formula bar at the top of the screen. What is displayed in the cell is determined by the formatting you have applied to the cell. If the format is set to General, then Excel displays only eleven digits, and counts the decimal point as one of those digits. Thus, what you see displayed would be 4.760546329. Notice that this is a rounded value, which is consistent with how Excel displays shorter numbers, as well.

If you want to change the formatting applied to a number in a cell, you can do so by choosing Cells from the Format menu, and then clicking on the Number tab. (Formatting cells is discussed in other ExcelTips issues.)

ExcelTips is your source for cost-effective Microsoft Excel training. This tip (1982) applies to Microsoft Excel 97, 2000, 2002, and 2003.

Author Bio

Allen Wyatt

With more than 50 non-fiction books and numerous magazine articles to his credit, Allen Wyatt is an internationally recognized author. He is president of Sharon Parq Associates, a computer and publishing services company. ...

MORE FROM ALLEN

Viewing Your Document Full-Screen

Want to see the absolute most of your document that you can? Then you need to become familiar with the full-screen ...

Discover More

Don't Update Links to Other Programs

If you have links in your workbook to data in other workbooks, you may want to control whether Excel updates those links ...

Discover More

Determining If a Year is a Leap Year

Need to figure out if a given year is a leap year? It's not as easy as you think! This tip provides a plethora of ways ...

Discover More

Solve Real Business Problems Master business modeling and analysis techniques with Excel and transform data into bottom-line results. This hands-on, scenario-focused guide shows you how to use the latest Excel tools to integrate data from multiple tables. Check out Microsoft Excel 2013 Data Analysis and Business Modeling today!

More ExcelTips (menu)

Getting Rid of the Startup Spreadsheet

When you start Excel, you normally see a blank worksheet displayed. Here's how to get rid of it.

Discover More

Changing the Color of Worksheet Gridlines

Want the gridlines in your worksheet to be a different color? You aren't limited to stodgy black; Excel lets you make ...

Discover More

Precision in Excel

The best way to understand why Excel provides the answers it does is to examine how it handles precision. This tip ...

Discover More
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

View most recent newsletter.

Comments

If you would like to add an image to your comment (not an avatar, but an image to help in making the point of your comment), include the characters [{fig}] (all 7 characters, in the sequence shown) in your comment text. You’ll be prompted to upload your image when you submit the comment. Maximum image size is 6Mpixels. Images larger than 600px wide or 1000px tall will be reduced. Up to three images may be included in a comment. All images are subject to review. Commenting privileges may be curtailed if inappropriate images are posted.

What is 7 - 7?

There are currently no comments for this tip. (Be the first to leave your comment—just use the simple form above!)


This Site

Got a version of Excel that uses the menu interface (Excel 97, Excel 2000, Excel 2002, or Excel 2003)? This site is for you! If you use a later version of Excel, visit our ExcelTips site focusing on the ribbon interface.

Newest Tips
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

(Your e-mail address is not shared with anyone, ever.)

View the most recent newsletter.