Precision and Number Display

by Allen Wyatt
(last updated March 23, 2013)

In a different ExcelTip you learned how Excel handles the internal precision of numbers you enter. With that understanding firmly in mind, you should also understand that the internal precision is entirely different from what Excel displays. While this may sound confusing, it is analogous to how Word deals with text: Text and the display of text (formatting) are intrinsically different. In Excel, numbers and the display of numbers are also intrinsically different.

For instance, let's say you enter the number 4.760546328934 in an Excel worksheet cell. This number is automatically converted to fifteen-digit precision by Excel. This means that it is stored as 4.76054632893400 internally. How you display that number, however, is entirely up to you. You must remember, as well, that regardless of how the number is displayed, Excel will use the entire fifteen digits in performing calculations.

When you first enter the number, Excel displays exactly what you entered (to fifteen digits) in the Formula bar at the top of the screen. What is displayed in the cell is determined by the formatting you have applied to the cell. If the format is set to General, then Excel displays only eleven digits, and counts the decimal point as one of those digits. Thus, what you see displayed would be 4.760546329. Notice that this is a rounded value, which is consistent with how Excel displays shorter numbers, as well.

If you want to change the formatting applied to a number in a cell, you can do so by choosing Cells from the Format menu, and then clicking on the Number tab. (Formatting cells is discussed in other ExcelTips issues.)

ExcelTips is your source for cost-effective Microsoft Excel training. This tip (1982) applies to Microsoft Excel 97, 2000, 2002, and 2003.

Author Bio

Allen Wyatt

With more than 50 non-fiction books and numerous magazine articles to his credit, Allen Wyatt is an internationally recognized author. He  is president of Sharon Parq Associates, a computer and publishing services company. ...

MORE FROM ALLEN

Viewing Your Custom Styles

If you develop a set of preferred styles, you may want to use those styles with a document you receive from someone else. ...

Discover More

Styles for Lines, Dashes, and Arrows

Create a simple drawing object, and Excel makes some assumptions about how that object should appear. Excel provides a wide ...

Discover More

Alerts About Approaching Due Dates

You may use Excel to track due dates for a variety of purposes. As a due date approaches, you may want that fact drawn to ...

Discover More

Excel Smarts for Beginners! Featuring the friendly and trusted For Dummies style, this popular guide shows beginners how to get up and running with Excel while also helping more experienced users get comfortable with the newest features. Check out Excel 2013 For Dummies today!

MORE EXCELTIPS (MENU)

Precision in Excel

The best way to understand why Excel provides the answers it does is to examine how it handles precision. This tip provides a ...

Discover More

Changing the Color of Worksheet Gridlines

Want the gridlines in your worksheet to be a different color? You aren't limited to stodgy black; Excel lets you make them ...

Discover More

Getting Rid of the Startup Spreadsheet

When you start Excel, you normally see a blank worksheet displayed. Here's how to get rid of it.

Discover More
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

View most recent newsletter.

Comments for this tip:

There are currently no comments for this tip. (Be the first to leave your comment—just use the simple form above!)

This Site

Got a version of Excel that uses the menu interface (Excel 97, Excel 2000, Excel 2002, or Excel 2003)? This site is for you! If you use a later version of Excel, visit our ExcelTips site focusing on the ribbon interface.

Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

(Your e-mail address is not shared with anyone, ever.)

View the most recent newsletter.

Links and Sharing
Share