![]() ![]() When you enter a number into Excel, it automatically converts the number to fifteen digits of precision-regardless of what you intended. Computer chips, on the other hand, deal in quite a different plane of precision. ![]() This means that we look at 4.76, and we know it is accurate to only two decimal places because that is all we can see and write. Humans generally store numbers (in our brains) as they are written. Jameson made such a big deal about the issue? The reason is because of the way in which calculators and spreadsheets store numbers and the way in which humans store numbers. Why, you might ask, is this true, particularly when Ms. To them, there is no difference between 4.76 and 4.760. Thus, 4.760 is more precise than 4.76 because of the number of significant digits each contains.Įnter the world of digital calculators and (by extension) spreadsheets, such as Excel. The reason, it turns out, is that 4.76 is precise (accurate) to only two decimal places, while 4.760 is precise (accurate) to three decimal places. Jameson, the math teacher, would tell us that 4.76 was different than 4.760. Thus, we often saw what at first appeared to be confusing information. When we were in junior high school, each of us learned the meaning of precision from the standpoint of math: it defined a degree of accuracy of a number.
0 Comments
Leave a Reply.AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |