Excel Calculations

[herme3]

I am putting a lot of data into Microsoft Excel that needs to be squared and totaled for a scientific experiment. However, the numbers I get from Excel differ significantly from the numbers I get from a graphing calculator and the scientific calculator in my computer.

 

When I try 0.18^2 in Excel I get 0.0316 but I get 0.0324 on a calculator. When I add all the numbers together, they aren't even close. I've tried changing the number of decimal places in an Excel cell, but that doesn't make a difference. Does anyone have any suggestions?

[Sayonara]

Check what number system your calculator is using.

[insane_alien]

use open office? never seen any mathematical discretions with that

[Klaynos]

use a proper statistical program like R or origin.

[herme3]

I tried Open Office, but I had the exact same problem. I know my calculator is correct because those numbers match my teacher's answers in the sample experiment he provided.

 

It works when I only cross-reference one cell at a time. For example, I can say cell A1=0.18 and cell B1=A1^2 and then I will get the answer 0.0324 in B1. However, there is a lot of data and I have cells that cross-reference other cells with equations.

 

For example, cell A1 has the number .5 and the average of all numbers in column A is .68. Cell B1 has the equation A1-.68 which is -0.18. Cell C1 has the equation B1^2 which is giving the incorrect answer of 0.0316.

[Klaynos]

what if you change it to B1 * B1?

[herme3]

I tried that, but I still got the same incorrect answer.

[Klaynos]

Well without seeing what you're doing I can offer little help.

[herme3]

You can visit http://eflierz.com/Ttest.xls to download the spreadsheet I'm working on.

 

Also, here is a screenshot of it. Thank you very much for any help you can provide.

 

[Klaynos]

Yep I see you're problem streight away. It's not doing 0.11^2

 

Have a look at the number of decimal places showing in cell D5.

[herme3]

Hi Klaynos,

 

Thank you for your reply. I looked at the decimal places, and I see 2 after the decimal. I'm not very good at math, but should I change this? My professor wanted column D to have 2 decimal places and column E to have 4. On his sample answers, cell D5 reads "0.11" and cell E5 reads "0.0121".

[Klaynos]

Well, you need to enforce rounding if you want it to be identicle to his.

 

At the moment your D5, displays the value 0.11, but it's actual values is:

 

0.108333333333333000

 

Which is the value it uses for all it's calculations. And it might only become 0 after that point because of a limitation in the presicion of openoffice.org which is what I used to open it.