herme3 Posted April 18, 2007 Share Posted April 18, 2007 I am putting a lot of data into Microsoft Excel that needs to be squared and totaled for a scientific experiment. However, the numbers I get from Excel differ significantly from the numbers I get from a graphing calculator and the scientific calculator in my computer. When I try 0.18^2 in Excel I get 0.0316 but I get 0.0324 on a calculator. When I add all the numbers together, they aren't even close. I've tried changing the number of decimal places in an Excel cell, but that doesn't make a difference. Does anyone have any suggestions? Link to comment Share on other sites More sharing options...
Sayonara Posted April 18, 2007 Share Posted April 18, 2007 Check what number system your calculator is using. Link to comment Share on other sites More sharing options...
insane_alien Posted April 18, 2007 Share Posted April 18, 2007 use open office? never seen any mathematical discretions with that Link to comment Share on other sites More sharing options...
Klaynos Posted April 18, 2007 Share Posted April 18, 2007 use a proper statistical program like R or origin. Link to comment Share on other sites More sharing options...
herme3 Posted April 18, 2007 Author Share Posted April 18, 2007 I tried Open Office, but I had the exact same problem. I know my calculator is correct because those numbers match my teacher's answers in the sample experiment he provided. It works when I only cross-reference one cell at a time. For example, I can say cell A1=0.18 and cell B1=A1^2 and then I will get the answer 0.0324 in B1. However, there is a lot of data and I have cells that cross-reference other cells with equations. For example, cell A1 has the number .5 and the average of all numbers in column A is .68. Cell B1 has the equation A1-.68 which is -0.18. Cell C1 has the equation B1^2 which is giving the incorrect answer of 0.0316. Link to comment Share on other sites More sharing options...
Klaynos Posted April 18, 2007 Share Posted April 18, 2007 what if you change it to B1 * B1? Link to comment Share on other sites More sharing options...
herme3 Posted April 18, 2007 Author Share Posted April 18, 2007 I tried that, but I still got the same incorrect answer. Link to comment Share on other sites More sharing options...
Klaynos Posted April 18, 2007 Share Posted April 18, 2007 Well without seeing what you're doing I can offer little help. Link to comment Share on other sites More sharing options...
herme3 Posted April 19, 2007 Author Share Posted April 19, 2007 You can visit http://eflierz.com/Ttest.xls to download the spreadsheet I'm working on. Also, here is a screenshot of it. Thank you very much for any help you can provide. Link to comment Share on other sites More sharing options...
Klaynos Posted April 19, 2007 Share Posted April 19, 2007 Yep I see you're problem streight away. It's not doing 0.11^2 Have a look at the number of decimal places showing in cell D5. Link to comment Share on other sites More sharing options...
herme3 Posted April 19, 2007 Author Share Posted April 19, 2007 Hi Klaynos, Thank you for your reply. I looked at the decimal places, and I see 2 after the decimal. I'm not very good at math, but should I change this? My professor wanted column D to have 2 decimal places and column E to have 4. On his sample answers, cell D5 reads "0.11" and cell E5 reads "0.0121". Link to comment Share on other sites More sharing options...
Klaynos Posted April 19, 2007 Share Posted April 19, 2007 Well, you need to enforce rounding if you want it to be identicle to his. At the moment your D5, displays the value 0.11, but it's actual values is: 0.108333333333333000 Which is the value it uses for all it's calculations. And it might only become 0 after that point because of a limitation in the presicion of openoffice.org which is what I used to open it. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now