I am currently in the process of coding a program that will analyze stock data from approximately 800 stocks to make estimated predictions of the profitability and timeframe of the profit, for my personal use. My current setup involves an excel macro to download the data, then pulling it into scilab for analysis. With the amount of data involved, I can only put so much into the excel file before it overloads scilab's ability to read it, and so I have had to split the excel macros into 20 different files, with each having to be activated and saved individually. This is obviously less than ideal, although still better than manual analysis of each stock. My question is, is there a way to use scilab to get the data directly, and skip excel entirely? If not, then what are some methods of programming that would allow me to download the data, and manipulate that data with the use of matrices, preferably without having to manually code the matrices, such as Matlab or Scilab. Open source or otherwise free compiler systems/programming languages are preferred as well.
I am a mechanical engineering student, not computer science, so I am not an expert, nor do I intend to be. I only need as much information as is required.
Thanks in advance for any help you can offer, as it will greatly decrease the time I spend researching.