Using Scilab or another program to download stock price data

[gmelancon]

I am currently in the process of coding a program that will analyze stock data from approximately 800 stocks to make estimated predictions of the profitability and timeframe of the profit, for my personal use. My current setup involves an excel macro to download the data, then pulling it into scilab for analysis. With the amount of data involved, I can only put so much into the excel file before it overloads scilab's ability to read it, and so I have had to split the excel macros into 20 different files, with each having to be activated and saved individually. This is obviously less than ideal, although still better than manual analysis of each stock. My question is, is there a way to use scilab to get the data directly, and skip excel entirely? If not, then what are some methods of programming that would allow me to download the data, and manipulate that data with the use of matrices, preferably without having to manually code the matrices, such as Matlab or Scilab. Open source or otherwise free compiler systems/programming languages are preferred as well.

 

I am a mechanical engineering student, not computer science, so I am not an expert, nor do I intend to be. I only need as much information as is required.

 

Thanks in advance for any help you can offer, as it will greatly decrease the time I spend researching.

[Sensei]

If it's Excel issue,

you know, Excel is not the only one application on the world.

I am using OpenOffice SpreadSheet.

You can try it to see whether it works better than Excel for free

http://www.openoffice.org

 

Does Scilab support CSV file format?

You can code downloading app, which is writing CSV, and then use that with SciLab, if it does.

 

Open source or otherwise free compiler systems/programming languages are preferred as well.

 

Visual Studio Express is free...

[gmelancon]

The only issue with excel is that the method I used to write the macro meant it was too big to compile, so it had to be split up into 20 macros. Then scilab could not read a single excel file that had all 20 macros as well as the stock data. I could try OpenOffice, but if it uses a similar compiler, I imagine size will still be an issue. Also, Scilab is able to read the file format that excel uses (.xls), does OpenOffice spreadsheet also use that format? I do not know if Scilab can use a CSV file directly, but I can look into it. As for Visual Studio Express, does that use the same language as the one in excel macros? Isn't it limited in functionality? I have considered using something of the sort, and getting a module that allows for matrix mathematics, but I was unsure if it would have the same capabilities as the excel macro for downloading the data.

 

Thanks

 

With some tinkering, I believe I could get scilab to use a CSV the way I need it to.

Edited February 20, 2015 by gmelancon

[Sensei]

The only issue with excel is that the method I used to write the macro meant it was too big to compile, so it had to be split up into 20 macros. Then scilab could not read a single excel file that had all 20 macros as well as the stock data. I could try OpenOffice, but if it uses a similar compiler, I imagine size will still be an issue. Also, Scilab is able to read the file format that excel uses (.xls), does OpenOffice spreadsheet also use that format? I do not know if Scilab can use a CSV file directly, but I can look into it.

CSV means comma separated values.

http://en.wikipedia.org/wiki/Comma-separated_values

I wrote about it, because it's easy to make loader/writer for it in any programming language.

Excel/OpenOffice obviously support it.

 

As for Visual Studio Express, does that use the same language as the one in excel macros?

Of course not.

It's C/C++/C# normal software compiler..

 

 

Isn't it limited in functionality?

In what sense "limited in functionality"?

 

You can always buy pro version.

 

Every application on this world written for Windows in C/C++ can be compiled by Visual Studio..

Everything what you use everyday is most likely written in it..

 

 

I have considered using something of the sort, and getting a module that allows for matrix mathematics, but I was unsure if it would have the same capabilities as the excel macro for downloading the data.

You will have to write everything by yourself.

Code downloading stuff from net, then writing files, and analyzing..

 

In C#/Managed C++ code downloading file from URL is simply one row of code f.e.:

 

String ^str = System::Net::WebClient::DownloadString::DownloadString( "http://www.google.com");

 

Then you split string to rows, then you analyze rows, etc. and write output file..

[Strange]

I have never used Scilab, but its remote file access module should allow you to do what you want:

http://forge.scilab.org/index.php/p/scicurl/

 

Otherwise, it might be better (far more efficient) to use a general purpose programming language (Python?) to pull the data to feed into Scilab.