My project

[bascule]

Well, here's something you probably haven't seen around here for awhile: a real computer science thread.

 

I'm working on a client/server streaming file transfer system which provides [math]O(log(n))[/math] scalability for n clients transferring a given file, as opposed to [math]O(n)[/math] for typical file transfer systems. It does this by leveraging all clients grabbing a file in a combined peer-to-peer network which offloads the file transfer process onto the peer network.

 

It's called DistribuStream, and you can read about it here:

 

http://pdtp.org/

[timo]

And what exactly scales with O(log(n)) ?

[bascule]

The server bandwidth required to serve [math]n[/math] clients

[the tree]

a real computer science thread.

*is in awe*

It does this by leveraging all clients grabbing a file in a combined peer-to-peer network which offloads the file transfer process onto the peer network.

So sort of along the lines of bittorrent?

Anyways, I'd be really interested to know how you arrived at O[log(n)]?

[timo]

The server bandwidth required to serve [math]n[/math] clients

 

C´mon, be little more elaborative. How does the network work? In what respect is any bandwidth reduced? If n people want to dowload a file of size X, I am pretty sure you cannot push the total amount of data transfer below nX, so either you are reducing the overhead caused by [??] or you are outsourcing tasks from the server (what´s that?) to the clients (is it server-client based at all?) or your network has a different strcuture than O(n) networks (what´s the structure of both network types?). There´s a lot of stuff that could be explained here and I am pretty sure that quite some people might be interested to learn a bit about file transfer systems.

[bascule]

C´mon, be little more elaborative. How does the network work? In what respect is any bandwidth reduced? If n people want to dowload a file of size X, I am pretty sure you cannot push the total amount of data transfer below nX, so either you are reducing the overhead caused by [??] or you are outsourcing tasks from the server (what´s that?) to the clients (is it server-client based at all?)

 

Yes... it's pretty extensively documented on the web site... however:

 

So sort of along the lines of bittorrent?

 

Hit the nail on the head. It's just a centrally organized BitTorrent which uses in-order transfers.

 

Essentially, it's a server-organized peer network for distributing a central repository of files.

 

Anyways, I'd be really interested to know how you arrived at O[log(n)]?

 

Well, I'm trying to apply a system for categorizing algorithm complexity to bandwidth consumption. I guess what I really should be saying is that server bandwith scales logramithically rather than linearly with the number of clients. There's the overhead of control messages, but that's substantially less than serving the actual data (which the server will still do some of in order to guarantee QoS and to make up for data which is no longer available within the peer network)

 

or your network has a different strcuture than O(n) networks (what´s the structure of both network types?). There´s a lot of stuff that could be explained here and I am pretty sure that quite some people might be interested to learn a bit about file transfer systems.

 

This is essentially a "trickle-down" system where peers with data farther in the file is passed from clients who do have it to those who don't (after initial "seeding" from the server). It's like BitTorrent, but instead of being a big swap meet, you can think of it more like letting hundreds of people read a book at the same time by giving out a few copies of each page to people in line and having them pass the page to the next person when they're done reading it.

[whap2005]

What is your role in this project. Are you asking for help with it? What do you need, programming help?, bandwidth? Hardware?

 

It sounds interesting and practical.

[bascule]

What is your role in this project.

 

Architectural and managerial. I designed it, and now I'm overseeing its execution. I'm doing no coding on it whatsoever (per my boss's request).

 

Are you asking for help with it? What do you need, programming help?, bandwidth? Hardware?

 

No, it's a pet project of mine being implemented by students at the local university as their senior project. Hardware/bandwidth are being provided through my company.

 

It sounds interesting and practical.

 

If all goes right, it should quickly become the biggest bandwidth consumer on the Internet.