Update: 3/7/2005, The torrent in this entry was just audio only, but Thomas Winningham has gotten permission from both Bram and Stanford ("Stanford holds copyright on the material but returns the copyright immediately to the speaker, that is, Bram. Get him to agree and go ahead.") to post their video as a torrent on Prodigem. Cool! Updated again since that video posted seems to only have the first 10 minutes. Anyway, the audio is below, or just check out my notes.

Bram Cohen gave a technical talk on Bit Torrent yesterday at Stanford. I had planned to make video from it available, but the video I captured somehow got corrupted (boo Panasonic). I salvaged the audio from the video and have released that via a torrent under a creative commons license (with Bram's approval). The audio is a bit low. It's okay, though, as I didn't realize that Stanford would be making it's video available to the general public (though in crummy windows streaming format). Here are some notes:

- Academic setting ... so how to benchmark/measure bit torrent
- benchmarking is hard because it needs to be like the internet (buy a bad router)
- key problem among swarming software is how to get everyone involved to maximize upload, people don't realize

- Single seeder problem
- must be careful not to at first trade with people who are likely to disappear

- Bit Torrent extremely non-cooperative
- each peer in it for himself
- tit for tat
- editorial note: isn't this cooperative? Peter Kollock: tit for tat as the optimal cooperative strategy

- How to deal with people behind and not behind NAT

- Centralized tracker is needed to produce randomized graph so as to avoid
network segmentation
- gossip (peers telling peers about other peers with content) very easily segments the network such that pieces of the content get isolated into islands

- Choking Algorithm
- sophistication
- people like to pretend it doesn't exit
- lots of use of made up magic numbers
- eg. how long to wait for reciprication?
- motivation opaque
- methodology (the traditional approach) is Bram firing up a client and observing behavior
- lots of room for study

- TCP does not look like RPC calls (BitTorrent treats TCP like a black box)
- don't avoid making a state machine, because no matter what you'll end up with a state machine anyway
- why threads are a bad idea

- Magic numbers
- makes them up based on what works
- pulls them from his "magic ass"
- if you need a magic number feel free to ask him for one

- Estimated Time Left Algorithm
- never gotten any fan mail on how well it works
- lots of effort and thought put into making this work sanely
- any time you see a computer telling you time left it is lieing
- research needs to be done on better algorithms
- would gladly place your better algorithm into bit torrent
- problem at end about going down 2 seconds per every 1 second
- tradeoff between smoothness now vs. smoothness later

- Current Transfer Rate Algorithm
- its a mess
- very important for tit-for-tat to work

- Bad idea to be downloading too many torrents at one time (e.g. 5)

- Peers at first never randomly tried new connections
- added optimistic unchoke to solve this
- if new person recipricates then continue
- otherwise move on to the next person
- may unchoke 4 or 7 clients depending
- it's voodoo
- nobody has seriously studied this

- Piece Selection Algorithm
- trade off between finishing the piece you are currently downloading vs going after a more valuable piece
- priority is currently finishing a piece you started even if many others have it
- downloading from the beginning of the content for everyone is a maximally bad strategy

Q: Who has what pieces is not centrally known?
A: Way too much overhead. Trackers currently put up 1/10,000th of all bandwidth used in torrents, yet people still complain.

Q: What if peers tell each other which magic numbers to use at the moment?
A: Way too clever. Peers don't trust each other.

Q: As for not trusting, do you have a specific model in mind? Stock market?
A: Want something that behaves like a commodity. Though bandwidth does not behave like a commodity at all.

- Bit Torrent is very much a reliability application
- how to deal with failures in the network
- google as a corp does this similarly to save cost

- Anyone who claims their app can scale to 100 times what they've tested is smoking crack
- 2nd order effects become 1st order effects
- torrents can handle 10,000s of peers before tracker gets overloaded
- need serious engineering to get to 1,000,000

Q: Exeem?
A: Totally irrelevant.

Q: Legal issues?
A: None. Develops for engineering not for avoiding legal liability.

- Gossip Algorithms
- each peer tells peers works terribly
- better strategy is to look for the least connected person within a peers of peers group and try to connect to them to get their peer list
- how to handle new peers with little content complicates things greatly

Q: Documented all of these anecdotal observations?
A: No, really bad at doing that. Just wade into the code base though some stuff is tricky. Implementation is well encapsulated. Maybe set up a FAQ.

By Gary Lerhaupt, 04:53 PM in torrents | Comments (4)  

Wow!!! Nice resource! Good job!

Posted by: Thomas at February 18, 2005 01:12 PM

man the video torrent is so much better.

http://netnews.nctu.edu.tw/~gslin/tmp/050216-ee380-100.wmv.torrent

Posted by: dan at March 7, 2005 01:40 PM

The video torrent available at 'get that torrent here' and located Prodigem at is only 8MB and quits after about 10 minutes. It also has no audio in Windows Media Player for me but does have audio in MPlayer

Posted by: Darrell at March 7, 2005 05:02 PM

I concur with Darrell, the video is incomplete, only roughly the first 10 minutes. I had no trouble with audio though.

Posted by: adam at March 7, 2005 09:07 PM