Browse Source

write out architechture for purpose-built tuber protocol

golang
forest 3 weeks ago
parent
commit
25a2b1192a
2 changed files with 58 additions and 5 deletions
  1. +1
    -0
      .gitignore
  2. +57
    -5
      ReadMe.md

+ 1
- 0
.gitignore View File

@ -0,0 +1 @@
node_modules

+ 57
- 5
ReadMe.md View File

@ -1,18 +1,70 @@
# 🍠Tuber
### Go Viral from a "Potato" Computer Hosted in Mom's Basement:
### Take our Beloved "Series of Tubes" to Full Power
### Serve Your Media Without Limits From a "Potato" Computer Hosted in Mom's Basement:
#### Take our Beloved "Series of Tubes" to Full Power
> _Tubers are enlarged structures used as storage organs for nutrients [...] used for regrowth energy during the next season, and as a means of asexual reproduction._ [Wikipedia](https://en.wikipedia.org/wiki/Tuber)
---------------------------------------------
Tuber is a stand-alone HTTP "middleware" or "reverse-proxy" server designed to enable a [WebTorrent](https://webtorrent.io/)-based P2P CDN (peer to peer content delivery network) on top of ANY* existing web server or web application, without requiring any modifications to the application being accelerated.
Tuber is a stand-alone HTTP "middleware" or "reverse-proxy" server designed to enable a P2P CDN (peer to peer content delivery network) on top of ANY* existing web server or web application, **without requiring modifications** to the application being accelerated.
Tuber works transparently in the web browser; when someone visits your site, they can instantly access your video, audio, or other downloadable content, including live-streamed content. Tuber is very similar to [WebTorrent](https://webtorrent.io), but in order to support live content and decrease latency (offer a seamless "snappy" experience to web surfers) it uses a purpose-built protocol.
Tuber works in the web browser; when someone visits your site, they can instantly access your video, audio, or other downloadable content accelerated by BitTorrent. Tuber provices a seamless experience from the web surfer's perspective 🏄.
Tuber's protocol includes a server that acts as a centralized authority for the entire network of peers. It's not really a "peer-to-peer" protocol, in fact, it has more in common with scalabale distributed streaming protocols like [Apache Kafka](https://kafka.apache.org).
The original post where I described this idea was published in 2017: https://sequentialread.com/serviceworker-webrtc-the-p2p-web-solution-i-have-been-looking-for
Since then, others have made progress in this direction by leaps and bounds, also my own ideas have shifted ever since I saw the [Novage](https://github.com/Novage) github repos and demos proving that a responsive, instant-gratification-type experience is possible with WebTorrent.
Originally, the idea was to simply serve the content directly from the origin server over WebRTC, as a way to sidestep bandwidth limitations associated with [greenhouse](https://greenhouse.server.garden/) / [threshold](https://git.sequentialread.com/forest/threshold).
Since then however, my ideas have shifted a bit: the [Novage](https://github.com/Novage) and WebTorrent projects got me excited about P2P content distribution, where the origin server doesn't even need to handle all the content requests from all the web surfers any more.
\* Tuber uses a [ServiceWorker](https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorker) to make this possible, so it may not work properly on advanced web applications which already register a ServiceWorker.
# Tuber Architechture
#### Tuber Node Types:
- Permanent/App (Written in Golang, Pion WebRTC)
- Ephemeral/Web (Written in TypeScript, Browser Only)
#### Tuber Operation Modes:
- **Central Planner**
- Maintain Content and Peer Database
- Assign Peer to Peer connection candidates to Peers
- Assign content partitions to Peers
- Give priority instructions to peers
- Priority #1: "trusted" permanent peers first
- Priority #2: fastest uploaders second
- Collate and Compare metrics
- rank peers by longevity and upload bandwidth
- detect and mitigate lying peers?
- **Content Origin**
- HTTP reverse proxy server
- inject serviceWorker into the page
- track content that should be p2p-distributed
- "which content will be requested next?" prediction/mapping
- auto-detect HLS or other "stream-like" behavior & poll
- support webhooks for new content
- **Globally Dialable Gateway**
- STUN server
- If it's also a Content Peer, maybe it can serve files directly over HTTPS?
- **Content Peer**
- Can be registered as "Trusted" with Central Planner
- Partition / File Store
- WebRTC / WebSocket connections to other peers.

Loading…
Cancel
Save