A development idea to aid in ROM distribution, solve bandwidth issues
If you flash a lot of ROMs, you’ve probably run into the issue at least once of the server being down, corrupted or incomplete downloads, or multiple 600MB ROM updates waiting for you to download. ROM distribution is currently a pain.
I’m going to talk about a program and method that doesn’t exist in this form yet, but should.
The problems
One of the major issues with ROM distribution is bandwidth and single points of failure. The typical scenario is this: a developer uploads his or her ROM to a server, checks the MD5 or re-downloads to compare, and then gives out the link to the new ROM in a development thread or updater app. Then hundreds, if not thousands, of people pile on to download the ROM, and the server that’s providing the bandwidth is overwhelmed.
That scenario is one of the reasons Google Play rolls out updates slowly to end users now. One billion people want the new version of Google Maps, but that’s not going to happen with a reasonable speed if everyone has access to it on Google’s servers at the same time.
The other major issue is these frequently updated 600MB ROMs are eating money out of the file hosts’ wallets. Two banner ads and a ten second delay asking people to donate isn’t a business model that can sustain a daily pull of several terabytes of data over the net for one ROM.
Users are encouraged to donate both at the point of download and to the developer, or the developer is paying for bandwidth and hosting.
The solution
I’ve discussed torrenting of ROMs before. The long and short of that is by torrenting a ROM, people can donate bandwidth to help get it to other people who want or need it. A person doesn’t have to feel like they’ve taken anything if they’ve given back as much or more than they downloaded. Unfortunately, even with a more tech-astute crowd in the root world, torrenting has not been greeted with the open arms it deserves.
There’s also a misconception that torrenting is somehow illegal, which it’s not – just a lot of people use it that way.
While we did see one ROM torrent site pop up about a week after I posted about torrenting ROMs on Twitter, it still lacked in elegance as the ROM developers had to have access, people were still bombarded with ads, and a computer was usually required. There was also the problem that the ROM was now out of control of the developer and if any issues arose in the upload, users were now getting perfectly transmitted corrupted data. The developer was out of control also if anything happened where they were unable to reach the site. It added another point of failure.
So, to summarize, torrents:
- let each downloader contribute to sending the ROM to someone else.
- decentralize distribution.
- verify that the file you downloaded contains no CRC or transmission errors.
- mirror the developer’s image exactly.
- allow for replacing damaged files.
- transfers continue on even if developer’s distribution computer breaks.
- allow developer to send only changed data.
That last part is the big one here. I have lost count of how many ROMs I’ve seen that were released and, by the time I downloaded the 500MB+ of the ROM, another one had been released due to an error in a very small text file. We have things like SVN that do this already; however, that requires a bit of an initial set up.
And finally, while torrenting used to be an activity that required a computer, there are several torrent clients and source available for Android now.
The ROM distribution implementation
Imagine this scenario: a user comes to a development thread, they’re greeted with a notice that if they want the ROM, they can download it here (perhaps AndroidFileHost) or if they’re already a user they can update it via the updater. For the more tech savvy, a magnet torrent link is thrown in there to take complete advantage.
Once the ROM is installed, much like you got with CyanogenMod Statistics, an app comes up and asks if you would like to donate some bandwidth, and how you’d like to donate it. Options I can think of on donating bandwidth are: WiFi Only, WiFi Only while charging, WiFi only while charging and screen off, and donate bandwith while roaming only because I want to be terminated early from my contract.
A small torrent client seeds the ROM out at some maximum bandwidth you specify while your phone is on the charger.
To keep the developer in control, the app checks the developer’s control page regularly to see if ROM_VERSION_THISONE is allowed to be seeded any more, or if it was too buggy and needs to be removed. The check also serves to notify a user that there’s an update available, and should the option be set, downloads it via torrent for the end user.
At any time the end user can say that they’re done donating bandwidth and that is that.
Benefits
- Decentralized file distribution – keeps going even if the developer’s computer is not
- Tiny updates – if the ROM is unzipped, only the changed files will be torrented
- Ten minute source-side setup using uTorrent or similar
- The copy the ROM developer seeds is the copy you get
- Smaller downloads for updates
- Developer still in complete control via the check in for updates page
- Users who update can get it in next to no time
- Unlike conventional distribution, the more popular the ROM is the faster it downloads due to more seeds
- Users can participate passively
Potential benefits
So, in my head, the scenario is you’ve got an unzipped ROM package sitting in a directory. The traditional method of updating would be to package this up into a large zip file and then flash it via recovery mode. But there’s no reason there needs to be a flash of data that’s mostly untouched. Only the modified data needs to be pushed, and that’s usually less than 1/20 of updates.
So, while the phone is running I envision three scenarios – one in which a full recovery ROM zip is built, one in which a smaller update package is build for flashing in recovery, and finally one in which an updater script is executed on boot to only copy over modified data.
The same basic method could work for the carriers when they want to push an OEM update out and don’t feel like pushing petabytes of data over the air and instead want to push the burden off to ISPs.
Wrap-up
That’s my idea, I’m hoping a developer takes it and runs with it, as I do not want to tackle this as my first Android programming scenario. It’s a little grander than the Hello World program I’ve been working on for the past little bit.
Any input, suggestions for things I may have missed, or claims against my ignorance or lineage are welcome in the comments section.
