Author Topic: Cheapest file hosting with reliable hotlinking?  (Read 6549 times)

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Cheapest file hosting with reliable hotlinking?
« on: October 02, 2010, 03:14:59 pm »
Right now we're using Amazon S3.  This is costing us around $100 per month for about 600GB of transfer per month, roughly.  That's before what's coming up: a slew of large update patches for 3.700.  We can't afford to pay more, and really what we're paying now is more than I'd like.  The cost is almost exclusively from patches, I think, and it's bandwidth -- not storage space -- that is our cost concern.

I'm thinking of switching to Diomede for the upcoming stuff, as that would cut bandwidth costs by 25%, which is significant.  That's for patches.  Anyone know of anything cheaper/better?

For the actual installers, I'm thinking of using a free plan on MediaFire from now on.  Previously it had been Amazon S3, and our costs certainly jumped when Tidalis came out and was in beta, etc.  Of course, there are other similar sites that are free, maybe I should go with one of those.  FileDropper is certainly interesting (and it supports larger uploads).

With all the recent bug reports for 3.700, I know we're in store for basically daily releases for a while.  What can't be allowed to happen because of this is us getting hit with hundreds of dollars of bandwidth bills.  Any suggestions from folks?
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Fleet

  • Hero Member
  • *****
  • Posts: 633
Re: Cheapest file hosting with reliable hotlinking?
« Reply #1 on: October 02, 2010, 03:46:43 pm »
Have torrents been considered (for installer, maybe not for each patch)? They could substantially reduce your bandwidth costs. Making sure users are getting legit software may be an issue, but checksums / hashes could be posted on your site. Sure, it won't be the way everyone gets it, but those of us that are savvy would love to save your company a few bucks, and verifying md5 is simple.

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Cheapest file hosting with reliable hotlinking?
« Reply #2 on: October 02, 2010, 06:00:27 pm »
Well, for the patches it really needs to be able to directly download them, and for the installers there are pretty straightforward options around that are no-cost already.  To be honest, I find torrents extremely irritating, as the download speed can just be so unreliable and it sucks up my own upload bandwidth at the same time, etc.  If it was a low/no cost alternative for the patches themselves that would be one thing, but that wouldn't help there.  Good thought, though!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Ancestral

  • Newbie Mark III
  • *
  • Posts: 40
  • Dyson !
Re: Cheapest file hosting with reliable hotlinking?
« Reply #3 on: October 02, 2010, 06:30:50 pm »
I don't know what compression algorithm is used for AI War Installers and Archives but you can reduce your bandwith usage if you upload more little files with a compression algorithm like the LZMA-2.
Dysons are coming !

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Cheapest file hosting with reliable hotlinking?
« Reply #4 on: October 02, 2010, 06:36:08 pm »
The installers I tried with LZMA, but it turned out ZIP was smaller in this specific case -- which I thought was odd.  For the actual patches, those are zip files because that's what the patcher can read, unfortunately...
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Ancestral

  • Newbie Mark III
  • *
  • Posts: 40
  • Dyson !
Re: Cheapest file hosting with reliable hotlinking?
« Reply #5 on: October 03, 2010, 02:27:06 am »
I did a little test :
AI War 3.700 Installer size is 95.6Mo. I installed it in a folder an doing an other installer with extracted files with Inno Setup and compression LZMA2/Ultra, the size of this new installer is 82.2Mo. It is almost 15% of size down. That is a potential 15% bandwidth gain on installer download. It is not very big, but is it always good to take.

For patchs, use LZMA2/Ultra may be more difficult to do, but this can be good for money saving.
Dysons are coming !

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Cheapest file hosting with reliable hotlinking?
« Reply #6 on: October 03, 2010, 02:33:10 am »
Interesting.  With Bitrock installbuilder, at least for Tidalis, LMZA was slightly larger.  That said, we will pay $0 for bandwidth on installers, so it's fairly moot to me.  The patches are where the trickiness comes in...
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline RCIX

  • Core Member Mark II
  • *****
  • Posts: 2,808
  • Avatar credit goes to Spookypatrol on League forum
Re: Cheapest file hosting with reliable hotlinking?
« Reply #7 on: October 03, 2010, 03:22:09 am »
I'd recommend Mediafire. You'd need a Gold plan, but mediafire is bar none the best upload service i've used.
Avid League player and apparently back from the dead!

If we weren't going for your money, you wouldn't have gotten as much value for it!

Oh, wait... *causation loop detonates*

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Cheapest file hosting with reliable hotlinking?
« Reply #8 on: October 03, 2010, 03:28:42 am »
Well, if I just had the patches (which is the plan anyway), that lone terabyte might be sufficient.  We get pushing on up towards that some months, though -- about 600 GB on Amazon S3 is our average, lately, but some of that is installers I'd take down -- or rather, that I wouldn't use hotlinking for.  The more I think about this, the more I think that's actually the way to go for all of it -- thanks!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline AcidWeb

  • Full Member
  • ***
  • Posts: 127
Re: Cheapest file hosting with reliable hotlinking?
« Reply #9 on: October 03, 2010, 04:45:14 am »
I also think that torrent is a good idea.

If you will start using someday I provide for free 24/7 100mb seed located in France.

Offline eRe4s3r

  • Core Member Mark II
  • *****
  • Posts: 2,825
Re: Cheapest file hosting with reliable hotlinking?
« Reply #10 on: October 03, 2010, 09:57:05 am »
I agree with mediafire, definitely the "easiest" http file hoster

Though torrents would work nicely as well, could just click on them links and have my NAS download the torrent and move it over to my archives all automagically.. so actually torrents would be nice ;P But whether thats workable for maybe daily patches...

If its a big release (200mb+) torrents would be much preferred to any filehoster though, as torrents are resume and corruption safe.

As Tracker theres always http://www.gameupdates.org/
Proud member of the Initiative for Bigger Weapons EV. - Bringer of Additive Blended Doom - Vote for Lore, get free cookie

Offline carlosjuero

  • Full Member Mark III
  • ***
  • Posts: 211
Re: Cheapest file hosting with reliable hotlinking?
« Reply #11 on: October 03, 2010, 01:34:40 pm »
One option you could consider (at least for beta downloads/temporary - or even permanent if you use DNS redirect) is something I stumbled across called orangeserve.com. They boast unlimited bandwidth and space - haven't messed much with speed tests to see what that is like, but it most likely would be faster than Mediafire (I do not like Mediafire at all :/ - I rarely get good download speeds from them)

Offline o1knives

  • Newbie Mark II
  • *
  • Posts: 14
Re: Cheapest file hosting with reliable hotlinking?
« Reply #12 on: October 03, 2010, 02:35:36 pm »
Not so much a file host, but for patches have you considered something like rsync?  (And I would second talk of bittorrent.  I believe there are ways of setting it up so that your S3 store will serve as one of the clients.)

TheMachineIsSentient

  • Guest
Re: Cheapest file hosting with reliable hotlinking?
« Reply #13 on: October 03, 2010, 03:18:17 pm »
I could see torrents working if they were integrated with the application, kind of like what blizzard does with starcraft 2. If I have to go downloading a bit torrent client ( I do not have one, I don't even like them), and then put together the game myself, it would really discourage me from updating.

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Cheapest file hosting with reliable hotlinking?
« Reply #14 on: October 04, 2010, 05:12:58 pm »
We don't have a way of integrating a torrent client that I'm aware of, and we can't expect nearly the seeding support that Blizzard would get with their stuff.  orangeserve.com is interesting, but they seem a bit new and rather... unpolished in a lot of areas.  I'd be worried they would disappear on me, honestly.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!