Author Topic: Brainstorming: Curtailing Infinite World Folder Size Growth  (Read 10415 times)

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,050
Brainstorming: Curtailing Infinite World Folder Size Growth
« on: December 01, 2011, 09:56:52 AM »
One of the things that Keith and I had been thinking about with the Continents idea is that it's a way for us to split up the world into yet more component files (continent files) so that not all of it has to be in RAM at once.  And not all of it has to be synced across the network to new players, and all of that sort of thing.  Of course there are solid gameplay reasons for doing continents as well, and that's what we focused on in the other thread that was specifically about continents.

The above said, I'm starting to think that the technical reasons for continents were really missing the point.  Gameplay-wise they're a great idea and are still coming.  But in terms of whether we literally need/want to split up the world file... that's something that's a lot less clear to me.  Here's my thinking about this stuff.

THE BROAD DATA STRUCTURE
-----------------------------------------------
There are two different parts to any given world:

1. The central world/region index info (basically what you see on the world map, the macro game items, info about players, info about all NPCs in the game, etc).

2. The per-region detail data about what is actually in any given screen when you go there, etc.  This is the part that actually has the terrain maps and monster locations and info on loot drops and all that sort of thing.

The central indexes (world.dat and regions.dat) are normally stored uncompressed on disk because they need to be written frequently and it uses less transient RAM and CPU to not compress them.  In a world with 519 regions in it, I'm looking at about 320kb uncompressed on the disk, which compresses down to about 47kb.  Larger worlds compress at an even better ratio.

The data in the region detail folders actually is compressed on disk already, so it tends not to compress any further at all.  In these folders is a regioninterior.dat which really tends to only be a few kb at most, and then 0-n "chunk files" that are -- compressed -- something like 3-20kb each on average.  Every side-view screen you enter in the game is called a "chunk," and has one of these files associated with it.

WHAT WORKS WELL ABOUT THIS
-------------------------

Central World Indexes:

The central world index stuff contains enough to run the world map, to have NPCs be able to talk about the overlord by name, to have info about quests or the status of various locations, and so on.  But it really doesn't have much more than that. 

This is ideal from both a RAM standpoint and a transferring-in-multiplayer standpoint.  In terms of RAM, even a really massive world with a hundred thousand regions is probably only looking at 63mb of data on disk in an uncompressed format.  For comparison, an uncompressed AI War savegame can range from 20-90mb easily (which is why we compress them, typically to 1-8mb on average).

Given that there is incredibly, vastly much more data in play with AVWW worlds than AI War galaxies, this sort of segmentation of data is really needed for having a world that can approach anything remotely like infinity.  But of course there's a finite cap when you've got linear RAM use; if it's 63mb on disk, I'm betting it would be 200mb in memory (because there's a lot of calculatable data that we reconstruct when stuff is loaded off disk, to keep the disk footprint smaller).

All of this, again, is comparable with AI War so that's great.  Though we're talking right now about a world with a theoretical 100,000 regions in it.  To reach that size of world would take somewhere in the neighborhood of 5,000 civ levels.  And to reach THAT point, in the ideal balance we're going to be shooting for, would take about 15,000-20,000 hours of gameplay.  Multiplayer wouldn't make this happen faster.

The average super-hardcore AI War player seems to get about 300-600 hours of gameplay out of the game over a two year span.  So if we extrapolate back into AVWW, that would put them at, at best, something like level 400.  Now, of course we already have some players who have racked up over 100 hours in AVWW and are already level 200 or so.  The time between levels is currently a bit out of kilter, closer to 30 minutes per level for these folks.

So even if we take that more pessimistic number, and we look at them getting to level 1200 instead of level 400, that's still only about 24,000 region tiles on the world map.  That, in turn, is about 16mb of data uncompressed on disk for their central world indexes.  Compressed, that's something that can be synced to a new player connecting into a game in about the same time that an unreal tournament map could be synced, or a midsize AI War savegame.  That's not horrible for being after 600 hours of gameplay in a single world.

Per-Region Detail Data:

This is where things REALLY shine.  The only region data that's loaded off disk are the regions that a player is in at the moment.  Afterward, they are just dumped back to disk.  Same thing with the chunks -- someone is in a chunk if it's in memory, otherwise it's dumped back to disk within about 10 seconds.

That keeps the actual data about gameplay areas (as opposed to central world map kind of stuff) incredibly small.  And there's really no way to spike the RAM on that at all, aside from having a really huge number of players in a world at once, all in different chunks from one another.  Our estimates are that it would take somewhere in the neighborhood of 30-50 players to become a RAM issue with this on the server, with Unity's current mono heap limit of 800mb.

This is also something that is really cool for multiplayer, because as you transition between the various screens in side view, the server is only having to send you 3-20kb of data (on average) as you do so.  Keeps things snappy to say the least, and it really means that in terms of actual CHUNK data, the game really is practically unlimited in what it can store, even in multiplayer.

THE PROBLEM
-------------------------
The actual problem here is that the disk space (and to a lesser extent, RAM) requirements of a world's folder will grow linearly with time, despite all the above benefits of our model.  And I don't think there's a better model we could use, it's simply a fact that bigger worlds are... bigger.  All that data has to be stored somewhere.

Right now we're really minimizing the impact on RAM from growth of anything except the world map itself, but the more screens you visit the larger your disk footprint gets on the server or your client.  Which is not a huge deal since disk space is cheap and we're only talking hundreds of megabytes at most in most cases, but still.

If left to just grow unchecked, then eventually the world would get so large that:

1. The game would become unplayable because of running out of RAM.  I don't think this would happen until about 5,000+ civ levels into the game (if not even further out -- could be as far as 10,000+ levels), but that depends on exactly how the game continues to evolve.  And that's presuming that we're keeping to a relatively slow world land-area-size growth like we currently are, but honestly I think I might like to bump that up a bit so that you get more land a little faster than now, yet without running into RAM issues suddenly at level 2,500 instead of level 5,000 or whatever winds up being.  Again, the RAM thing is kind of a moving target because the game is still evolving and the actual average RAM footprint per region isn't yet fully known (but it's not going to do anything but grow, if AI War is any indication).

2. Additionally, as the world gets progressively larger, it is taking up more disk space.  Mainly in terms of the per-region detail data (mostly chunk files).  This isn't a huge problem, as I noted above, but it is annoying.  If you want to periodically back up your world folder, for instance, it's going to take progressively longer and longer to back up the larger your world gets.  And the most annoying part is that most of the data that it's spending so long backing up is old areas you haven't visited in dozens of hours, and probably never will visit again.

3. Players connecting into a multiplayer server will get increasingly long waits, and this will also impact other players who are currently playing on the server.  Transferring a few hundred kb to an incoming player is no big deal to get them up to speed on the world map state.  And once they are online, it is transferring updates in chunks to them.  And there are other optimizations we could make here to mitigate this, too, and Keith may already have made some of them (I can't recall anymore).  But the simple axiom is Big Worlds Are Big, and there's no escaping that to some extent.

WHY CONTINENTS DON'T HELP THIS
-------------------------
This whole linear-growth phenomomena is something we've obviously been putting a lot of thought into for a long time, and we've mitigated it to an extreme degree already, I think.  But our next planned mitigation, of splitting the world files into continent subdivisions, I'm realizing really doesn't help as much as I'd want.  It would help with initial multiplayer syncs to some extent, and it would certainly help with the RAM use of the central indexes since any continents that are not in use could be on disk.

However, each continent really represents a pretty large amount of data, so moving between continents could get annoying fast.  And there could be some severe limitations in what we can do cross-continent if we ever felt like implementing trade or something, since we can't do anything with settlements that aren't in RAM because they're on a different continent.

Then when we come back to multiplayer, if players are wanting to be on different continents to work on different side missions, then those are all in RAM anyway, and all you've really saved is some bandwidth.  Which is still useful, but there are easier ways to save bandwidth, like just sending regions that are actually in proximity to each client regardless of considerations about continents, etc.  We may actually already be doing that, that's certainly how we're doing updates about that data.

Coming back to solo play, when you think about mission time and how that's something that causes new side missions to pop up and disappear, that's another thing that would be somewhat problematic if older continents were not stored in memory at the time.  They'd get left behind in a major way.

And none of this addresses the problem of disk space at all, so backups still take linearly longer, etc.  Which is still really annoying!

WHAT REALLY NEEDS TO HAPPEN, IN SOME FORM
-------------------------
Old chunk data, and old region data, needs to get purged from the world at some point.  Plain and simple.  There are things that players simply don't care about, and that stuff should get removed so that there isn't linear growth anymore.  This does NOT mean removing old settlements or anything like that -- even key locations like old overlord keeps are trivial to keep, in terms of their data.

What I think is that there are some "key locations" that players wind up having longer-term interest in and wanting to maintain, while the rest of the "I was just off doing some scavenging or a minor side mission" kinds of areas can be dumped because the player has already forgotten about their details anyhow (and probably wouldn't recognize them if they returned to them anyway, if it's been long enough).

So.  This is what we're brainstorming about, and why.


MY IDEA: ONGOING CATACLYSM
-------------------------
Okay, so this is a post-disaster world, right?  But in a lot of respects the world really still is in the midst of a disaster.  In another thread, Keith and I were talking about having disasters that would strike older continents to up their region levels and basically those would purge whatever had been there.

Well... what if we took that a lot further than just meteor strikes or flooding or whatever, and made it so that after a certain amount of time had passed that continents became volatile and started to sink into the sea?  Bam, that's a huge purge of data right there. 

Once you have something like 5-6 continents, the oldest one goes into a crisis and will sink into the sea after some amount of mission time.  What do you do in the interim?  Well, you now have the fun of trying to rescue whatever parts of the continent you care about. 

Maybe you can magic the settlement(s) from there to any of the other continents of your choosing to save them and the citybuilding stuff that was going on with them.

Or maybe it's really more of a loss of the settlement itself, and you're just saving the people, but they bring across their know-how and goods and stuff and are able to quickly rebuild an equivalent-but-different-looking settlement in a new place.

Maybe you can choose a couple of other regions to also magically move to safety of another continent, but at the cost of a unit of mission time for each region you're saving.  So if there was some really iconic overlord keep that you want to retain access to, or you've got a near-infinite cave system that you're down 15 caverns into, you could move those to safety before the rest of the continent goes down into the sea.

Or if you're not sentimental, you just let the whole thing sink into oblivion and all those older NPCs die, too.  It's up to you.

But either way your world stays leaner in filesize, and actually with this sort of system the amount of playtime in this world moves a lot closer to infinite -- you'd start having problems at about region level 2.3 billion, rather than civ level 5,000.


Thoughts?  Other ideas?
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Bluddy

  • Sr. Member Mark III
  • ****
  • Posts: 433
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #1 on: December 01, 2011, 10:23:08 AM »
Your solution is exactly what I was thinking of as I read the description of the problem.

The problem I see is, why are you fighting the overlords when the cataclysm is going to destroy anything anyway?

Maybe this can be turned into a more central gameplay dynamic? The overlords aren't just oppressing the people, they're blocking off their escape route (which is magical and transports the whole settlement, or perhaps not -- just the people escaping). The thing tearing apart the world is the mystical "wind", and your windmills can only hold it back for so long. Your goal is to eventually find the mythical valley without wind, from which you won't have to escape any longer.

One effect that could happen over shorter time periods is that if you come back to an area you've already visited after a while, it'll be more run down. There will be more destroyed rooms and buildings. Chaos is slowly eating away at this world and the only thing you can do is escape. Eventually random tiles would disappear and whole continents would be torn apart.

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,050
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #2 on: December 01, 2011, 10:46:06 AM »
That's a really interesting twist on it!  I like how that ties together the wind and the overlords with this other new concept for the continents.  Very cool!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Underfot

  • Jr. Member
  • **
  • Posts: 57
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #3 on: December 01, 2011, 11:04:45 AM »
I think I'd be okay with losing regions from 100 game play hours ago and given how much of the moon is missing, I imagine a few de-orbiting chunks could wipe out a continent...
Rescuing a settlement from a continent could range from an enormous undertaking (everybody pack up and get on the Ark, the rains are coming) to a god-like super spell (enlist the help of multiple Ilari to teleport things over).
Thematically, I  like Bluddy's fleeing disaster idea.

Offline TNSe

  • Newbie Mark III
  • *
  • Posts: 34
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #4 on: December 01, 2011, 11:38:47 AM »
Considering everything is procedurally generated, isn't it enough to save the seed and the relative state of objects compared to the seed? (Thus regenerating this if it is lacking).

Kinda like storing the delta from original seed to current state. (Or did I read something wrong)

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,050
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #5 on: December 01, 2011, 11:45:55 AM »
Considering everything is procedurally generated, isn't it enough to save the seed and the relative state of objects compared to the seed? (Thus regenerating this if it is lacking).

Kinda like storing the delta from original seed to current state. (Or did I read something wrong)

That only works if you never upgrade the game at all.  As soon as there's a new version and anything new that could go in that region, the from-seed-only model breaks down.  As I noted above, there's actually more than a bit that we DO reconstruct from seeds only, but it's the only stuff that is safe to do so.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline TNSe

  • Newbie Mark III
  • *
  • Posts: 34
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #6 on: December 01, 2011, 11:57:39 AM »
I see, there are some solutions to that too, but sadly they would at one time or another end up limiting you.

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,050
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #7 on: December 01, 2011, 12:14:14 PM »
I see, there are some solutions to that too, but sadly they would at one time or another end up limiting you.

Yep, or costing more data -- or, worse, dev time -- to store them in order to try to solve them.  Also, I didn't mention the fact that players can add 9or delete) their own custom content in terms of the xml population files and the room map templates and all that.  Just changing those things around changes how the seeding happens in pretty major ways, so if that's not baked in then you'd see massive swings every time it tried to recreate that stuff.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Hearteater

  • Core Member
  • *****
  • Posts: 2,334
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #8 on: December 01, 2011, 12:14:41 PM »
I think you've probably hit the best solution for world size on disk by removing older sections either through in-game cataclysms and/or by just purging detailed data of boring old chunks.

For in memory, is there a reason you need every player to have all the same stuff in memory?  Especially if they are on different continents.  It seems like the ability to affect each other is very very low and so they don't really need to have all that in memory.  They only need to pass up changes that need to happen.  I'm guessing you keep the sections in memory so you can apply those changes, but you don't have to do that immediately.  You can store all the changes made (in RAM and/or disk) and processes them as RAM permits.

For example, say 20 people are playing on different continents, but I only have the RAM to hold 10 continents at once.  As changes come in for continents I don't have in memory, I set them aside (possibly to disk) for later handling.  Periodically the game drops one of the 10 continents in memory (not the one my player is in) and loads one of the other continents with a lot of changes pending and processes those.  Basically this is manual memory paging, and not ideal, but at least it scales on disk space instead of RAM and more importantly the heap limit.

For settlement trading, could you pull the settles out as a separate block of data with a much smaller footprint than the continent so you can always interact with all settlements because they are all in memory?

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,050
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #9 on: December 01, 2011, 12:25:20 PM »
Hearteater: A lot of that gets kind of tricky as the game evolves.  You wind up later with "oh crap, no settlement trading is possible because of the way we're managing RAM" type situations.  Or "sorry, can't migrate NPCs between continents!"  Or whatever else.  Such as, taking a boat ride as discussed on the continents thread would be impossible, it would have to just warp you between continents because trying to stream in all the continents belatedly (only to drop them once you arrive) would make boat rides incredibly painful to initiate.

Plus, if we ever do point to point warps that cross continents.  Or a whole other mess of things.  It winds up being something that impacts the design of the whole game, usually in a negative way, on an ongoing basis.  When instead it really should be a little more freeform, ideally.  RAM management techniques should just kind of fade into the background.  Which works with the ones we currently have, because we keep the broad metadata that we might need in a central place, such that you really don't have to have any of the other region data until you physically go there.  But in terms of making metadata about the metadata, or heavily segmenting the metadata... well, it really makes it a lot less of one world, and more a collection of smaller isolated worlds, if that makes sense.

Which is what we were considering doing, but once I did the math I started thinking that wasn't even needed, essentially.  Plus, players with enormous worlds have already complained some in terms of having trouble just remembering where everything is.  In the early alpha, c4sc4 racked up a world that had something like 12,000 regions in it two months ago.  And he's still playing on that world, but it's just incredibly too vast in a negative way I think.  If you're trying to mentally keep track of 10 continents and what is going on on each of them, I think that's really quite a lot to go with, even in the late game.  Closer to 5 would seem to be an ideal number, and matching what you see with a lot of JRPGs in particular, it seems.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Dizzard

  • Sr. Member Mark II
  • ****
  • Posts: 380
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #10 on: December 01, 2011, 02:27:26 PM »
The central world index stuff contains enough to run the world map, to have NPCs be able to talk about the overlord by name

Does an npc in one continent need to be able to talk about an overlord in another continent though? Npcs are obsessed enough with the overlords without them talking about ones across the sea too. I see your point though....to some degree npcs and the game in general will need to be aware of things happening in different continents.

2. Additionally, as the world gets progressively larger, it is taking up more disk space.  Mainly in terms of the per-region detail data (mostly chunk files).  This isn't a huge problem, as I noted above, but it is annoying.  If you want to periodically back up your world folder, for instance, it's going to take progressively longer and longer to back up the larger your world gets.  And the most annoying part is that most of the data that it's spending so long backing up is old areas you haven't visited in dozens of hours, and probably never will visit again.

Even now at around lv 140 it takes a little bit longer to backup then I'd like. (would be nice to maybe see some kind of progress bar at some stage)

As for the problem, I might be far out with this....but what if each region tile had an age associated with it. A region past a certain age (time since last load) wouldn't be seen as necessary or important by the back up process. So if your world has a major error it doesn't matter if these regions are replaced by fresh ones when you back up your game. (Players could also have the ability to mark regions as special so they would stick around even if you don't go near them for a while) Although applying an age to each region and keeping track of it might be more trouble than it's worth.

And there could be some severe limitations in what we can do cross-continent if we ever felt like implementing trade or something, since we can't do anything with settlements that aren't in RAM because they're on a different continent.

Coming back to solo play, when you think about mission time and how that's something that causes new side missions to pop up and disappear, that's another thing that would be somewhat problematic if older continents were not stored in memory at the time.  They'd get left behind in a major way.

This would be annoying, I'd hate to see things like trade or cross continent relations being excluded because of this. It would be a major shame and make the continents seem more like individual mini worlds.

What I think is that there are some "key locations" that players wind up having longer-term interest in and wanting to maintain, while the rest of the "I was just off doing some scavenging or a minor side mission" kinds of areas can be dumped because the player has already forgotten about their details anyhow (and probably wouldn't recognize them if they returned to them anyway, if it's been long enough).

There are many regions just sitting there that I have no interest in entering because their level is now so far below mine. I think there are some lv 1 areas I haven't entered even. The only exception to low level areas is if they have a resource of some kind (and if a mob heading for my settlement is hanging out there)

Okay, so this is a post-disaster world, right?  But in a lot of respects the world really still is in the midst of a disaster.  In another thread, Keith and I were talking about having disasters that would strike older continents to up their region levels and basically those would purge whatever had been there.

I'm behind this, regions that are so below my level are entirely irrelevant and serve no real purpose other than to have something there as background.

Well... what if we took that a lot further than just meteor strikes or flooding or whatever, and made it so that after a certain amount of time had passed that continents became volatile and started to sink into the sea?  Bam, that's a huge purge of data right there. 

Once you have something like 5-6 continents, the oldest one goes into a crisis and will sink into the sea after some amount of mission time.  What do you do in the interim?  Well, you now have the fun of trying to rescue whatever parts of the continent you care about. 

Maybe you can magic the settlement(s) from there to any of the other continents of your choosing to save them and the citybuilding stuff that was going on with them.

Or maybe it's really more of a loss of the settlement itself, and you're just saving the people, but they bring across their know-how and goods and stuff and are able to quickly rebuild an equivalent-but-different-looking settlement in a new place.

Maybe you can choose a couple of other regions to also magically move to safety of another continent, but at the cost of a unit of mission time for each region you're saving.  So if there was some really iconic overlord keep that you want to retain access to, or you've got a near-infinite cave system that you're down 15 caverns into, you could move those to safety before the rest of the continent goes down into the sea.

Or if you're not sentimental, you just let the whole thing sink into oblivion and all those older NPCs die, too.  It's up to you.

Hmm this interests me and sort of scares me at the same time. For one thing if you're really driving home the whole "survival" aspect the whole need to evacuate npcs from settlements falling into the sea would work really well....but from a civilization point of view it might be in danger of making things look a bit weak and frivolous if not done correctly.

I'd consider myself to be extremely sentimental with things like this. So in some ways it would feel like the game is directly attacking the vision I have amassed over time. I could totally see myself with settlements called "New new new new new Mahbakk" (The capital of my current world :3) You can just see from the ":3" how much I dote on little things like settlements and the people. While other players may be all for the action I do place a lot of value in the people of my civilization and their home. The progression of my civilization and it's people are the main reason I feel motivated to go exploring and defeat the overlords.

On the other hand I can definitely see the entertainment value with all the evacuating npcs and trying to save peoples lives from impending doom. Surviving with your npcs could make those npcs all the more special.
« Last Edit: December 01, 2011, 02:32:20 PM by Dizzard »

Offline c4sc4

  • Sr. Member
  • ****
  • Posts: 253
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #11 on: December 01, 2011, 02:39:00 PM »
What about just deleting data for any regions that are 20+ levels below the current civ level? There really is little need to ever go back to those levels, especially because any tiered items in those levels would be worthless. It could fit in with the cataclysm where those tiles would be destroyed, and on the world map they could have a special tile, like a glowing red cracks or something. That tile would essentially be un-enterable and just allow for ways to cross on the world map. Special things like settlements and overlords could be saved from this and perhaps a 3x3 square of tiles around each settlement so you could fight of rampaging monsters.

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,050
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #12 on: December 01, 2011, 03:17:24 PM »
As for the problem, I might be far out with this....but what if each region tile had an age associated with it. A region past a certain age (time since last load) wouldn't be seen as necessary or important by the back up process. So if your world has a major error it doesn't matter if these regions are replaced by fresh ones when you back up your game. (Players could also have the ability to mark regions as special so they would stick around even if you don't go near them for a while) Although applying an age to each region and keeping track of it might be more trouble than it's worth.

Well, that would help with backup speed but not much beyond that really.

Hmm this interests me and sort of scares me at the same time. For one thing if you're really driving home the whole "survival" aspect the whole need to evacuate npcs from settlements falling into the sea would work really well....but from a civilization point of view it might be in danger of making things look a bit weak and frivolous if not done correctly.

I'd consider myself to be extremely sentimental with things like this. So in some ways it would feel like the game is directly attacking the vision I have amassed over time. I could totally see myself with settlements called "New new new new new Mahbakk" (The capital of my current world :3) You can just see from the ":3" how much I dote on little things like settlements and the people. While other players may be all for the action I do place a lot of value in the people of my civilization and their home. The progression of my civilization and it's people are the main reason I feel motivated to go exploring and defeat the overlords.

On the other hand I can definitely see the entertainment value with all the evacuating npcs and trying to save peoples lives from impending doom. Surviving with your npcs could make those npcs all the more special.

The sentiment of exactly that civilization is part of why I was suggesting just transplanting the entire region.  It's also possible that old regions that "sink into the sea" could actually work a little differently than I was proposing.  They could literally sink into the sea except for the key locations (settlements, maybe a few other things), which stay on the surface and then there's some mud flats that is level-less and that you can't venture into, which takes the place of the actual land that had been there.  It still lets us dump the old regions, and the metadata would be very small, but it wouldn't completely remove the old landmass.

So there are several ways we could go.  I figure that actually transplanting the region onto a new continent with higher-level stuff around it would be the most interesting way to do it because then you could actually keep doing side missions that would help that settlement, so that you could grow it further.  Frankly, maybe a way to do a "settlement swap" so that your best settlement carries forward and then you have to rescue the few strangers from the new settlement that is about to sink into the sea, would be good. ;)
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline FallingStar

  • Jr. Member
  • **
  • Posts: 53
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #13 on: December 01, 2011, 03:31:01 PM »
Wanted to post a few thoughts here.  Ahhh the issues with infinite worlds and finite systems.

First off, a few alternatives that went swimming through my mind.  The first one was what I first thought of when I even heard of continents, before hearing about the cleanup need.  I was thinking that perhaps old continents would get abstracted in some form, rather like in AOE3 or Civ Colonization, where a European city exists, but you don't actually actively play there.  Its there for trade or trophies, but not actually a place that you go and adventure.  Probably only for the "oldest" civilizations (ie 5-6 continents ago), but at least it would keep them in a way, for attached players.

Another thought as I read through was a mutable world system.  The chunks/ regions on old continents would slowly be changed/ transformed/ altered by the saved settlement there.  So you could still go there and check things out, but it would be new regions on the old landmass.  I think the trick would be saving the regions players cared about (like if there was one that players had used the shrink/ enlarge spell to add trophies).  Or the settlements themselves.  I'm not sure if this would retain enough of the flavor of a "legacy" continent to justify the effort, and it wouldn't save as much on the world.dat probably.

If it does go to a more cataclysm based system, I think it would be a bit sad to lose those old cities /settlements /whatnot.  It is a more "well why did I save you and why save these people if the same happens to them" thing that impinges on the infinite world satisfaction.  A player does ultimately want to save the princess, even if we are told she's in another castle a number of times.

Bluddy's twist helps on that, though I'd chip in a few things.  First it seems a bit odd for the Overlord to do something like this, as they're stockpiling and the like, seems like they're more robber barons than trying to wipe out humanity by drowning us.  Also would make them seem a bit more like puppets of some other over-overlord to get them all to do this to various continents.  I guess rewording lores and the like would help.  An alternate twist might be that overlords were maintaining the continent, in their oppressive way, and in killing them the continent now decays.  Kind of blur the line between what is good or evil.

Either way if there is permanent continent loss, and the idea is seeking out a valley without wind . . I think that its important that it actually exists.  Not as something eternally dangled in front of us, but perhaps as an epic sidequest or whatever after killing an overlord, being able to seek out and claim that trophy continent.  Perhaps you could garden the regions to particular chunk types (mentioned that idea before but seems to work here, and would help refresh old regions/chunks on an otherwise unchanging place without feeling like you were destroying them for the player).  The player might also be able to move the populations or entire settlements over, etc.  In this case, even if there was an area that was permanent, it is at least finite, so is less of a long term issue.  And it has a player satisfaction of grabbing that long term goal, even if there's still things they can change to it /move to it/ help or hinder other parts of the world to grow your own chunk.  But in essence transferring over the emotional attachment to a limited space that can stay eternally, and grabbing the chunks of old sentiment to save as well.

Anyways, so a boatload of brainstorming thoughts, hope some help somewhere.

Offline TNSe

  • Newbie Mark III
  • *
  • Posts: 34
Re: Brainstorming: Curtailing Infinite World Folder Size Growth
« Reply #14 on: December 01, 2011, 03:32:30 PM »
Mudsquares, maybe settlements after a while when civ level becomes significantly higher, simply take control of these squares in a way such that everything hostile in there is removed, and things that can be gathered is gathered.

If anything, these generic tiles have less information tied to them, and will improve compression.