In a similar direction of Baleyg's thought, not all stacked bonuses need to be metrovania-related.
Say each time you level up your civilization four levels with the same character, he gains a random trait from a fairly large pool of traits. These traits provide a benefit when the character is left in your settlement, but not otherwise. Further, you can only "equip" one trait at a time, although you can change which trait is equipped on a character in your settlement at any time to any they have previously acquired. If you acquire the same trait again on a character, that trait ranks up providing greater benefits.
Example Traits:
Horder - Gives you a small amount of extra ammo of most types when you return to your settlement.
X Freak - Gives you extra X-type ammo when you return to your settlement (more than the Horder, but only of one type).
Scavenger - You may select one item from a list of random equipment once per mission.
Healer - Gives you additional health tank pieces once per mission.
With a largish pool of traits, say 20, you'd rarely get rank 2 traits, much less rank 3. Those characters would be very valuable outside the metrovania mode and it would effectively create a retirement system which would limit power creep. But it would be entirely voluntary, and you could bring characters out of retirement if you needed to do something their combat skills were really suited to.
