Alright, how's about a combination of realtime and turn-based, sprinkled with some ideas from what I understand the existing implementation used. This idea is assuming a turn per player.
As I understand it, with the original implementation it was possible for two players to be off on their own and the game would behave as if they were playing solo? If so, this is great! I don't know exactly how you achieved this, but it seems nice to me. The problems arose when playing with two players in the same area.
OK, so let's say that whenever the players are not in their own areas, together, and also not engaged in combat the game would switch to realtime (or free move) -- allowing painless traversal of corridors. Let's then say that whenever a player is engaged in combat (with each player not in their own zone) the game switches back to turn-based mode allowing both players to tactically think about their actions. These players are now in the combat zone. Since both players are in this combat zone (close together and under attack) each player would get a turn. There should be minimal problems with players just chilling around -- why would you when under fire?
If one of the players then decided to retreat, eventually that cowardly player would exit the combat zone (joint turn-based) and would eventually enter their own zone again leaving each player alone as if they were playing solo. If all the enemies are destroyed the game would then enter free move mode again.
To key to this solution is the idea of a combat zone. The players would be notified when they are in a combat zone together and possibly would even be notified of the zone boundary via an overlay. When in this combat zone both players should realize "right, we're in this together now, either man up and let's work together or flee to your own zone".
Actually, that's about it for the basic idea. Feel free to tear it to shreds. I think together we can solve this.