Shouldn't this be in the AI War 2 section? It is related and I am fairly sure that that section gets more attention than the off topic section, so if it was moved there more people might see it and come up with ideas.
It's a programming question rather than an AIW2 question, and in general the folks who answer programming questions look here
If I were looking for help answering "what should we do here?" I'd ask there. But in this case the goal is clear and I'm asking a nuts-and-bolts "so how is this actually done?" question.
But it failed your "Exists" test, when I tried it.
I don't particularly mind that as long as it can't read/write disk or network. Further tightening can probably be achieved in another way. Thanks for linking that, it's got very interesting info.
In Unity I've got it creating the sandboxed domain,
and I've got the sandboxed domain creating the untrusted objects from the external dll containing the mapgen logic,
and when the game calls methods on those objects it shows a "(wrapper xdomain-invoke)" in the call stack...
but it's still not actually enforcing the permissions.
Specifically, if I run this:
string path = "C:\\vcprojs\\test.txt";
string text = "";
bool result = File.Exists( path );
text += "Result1:" + result;
File.WriteAllText( path, "IReallyShouldNotBeAllowedToDoThis" );
result = File.Exists( path );
File.Delete( path );
text += "\nResult2:" + result;
In .NET I get an exception:
An exception of type 'System.Security.SecurityException' occurred in mscorlib.dll but was not handled in user code
Additional information: Request for the permission of type 'System.Security.Permissions.FileIOPermission, mscorlib, Version=126.96.36.199, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.
But in Unity I get no exception, and this returned:
FWIW, the top of the callstack just before that return is:
at (wrapper xdomain-invoke) MyMapgen:TestMethod ()
I suppose the next step is to try to create a barebones unity project demonstrating this, and seeing if it can be made to better behave there.
On the other hand, one of the things I discovered when running the real mapgen code this way (rather than the test code that's intentionally doing errant file IO) is that it throws runtime errors when trying to call a method with a parameter that doesn't have the Serializable attribute. Looking into it further it looks like it would have to do a binary serialization on every paramater every time the call is made... uh, that's not gonna work for stuff like each AI ship's targeting comparisons
I'm guessing that particular problem can be sidestepped by making sure the necessary info for the method is stored somewhere it can get to it, and just calling the methods with no actual parameters (or only primitive-type parameters or others where it's not bad performance to constantly serialize them), but this does raise the question of whether crossing the app-domain boundary potentially thousands of times per second is going to do excessive violence to performance.
Then there are the other considerations raised by that project you linked, like the garbage collector not really knowing what to do with cross-domain stuff and basically guessing, etc.
"Just let people install DLL mods (as opposed to XML mods) manually and do their own vetting" is looking like a better and better option