Author Topic: Sandboxing code in C# (for AIW2 dll modding)  (Read 818 times)

Offline keith.lamothe

  • Arcen Games Staff
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 19,029
Sandboxing code in C# (for AIW2 dll modding)
« on: December 24, 2016, 04:37:05 PM »
(Yes, this is the sort of thing I do during downtime on family trips)

Ok, I've been banging my head against this wall, and I can't get it to stay up... normally that's not my problem, but in this particular instance I want the wall to stay up. I want a way of running C# code (from an external dll, but anything really) and preventing it from doing anything with system side-effects (notably, File IO and Network IO).

Basically I've been googlilng and stackexchanging on this for quite a while, and I can get the permissions set, but they just don't do anything. Here's my boiled-all-the-way-down example in a standard .NET 4.5 console app:

Code: [Select]
using System;
using System.Security.Permissions;

public class Program
{
    public static void Main( string[] args )
    {
        Console.WriteLine( "Welcome!" );

        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( PermissionState.None ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows\\" ) );

        bool result = System.IO.Directory.Exists( "C:\\Windows" );
        Console.WriteLine( "Result:" + result );

        Console.ReadKey();
    }
}
The expected output includes some kind of error, but I get "Welcome!" followed by "Result:true" every time, with no errors. The large number of AddPermission calls is just me trying to cover every base.

Anyway, if any of you know how this is actually supposed to be done, it will be extremely helpful in securing dll mods in AIW2 so that folks can actually run mods from other people without worrying about it deleting chunks of their disk or sending data off to random servers, etc. If I can't get something along these lines working we'll probably have to make it so dll mods can only be installed manually, and only xml mods will be distributed through any reasonably usable channel.

I've attached a zip of the whole vs-2015 solution that my example above comes from. It's literally just that code in one file, plus the csproj file and the sln file. If you have vs-2015 installed (we use the free community one) it should just be a matter of unzipping it and double-clicking the sln file, and running it in vs. If you trust the code, anyhow ;)

Note: we target 2.0 or 3.5 for Unity games, but this is separate from that; they changed how they did security policy between 3.5 and 4.5, so that's an extra wrinkle, but I couldn't get that to work there either. I'm just trying to find some place, some way, where this can be done. Then I can try to work back towards our actual use case.

Edit: I do know that normally this is done by creating a separate AppDomain and setting it to have the more restricted permissions; I've tried that and had the same results, so I figured I'd start with the more basic case: can I lock a door and then walk into it without it swinging open on contact?
« Last Edit: December 24, 2016, 04:38:37 PM by keith.lamothe »
Have ideas or bug reports for one of our games? Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline keith.lamothe

  • Arcen Games Staff
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 19,029
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #1 on: December 24, 2016, 04:54:22 PM »
I also just tried with this variant, to test write access:

Code: [Select]
using System;
using System.Security.Permissions;

public class Program
{
    public static void Main( string[] args )
    {
        Console.WriteLine( "Welcome!" );
        Console.WriteLine( "ReadOnly:" + AppDomain.CurrentDomain.PermissionSet.IsReadOnly );

        AppDomain.CurrentDomain.PermissionSet.RemovePermission( typeof( FileIOPermission ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( PermissionState.None ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows\\" ) );

        string path = "C:\\vcprojs\\test.txt";

        System.IO.File.WriteAllText( path, "IReallyShouldNotBeAllowedToDoThis" );

        bool result = System.IO.File.Exists( path );
        Console.WriteLine( "Result:" + result );

        System.IO.File.Delete( path );

        Console.ReadKey();
    }
}
Have ideas or bug reports for one of our games? Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline zespri

  • Hero Member Mark III
  • *****
  • Posts: 1,103
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #2 on: December 24, 2016, 05:30:11 PM »
I'm sorry I cannot look at it in more depth until I return from my xmas trip (literally I'm almost out of the door leaving, and I'm not taking a laptop with me), but what I would try is to create a separate AppDomain, not use the standard one. It could be that what I you trying to do is not possible with the start-up appdomain at all.

I may be wrong of course but this is the best I can offer in the 3 minutes I have right now.

Google for .net sandbox, there are quite a few hits like https://msdn.microsoft.com/en-us/library/bb763046(v=vs.110).aspx

« Last Edit: December 24, 2016, 05:37:11 PM by zespri »

Offline Draco18s

  • Resident Velociraptor
  • Core Member Mark IV
  • *****
  • Posts: 3,801
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #3 on: December 24, 2016, 07:53:33 PM »
Unfortunately I'm not going to have any ideas either. :\

Offline keith.lamothe

  • Arcen Games Staff
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 19,029
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #4 on: December 24, 2016, 10:04:15 PM »
what I would try is to create a separate AppDomain, not use the standard one. It could be that what I you trying to do is not possible with the start-up appdomain at all.
Yea, I tried that in earlier testing and it lets it do whatever.

Quote
Google for .net sandbox, there are quite a few hits like https://msdn.microsoft.com/en-us/library/bb763046(v=vs.110).aspx
Yea, I've seen that one several times, and a couple dozen other msdn/stackoverflow/etc articles.

Anyway, thanks for the responses :)

I'm sure we'll figure something out, this is just one of the most intransigent and arcane things I've run into in nearly 7 years of working with AIW. And that's saying something.

... maybe something in there wants unrestricted permissions for modded code...
Have ideas or bug reports for one of our games? Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline Dominus Arbitrationis

  • Arcen Games Contractor
  • Administrator
  • Sr. Member Mark III
  • *****
  • Posts: 428
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #5 on: December 24, 2016, 11:09:25 PM »
I think instead of devoting time and energy into making a way for mods to be loaded, you just work on making the AI fully sentient, and let it handle the rest.

What could possibly go wrong?

Are there any cases where we want the mod to alter files at all? Or would we want it to just say "Don't run this, load this XML file"?

Have you tried saying "You have permission to do edits here, but not there"?
Come help out at the Wiki!

Have ideas or bug reports for one of Arcen's games or any part of the site? Use  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games and site better!

Offline Draco18s

  • Resident Velociraptor
  • Core Member Mark IV
  • *****
  • Posts: 3,801
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #6 on: December 25, 2016, 01:20:04 AM »
Are there any cases where we want the mod to alter files at all? Or would we want it to just say "Don't run this, load this XML file"?

Have you tried saying "You have permission to do edits here, but not there"?

I think the problem is that even in denying permission, access is still granted. Not the other way around. I.e. Even if you say "here and not there" it can still access there.

Offline zespri

  • Hero Member Mark III
  • *****
  • Posts: 1,103
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #7 on: December 26, 2016, 11:24:01 PM »
I'm sure we'll figure something out, this is just one of the most intransigent and arcane things I've run into in nearly 7 years of working with AIW. And that's saying something.

I'm quite sure it is possible. Let me get back to you in the next few days - I'm back from the trip now.

Offline zespri

  • Hero Member Mark III
  • *****
  • Posts: 1,103
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #8 on: December 27, 2016, 12:47:33 AM »
Keith,

I ran the example from https://msdn.microsoft.com/en-us/library/bb763046(v=vs.110).aspx

I substituted
           
Code: [Select]
File.ReadAllText("C:\\Temp\\file.txt");with
           
Code: [Select]
bool result = System.IO.Directory.Exists("C:\\Windows");
            Console.WriteLine("Result:" + result);

I received:
           
Code: [Select]
Result:False
Are you getting a different result?

PS. This could be down to mono implementation. As far as I know, Unity, even on windows hosts entire mono runtime withing itself. If mono implementation somehow differs here, result could be different.
« Last Edit: December 27, 2016, 01:02:38 AM by zespri »

Offline keith.lamothe

  • Arcen Games Staff
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 19,029
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #9 on: December 27, 2016, 01:39:27 PM »
Good idea on trying that example again. I'd tried it before and just kept getting errors (before the bit with running code in the other domain) but in this simpler test case it's at least running.

What I've found is that it correctly prevents ReadAllBytes and WriteAllBytes, but allows Exists. That's probably just a matter of setting the permissions correctly.

And then a matter of convincing it to work in Unity ;)

Quote
PS. This could be down to mono implementation. As far as I know, Unity, even on windows hosts entire mono runtime withing itself. If mono implementation somehow differs here, result could be different.
The example I posted here has nothing to do with mono or Unity, it's just a straight-up console application written in and run in vs-2015. Obviously I also have to deal with what Unity/mono does, but I was trying to find a way of making it work "in its native habitat" first. Seems we've got that, so we'll see if it carries over well enough.

Thanks for the help :)
Have ideas or bug reports for one of our games? Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline WolfWhiteFire

  • Full Member
  • ***
  • Posts: 144
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #10 on: December 27, 2016, 03:00:32 PM »
Shouldn't this be in the AI War 2 section? It is related and I am fairly sure that that section gets more attention than the off topic section, so if it was moved there more people might see it and come up with ideas.
That crazy non-hardcore player who decided that when AI War 2 comes out focusing on hunting the Nemesis as much as possible regardless of risk will be a good idea.

Offline zespri

  • Hero Member Mark III
  • *****
  • Posts: 1,103
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #11 on: December 27, 2016, 03:05:07 PM »
What I've found is that it correctly prevents ReadAllBytes and WriteAllBytes, but allows Exists. That's probably just a matter of setting the permissions correctly.

Yes. You do not have to have a permission on a folder for exists to work, you need a permission on the parent.
Same as you, in many attempts I saw Exists working but actually accessing the content of a file would fail as expected. Same as you I'd expect Exists to fail in these circumstances as well, because to me it did not seem that the domain had permission even on the parent. So, yes it definitely looks finicky.

If you get an implementation to work as we would expect, I encourage you to post the sandbox system on github (you should not end up with more than a handful of classes), it could help other fellow developers in future. Given that you are on the clock, I volunteer to clean it up for you to make it publishable. if that's the problem.

Threre is an implementation here

https://github.com/fadden/DynamicScriptSandbox

But it failed your "Exists" test, when I tried it.

Offline keith.lamothe

  • Arcen Games Staff
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 19,029
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #12 on: December 27, 2016, 04:08:25 PM »
Shouldn't this be in the AI War 2 section? It is related and I am fairly sure that that section gets more attention than the off topic section, so if it was moved there more people might see it and come up with ideas.
It's a programming question rather than an AIW2 question, and in general the folks who answer programming questions look here :) If I were looking for help answering "what should we do here?" I'd ask there. But in this case the goal is clear and I'm asking a nuts-and-bolts "so how is this actually done?" question.

But it failed your "Exists" test, when I tried it.
I don't particularly mind that as long as it can't read/write disk or network. Further tightening can probably be achieved in another way. Thanks for linking that, it's got very interesting info.

In Unity I've got it creating the sandboxed domain,
and I've got the sandboxed domain creating the untrusted objects from the external dll containing the mapgen logic,
and when the game calls methods on those objects it shows a "(wrapper xdomain-invoke)" in the call stack...
but it's still not actually enforcing the permissions.

Specifically, if I run this:

Code: [Select]
        string path = "C:\\vcprojs\\test.txt";

        string text = "";

        bool result = File.Exists( path );

        text += "Result1:" + result;

        File.WriteAllText( path, "IReallyShouldNotBeAllowedToDoThis" );

        result = File.Exists( path );

        File.Delete( path );

        text += "\nResult2:" + result;

        return text;

In .NET I get an exception:

Code: [Select]
An exception of type 'System.Security.SecurityException' occurred in mscorlib.dll but was not handled in user code

Additional information: Request for the permission of type 'System.Security.Permissions.FileIOPermission, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.

But in Unity I get no exception, and this returned:

Code: [Select]
Result1:False
Result2:True

FWIW, the top of the callstack just before that return is:

at (wrapper xdomain-invoke) MyMapgen:TestMethod ()


I suppose the next step is to try to create a barebones unity project demonstrating this, and seeing if it can be made to better behave there.

On the other hand, one of the things I discovered when running the real mapgen code this way (rather than the test code that's intentionally doing errant file IO) is that it throws runtime errors when trying to call a method with a parameter that doesn't have the Serializable attribute. Looking into it further it looks like it would have to do a binary serialization on every paramater every time the call is made... uh, that's not gonna work for stuff like each AI ship's targeting comparisons :)

I'm guessing that particular problem can be sidestepped by making sure the necessary info for the method is stored somewhere it can get to it, and just calling the methods with no actual parameters (or only primitive-type parameters or others where it's not bad performance to constantly serialize them), but this does raise the question of whether crossing the app-domain boundary potentially thousands of times per second is going to do excessive violence to performance.

Then there are the other considerations raised by that project you linked, like the garbage collector not really knowing what to do with cross-domain stuff and basically guessing, etc.

"Just let people install DLL mods (as opposed to XML mods) manually and do their own vetting" is looking like a better and better option ;)
Have ideas or bug reports for one of our games? Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline zespri

  • Hero Member Mark III
  • *****
  • Posts: 1,103
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #13 on: December 27, 2016, 10:57:55 PM »
Do you insist on c# scripting? Sandboxing something like lua or javascript might be easier. Or even python.

Can you give me an idea (if there is a pre-kickstarter document feel free to point me to it) as to what do you want to achieve. In particular, why custom (mod) code would need to be called that frequently?

What are the extension points for your mod system?

In practice it's very rare that someone would pwn you via a game mode. But if I were a game developer (which I'm not) I would be very hesitant to let people do their own vetting. Not many people have enough knowledge to do this. On the off-chance someone does write a malicious mod PR fallout is potentially disastrous. But then again, you might have more experience in this than me so just voicing concern.

I know, if you do not know lua or javascript, integrating can be a PITA, and you need not just to the integration, part of your game will be in that language, and if you are not comfortable with the language, it can slow you down considerably. Again, based on my own experience, I'd probably go with lua. Not because this is the best thing since sliced bread, but just because it's been done many times by other people, so it will be easy to get help. Then again, I programmed WoW mods with lua and I read the lus spec several times from start to end and I actually had a look at the source code for the lua VM, so to me it may sound much less daunting than to you.

To me lua feels quite old and not very cool, but it would probably be the quickest to get working.
Possible routes are NLua, KopiLua, MoonSharp
« Last Edit: December 28, 2016, 01:07:59 AM by zespri »

Offline Draco18s

  • Resident Velociraptor
  • Core Member Mark IV
  • *****
  • Posts: 3,801
Re: Sandboxing code in C# (for AIW2 dll modding)
« Reply #14 on: December 28, 2016, 12:59:51 AM »
Do you insist on c# scripting? Sandboxing something like lua or javascript might be easier. Or even python.

What are the extension points for your mod system?

It's all about API hooks.  Letting mods define new functions for AI tasks or map generation or AI personality traits, or...