Arcen Games

Other => Off Topic => Topic started by: keith.lamothe on December 24, 2016, 04:37:05 PM

Title: Sandboxing code in C# (for AIW2 dll modding)
Post by: keith.lamothe on December 24, 2016, 04:37:05 PM
(Yes, this is the sort of thing I do during downtime on family trips)

Ok, I've been banging my head against this wall, and I can't get it to stay up... normally that's not my problem, but in this particular instance I want the wall to stay up. I want a way of running C# code (from an external dll, but anything really) and preventing it from doing anything with system side-effects (notably, File IO and Network IO).

Basically I've been googlilng and stackexchanging on this for quite a while, and I can get the permissions set, but they just don't do anything. Here's my boiled-all-the-way-down example in a standard .NET 4.5 console app:

Code: [Select]
using System;
using System.Security.Permissions;

public class Program
{
    public static void Main( string[] args )
    {
        Console.WriteLine( "Welcome!" );

        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( PermissionState.None ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows\\" ) );

        bool result = System.IO.Directory.Exists( "C:\\Windows" );
        Console.WriteLine( "Result:" + result );

        Console.ReadKey();
    }
}
The expected output includes some kind of error, but I get "Welcome!" followed by "Result:true" every time, with no errors. The large number of AddPermission calls is just me trying to cover every base.

Anyway, if any of you know how this is actually supposed to be done, it will be extremely helpful in securing dll mods in AIW2 so that folks can actually run mods from other people without worrying about it deleting chunks of their disk or sending data off to random servers, etc. If I can't get something along these lines working we'll probably have to make it so dll mods can only be installed manually, and only xml mods will be distributed through any reasonably usable channel.

I've attached a zip of the whole vs-2015 solution that my example above comes from. It's literally just that code in one file, plus the csproj file and the sln file. If you have vs-2015 installed (we use the free community one) it should just be a matter of unzipping it and double-clicking the sln file, and running it in vs. If you trust the code, anyhow ;)

Note: we target 2.0 or 3.5 for Unity games, but this is separate from that; they changed how they did security policy between 3.5 and 4.5, so that's an extra wrinkle, but I couldn't get that to work there either. I'm just trying to find some place, some way, where this can be done. Then I can try to work back towards our actual use case.

Edit: I do know that normally this is done by creating a separate AppDomain and setting it to have the more restricted permissions; I've tried that and had the same results, so I figured I'd start with the more basic case: can I lock a door and then walk into it without it swinging open on contact?
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: keith.lamothe on December 24, 2016, 04:54:22 PM
I also just tried with this variant, to test write access:

Code: [Select]
using System;
using System.Security.Permissions;

public class Program
{
    public static void Main( string[] args )
    {
        Console.WriteLine( "Welcome!" );
        Console.WriteLine( "ReadOnly:" + AppDomain.CurrentDomain.PermissionSet.IsReadOnly );

        AppDomain.CurrentDomain.PermissionSet.RemovePermission( typeof( FileIOPermission ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( PermissionState.None ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows" ) );
        AppDomain.CurrentDomain.PermissionSet.AddPermission( new FileIOPermission( FileIOPermissionAccess.NoAccess, "C:\\Windows\\" ) );

        string path = "C:\\vcprojs\\test.txt";

        System.IO.File.WriteAllText( path, "IReallyShouldNotBeAllowedToDoThis" );

        bool result = System.IO.File.Exists( path );
        Console.WriteLine( "Result:" + result );

        System.IO.File.Delete( path );

        Console.ReadKey();
    }
}
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: zespri on December 24, 2016, 05:30:11 PM
I'm sorry I cannot look at it in more depth until I return from my xmas trip (literally I'm almost out of the door leaving, and I'm not taking a laptop with me), but what I would try is to create a separate AppDomain, not use the standard one. It could be that what I you trying to do is not possible with the start-up appdomain at all.

I may be wrong of course but this is the best I can offer in the 3 minutes I have right now.

Google for .net sandbox, there are quite a few hits like https://msdn.microsoft.com/en-us/library/bb763046(v=vs.110).aspx

Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Draco18s on December 24, 2016, 07:53:33 PM
Unfortunately I'm not going to have any ideas either. :\
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: keith.lamothe on December 24, 2016, 10:04:15 PM
what I would try is to create a separate AppDomain, not use the standard one. It could be that what I you trying to do is not possible with the start-up appdomain at all.
Yea, I tried that in earlier testing and it lets it do whatever.

Quote
Google for .net sandbox, there are quite a few hits like https://msdn.microsoft.com/en-us/library/bb763046(v=vs.110).aspx
Yea, I've seen that one several times, and a couple dozen other msdn/stackoverflow/etc articles.

Anyway, thanks for the responses :)

I'm sure we'll figure something out, this is just one of the most intransigent and arcane things I've run into in nearly 7 years of working with AIW. And that's saying something.

... maybe something in there wants unrestricted permissions for modded code...
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Dominus Arbitrationis on December 24, 2016, 11:09:25 PM
I think instead of devoting time and energy into making a way for mods to be loaded, you just work on making the AI fully sentient, and let it handle the rest.

What could possibly go wrong?

Are there any cases where we want the mod to alter files at all? Or would we want it to just say "Don't run this, load this XML file"?

Have you tried saying "You have permission to do edits here, but not there"?
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Draco18s on December 25, 2016, 01:20:04 AM
Are there any cases where we want the mod to alter files at all? Or would we want it to just say "Don't run this, load this XML file"?

Have you tried saying "You have permission to do edits here, but not there"?

I think the problem is that even in denying permission, access is still granted. Not the other way around. I.e. Even if you say "here and not there" it can still access there.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: zespri on December 26, 2016, 11:24:01 PM
I'm sure we'll figure something out, this is just one of the most intransigent and arcane things I've run into in nearly 7 years of working with AIW. And that's saying something.

I'm quite sure it is possible. Let me get back to you in the next few days - I'm back from the trip now.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: zespri on December 27, 2016, 12:47:33 AM
Keith,

I ran the example from https://msdn.microsoft.com/en-us/library/bb763046(v=vs.110).aspx

I substituted
           
Code: [Select]
File.ReadAllText("C:\\Temp\\file.txt");with
           
Code: [Select]
bool result = System.IO.Directory.Exists("C:\\Windows");
            Console.WriteLine("Result:" + result);

I received:
           
Code: [Select]
Result:False
Are you getting a different result?

PS. This could be down to mono implementation. As far as I know, Unity, even on windows hosts entire mono runtime withing itself. If mono implementation somehow differs here, result could be different.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: keith.lamothe on December 27, 2016, 01:39:27 PM
Good idea on trying that example again. I'd tried it before and just kept getting errors (before the bit with running code in the other domain) but in this simpler test case it's at least running.

What I've found is that it correctly prevents ReadAllBytes and WriteAllBytes, but allows Exists. That's probably just a matter of setting the permissions correctly.

And then a matter of convincing it to work in Unity ;)

Quote
PS. This could be down to mono implementation. As far as I know, Unity, even on windows hosts entire mono runtime withing itself. If mono implementation somehow differs here, result could be different.
The example I posted here has nothing to do with mono or Unity, it's just a straight-up console application written in and run in vs-2015. Obviously I also have to deal with what Unity/mono does, but I was trying to find a way of making it work "in its native habitat" first. Seems we've got that, so we'll see if it carries over well enough.

Thanks for the help :)
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: WolfWhiteFire on December 27, 2016, 03:00:32 PM
Shouldn't this be in the AI War 2 section? It is related and I am fairly sure that that section gets more attention than the off topic section, so if it was moved there more people might see it and come up with ideas.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: zespri on December 27, 2016, 03:05:07 PM
What I've found is that it correctly prevents ReadAllBytes and WriteAllBytes, but allows Exists. That's probably just a matter of setting the permissions correctly.

Yes. You do not have to have a permission on a folder for exists to work, you need a permission on the parent.
Same as you, in many attempts I saw Exists working but actually accessing the content of a file would fail as expected. Same as you I'd expect Exists to fail in these circumstances as well, because to me it did not seem that the domain had permission even on the parent. So, yes it definitely looks finicky.

If you get an implementation to work as we would expect, I encourage you to post the sandbox system on github (you should not end up with more than a handful of classes), it could help other fellow developers in future. Given that you are on the clock, I volunteer to clean it up for you to make it publishable. if that's the problem.

Threre is an implementation here

https://github.com/fadden/DynamicScriptSandbox

But it failed your "Exists" test, when I tried it.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: keith.lamothe on December 27, 2016, 04:08:25 PM
Shouldn't this be in the AI War 2 section? It is related and I am fairly sure that that section gets more attention than the off topic section, so if it was moved there more people might see it and come up with ideas.
It's a programming question rather than an AIW2 question, and in general the folks who answer programming questions look here :) If I were looking for help answering "what should we do here?" I'd ask there. But in this case the goal is clear and I'm asking a nuts-and-bolts "so how is this actually done?" question.

But it failed your "Exists" test, when I tried it.
I don't particularly mind that as long as it can't read/write disk or network. Further tightening can probably be achieved in another way. Thanks for linking that, it's got very interesting info.

In Unity I've got it creating the sandboxed domain,
and I've got the sandboxed domain creating the untrusted objects from the external dll containing the mapgen logic,
and when the game calls methods on those objects it shows a "(wrapper xdomain-invoke)" in the call stack...
but it's still not actually enforcing the permissions.

Specifically, if I run this:

Code: [Select]
        string path = "C:\\vcprojs\\test.txt";

        string text = "";

        bool result = File.Exists( path );

        text += "Result1:" + result;

        File.WriteAllText( path, "IReallyShouldNotBeAllowedToDoThis" );

        result = File.Exists( path );

        File.Delete( path );

        text += "\nResult2:" + result;

        return text;

In .NET I get an exception:

Code: [Select]
An exception of type 'System.Security.SecurityException' occurred in mscorlib.dll but was not handled in user code

Additional information: Request for the permission of type 'System.Security.Permissions.FileIOPermission, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.

But in Unity I get no exception, and this returned:

Code: [Select]
Result1:False
Result2:True

FWIW, the top of the callstack just before that return is:

at (wrapper xdomain-invoke) MyMapgen:TestMethod ()


I suppose the next step is to try to create a barebones unity project demonstrating this, and seeing if it can be made to better behave there.

On the other hand, one of the things I discovered when running the real mapgen code this way (rather than the test code that's intentionally doing errant file IO) is that it throws runtime errors when trying to call a method with a parameter that doesn't have the Serializable attribute. Looking into it further it looks like it would have to do a binary serialization on every paramater every time the call is made... uh, that's not gonna work for stuff like each AI ship's targeting comparisons :)

I'm guessing that particular problem can be sidestepped by making sure the necessary info for the method is stored somewhere it can get to it, and just calling the methods with no actual parameters (or only primitive-type parameters or others where it's not bad performance to constantly serialize them), but this does raise the question of whether crossing the app-domain boundary potentially thousands of times per second is going to do excessive violence to performance.

Then there are the other considerations raised by that project you linked, like the garbage collector not really knowing what to do with cross-domain stuff and basically guessing, etc.

"Just let people install DLL mods (as opposed to XML mods) manually and do their own vetting" is looking like a better and better option ;)
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: zespri on December 27, 2016, 10:57:55 PM
Do you insist on c# scripting? Sandboxing something like lua or javascript might be easier. Or even python.

Can you give me an idea (if there is a pre-kickstarter document feel free to point me to it) as to what do you want to achieve. In particular, why custom (mod) code would need to be called that frequently?

What are the extension points for your mod system?

In practice it's very rare that someone would pwn you via a game mode. But if I were a game developer (which I'm not) I would be very hesitant to let people do their own vetting. Not many people have enough knowledge to do this. On the off-chance someone does write a malicious mod PR fallout is potentially disastrous. But then again, you might have more experience in this than me so just voicing concern.

I know, if you do not know lua or javascript, integrating can be a PITA, and you need not just to the integration, part of your game will be in that language, and if you are not comfortable with the language, it can slow you down considerably. Again, based on my own experience, I'd probably go with lua. Not because this is the best thing since sliced bread, but just because it's been done many times by other people, so it will be easy to get help. Then again, I programmed WoW mods with lua and I read the lus spec several times from start to end and I actually had a look at the source code for the lua VM, so to me it may sound much less daunting than to you.

To me lua feels quite old and not very cool, but it would probably be the quickest to get working.
Possible routes are NLua, KopiLua, MoonSharp
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Draco18s on December 28, 2016, 12:59:51 AM
Do you insist on c# scripting? Sandboxing something like lua or javascript might be easier. Or even python.

What are the extension points for your mod system?

It's all about API hooks.  Letting mods define new functions for AI tasks or map generation or AI personality traits, or...
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: keith.lamothe on December 28, 2016, 04:51:48 AM
Do you insist on c# scripting? Sandboxing something like lua or javascript might be easier. Or even python.
My core goals:
1) Be able to write any part of the game rules in "external" code.
2) Have modders be able to see how I wrote those rules and be able to rewrite them with the same tools I used.
3) Not increase cpu/ram cost of my implementation more than, say, 10-15%.
4) Not increase development time of my implementation of those rules more than, say 20-25%.
5) Not increase development time of 1.0 as a whole more than 5%.

I considered lua for a while, and stuff like that, but those fail 3, 4, and 5 pretty hard. C# passes them all just fine, so while I care about security and ease of installation I won't let them overrule everything else.


Quote
Can you give me an idea (if there is a pre-kickstarter document feel free to point me to it) as to what do you want to achieve. In particular, why custom (mod) code would need to be called that frequently?
The clearest case is the function where a ship takes two eligible targets and decides which one it prefers.

Or, more broadly, the logic which takes a ship, builds a list of eligible targets for it, and then sorts that list by preference.

These things happen very often.

Less frequent things would be like mapgen. Or the function which takes the entire ai "side" and has each ship decide where to go. That can take a few seconds and happens on a separate thread.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: zespri on December 28, 2016, 03:31:54 PM
The clearest case is the function where a ship takes two eligible targets and decides which one it prefers.

How does it do it? Presumably in the simplest keys, each of the targets have properties, and it will compare a property of first target with a property of a second target.
Does this sound right? I'd like to get down to something measurable here.  (Point 3).

Understood your point of being uncomfortable with lua/js programming (poitns 4/5).

Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Cyborg on December 28, 2016, 06:15:44 PM
C# mods is fine by me. The Linux folks have some extra hurdles, though. Honestly, the very fact we are getting modding is going to completely make this game bananas. It will be completely awesome. If steam works is not supported, I think we should get something going on Nexus mods to ease some of the installation burden. Or some kind of mod pack.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Aklyon on December 28, 2016, 08:32:32 PM
We wouldn't need a mod pack, just a functional mod manager or launcher.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: eRe4s3r on December 29, 2016, 04:54:58 AM
C# mods is fine by me. The Linux folks have some extra hurdles, though. Honestly, the very fact we are getting modding is going to completely make this game bananas. It will be completely awesome. If steam works is not supported, I think we should get something going on Nexus mods to ease some of the installation burden. Or some kind of mod pack.

Assuming we can replace graphic and sound assets a lot easier than we can replace game core functions Nexus would be acceptable, but dll mods are gonna be a tiny niche (and a security problem) no matter how you look at it. I wouldn't download random DLL mods from this forum nor from the nexus tbh... and sandboxing externally loaded dlls is not an easy thing to do right. And can very easily make otherwise possible mods impossible (like persistent progression - saving stuff into mod directory xml or what)

Also with the way I understand this works, we would require someone to merge mods... if we had more than 1 ;p Because I know from KSP what a huge performance penalty even -1- slightly complicated external dll can be.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Draco18s on December 29, 2016, 09:40:50 AM
And can very easily make otherwise possible mods impossible (like persistent progression - saving stuff into mod directory xml or what)

The sandbox would still have access to the game's own directories.  That's part of what this thread is about: preventing access to system files.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Cyborg on December 29, 2016, 09:58:12 AM
C# mods is fine by me. The Linux folks have some extra hurdles, though. Honestly, the very fact we are getting modding is going to completely make this game bananas. It will be completely awesome. If steam works is not supported, I think we should get something going on Nexus mods to ease some of the installation burden. Or some kind of mod pack.

Assuming we can replace graphic and sound assets a lot easier than we can replace game core functions Nexus would be acceptable, but dll mods are gonna be a tiny niche (and a security problem) no matter how you look at it. I wouldn't download random DLL mods from this forum nor from the nexus tbh... and sandboxing externally loaded dlls is not an easy thing to do right. And can very easily make otherwise possible mods impossible (like persistent progression - saving stuff into mod directory xml or what)

Also with the way I understand this works, we would require someone to merge mods... if we had more than 1 ;p Because I know from KSP what a huge performance penalty even -1- slightly complicated external dll can be.

For what it's worth, you could trust my dll's. But yes, I would agree with you. I think that the code needs to be provided and able to be compiled by the user.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: keith.lamothe on December 29, 2016, 10:16:10 AM
How does it do it? Presumably in the simplest keys, each of the targets have properties, and it will compare a property of first target with a property of a second target.
Does this sound right? I'd like to get down to something measurable here.  (Point 3).
Yea, it's just a bunch of property comparisons. This article has a lot of the details from the "normal" target-sorter (engine-damage-priority stuff uses a modified version, as do some other types) : https://www.kickstarter.com/projects/arcengames/ai-war-ii-0/posts/1763996

C# mods is fine by me. The Linux folks have some extra hurdles, though.
Linux folks would need to use MonoDevelop or something like that for their C# IDE and compiler, which isn't as nice as Visual Studio on Windows, but they can definitely still write and compile C# that runs on all three platforms.

Assuming we can replace graphic and sound assets a lot easier than we can replace game core functions
It won't involve messing with dlls, and music will still be a directory you can change as you see fit (though we'll probably add support for multiple music folders that turn on/off with their respective mod), but graphics and sounds are baked into the unity .asset bundle now, as otherwise it takes absolutely forever to load them off disk. There's probably some alternate loading method we can use to just grab a texture and mesh off disk for when you're testing something, but even there we haven't worked out the workflow. And presumably before publishing a mod one would want to bake the assets (into a separate bundle for that mod, not stomping on other stuff); presumably this can be done with Unity Free, but again we haven't worked out the workflow.

If y'all know of other unity games using the .asset bundles for graphics/sounds and allowing modding of those, I'd like to know about them and how they do it. I don't anticipate it being as big a bear as working out this dll workflow (actually doing it was easy, securing it is a fool's errand apparently) but it would probably save me some time finding the examples, and if you know of such unity games where the mod support is actually good to work with in your experience, then that's very helpful info I wouldn't be able to find myself.

Also with the way I understand this works, we would require someone to merge mods...
No, I don't think so, because these would be external dlls, and the external dlls only get called when the xml tells the game to do so. So your xml would reference your dlls, and the other guy's xml would reference his dlls, and whichever mods you had enabled would determine which xml is in the game. It's possible you could have different behaviors that don't make sense together, and if you both replaced the targeting algorithm for the Bomber, or whatever, then you'd have a conflict in the xml. But in general merging would only be necessary if you were both trying to modify the same thing in different ways.

Because I know from KSP what a huge performance penalty even -1- slightly complicated external dll can be.
The mere presence of extra dlls loaded at runtime doesn't bog things down. It's all a matter of what they're doing. And if they're really bogging things down they'd still do that if they were combined into a single dll.

The sandbox would still have access to the game's own directories.  That's part of what this thread is about: preventing access to system files.
I don't think I'd let the sandbox do any File IO if I could help it, and just provide some kind of call it could make that the game's normal app-domain would later consider writing out for you, so you could do persistent state or whatever.

But at this point I still haven't found any method that will prevent file io by any app-domain in a unity game. I'm moving on to the gui workflow for Blue and I.

I think that the code needs to be provided and able to be compiled by the user.
Yea, I tried very hard to build in an environment where you'd just get the source code, and then the first time you ran with that mod enabled the game would compile the code into a mod (and if you changed it and restarted the game it would recompile it). I tried the old CodeDom approach, but unity's version of mono needs mono.exe installed on the system for its replacement for that to work. I tried the Mono-Compiler-Services appoach, but that appears to have the same restriction, and I tried Roslyn, but unity's version of mono isn't new enough to run it.

So there are various things I could do for the mod tools but they would likely be windows-specific (and exclude older stuff like XP, which the game doesn't exclude) and require a particular version of .NET . And why add that restriction when modding itself doesn't require it?

I'd like to just bundle gmcs (mono's compiler) with the game and have it be run by an external mono app, but at that point why on earth are we doing so much work when the simple approach that only cares about runnable code... works? I'd still like to require the distribution of source with the dll, though, so maybe I'll have to revisit that.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Cyborg on December 29, 2016, 11:26:01 AM
Something to think about. But in general, most of the savvy folks are not going to be installing some anonymous person's DLL blind.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Draco18s on December 29, 2016, 11:31:14 AM
Something to think about. But in general, most of the savvy folks are not going to be installing some anonymous person's DLL blind.

Just for reference, Minecraft mods can do all the same stuff a random DLL can.

Sure, it has to go through the JVM, but it can still read and write files anywhere on your computer, it can still read the keyboard, it can still download files from the internet, and still execute external programs.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Cyborg on December 29, 2016, 11:51:15 AM
Something to think about. But in general, most of the savvy folks are not going to be installing some anonymous person's DLL blind.

Just for reference, Minecraft mods can do all the same stuff a random DLL can.

Sure, it has to go through the JVM, but it can still read and write files anywhere on your computer, it can still read the keyboard, it can still download files from the internet, and still execute external programs.

And I don't install those.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Mad Rubicant on December 29, 2016, 02:30:44 PM
Have you tried doing a whitelist of permissions instead of a blacklist, as shown here: http://stackoverflow.com/a/26287330 (http://stackoverflow.com/a/26287330)?
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: eRe4s3r on December 29, 2016, 07:03:40 PM
Assuming we can replace graphic and sound assets a lot easier than we can replace game core functions
It won't involve messing with dlls, and music will still be a directory you can change as you see fit (though we'll probably add support for multiple music folders that turn on/off with their respective mod), but graphics and sounds are baked into the unity .asset bundle now, as otherwise it takes absolutely forever to load them off disk. There's probably some alternate loading method we can use to just grab a texture and mesh off disk for when you're testing something, but even there we haven't worked out the workflow. And presumably before publishing a mod one would want to bake the assets (into a separate bundle for that mod, not stomping on other stuff); presumably this can be done with Unity Free, but again we haven't worked out the workflow.

If y'all know of other unity games using the .asset bundles for graphics/sounds and allowing modding of those, I'd like to know about them and how they do it. I don't anticipate it being as big a bear as working out this dll workflow (actually doing it was easy, securing it is a fool's errand apparently) but it would probably save me some time finding the examples, and if you know of such unity games where the mod support is actually good to work with in your experience, then that's very helpful info I wouldn't be able to find myself.


I know of none. All unity mods are either based on direct edit of assets bundles or loose files with directory structure (which by the way necessitates unpacking the games original assets, since you can't mod what you don't know is there). The big outlier is Cities Skylines which has a very very fancy integrated package creation and loads mods packages, but it has an integrated mod manager and in-depth steam workshop integration.

Fun-fact, you can't easily mod (add) sounds (not music) in Skylines at all ;P And it gets even more problematic when you wanna edit fonts, strings or GUI in general. If it works through Unity Free then that would be fine I guess, but again, you need to give modders a usable example for the workflow.

My disdain from Unity comes directly from the fact that so far, no Unity game really was moddable the big exception would be KSP, but KSP was really more hacked apart and extended/overhauled than modded. The way DLL mods work is absurdly dangerous (there is zero sandboxing, especially of network traffic) but at least there are trusted dependencies now, there were plenty of malicious dlls though.

It's important to remember that moddable and adding new meshes are 2 entirely different things to a modder. Skylines allows you to add new buildings, but it has very few moddable elements aside that. DLL mods now figured out how to inject in the road/traffic parts of the game, but again, we are talking about DLL mods, with all the problems these entail. KSP is the opposite (to me) it has tons of trusted DLL dependencies, and people use these and build upon them their mods. KSP itself? Is still moddable, adding new parts is easy, adding new meshes is however not. But this is primarily due to KSP using a really iffy (and horribly slow) loading system ;P

With Skylines in Steam workshop, there is also ZERO oversight over malicious dll's, but Skylines does somehow restrict "plugin" with versioning and some methods blocked afaik. But yeah.. dll mods are not safe

Minecraft mods are also a big problem, obviously as it is Java, but big (many millions dl's) modpacks are fine.


Anywaaayyyyyy.... Modding isn't easy to support. Easiest would be xml files and obj / dds  ;p Anything that obfuscates (like package files) is by default not helping modding. To know how to add a new unit, we have to see how to actually do that and how it was actually done, after all. Thinking about this in terms of workflow might be much better approach. Define workflows for each element of modding you wanna support (adding new ships/buildings to the AI) (adding new AI) (changing AI loadouts) etc. and then try to unify and condense these into workflow that a modder can follow without access to source files.


Btw. @ Keith
Mod Merging would be a thing if mods add 1 new AI (with new ships, behavior) etc. and what now, if we wanted to use 2 such mods? ;P Targeting behavior has to be merged and expanded for 2 new AI and new ships, no? If I loaded both, then whatever is loaded last would override targeting changes of the first? Mind you, that is theorethical, in reality we'd probably have cooperation between modders ;P
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Aklyon on December 29, 2016, 11:50:26 PM
And then theres stuff like Xcom2 having with a modding download as large as the game itself, and the game alone is 40gb.
Modding is complicated.
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: eRe4s3r on December 30, 2016, 02:40:39 AM
And then theres stuff like Xcom2 having with a modding download as large as the game itself, and the game alone is 40gb.
Modding is complicated.

Only when you use engines without making modding a thing you want to support. Skyrim modding is not (per sé) complicated, and adding meshes to Skyrim is perfectly doable with a short read of the tutorials and 1h of time to understand them. Heck it took me a few hours to import a skeleton, modify existing clothing and fix a few animation issues I had (with cloaks) and adapt them for various body types of other mods including advanced skeletons...  modifying the game world is basically easy, doing anything worthwhile with the creation kit obviously is not.

Divinity Original Sin had this problem, super fancy editor, but it was heavily restricted by not giving us any of the game scripts and a tutorial which didn't even explain how to have a mod behave like the original GAME. (when you create a new mod, it is essentially a new game within that engine, not a mod of the vanilla game, because the game doesn't exist as such, the entire campaign, scripted to the core, is "the game script"). As you can imagine, this killed D:OS modding entirely. I have never seen a game with worse mod support than D:OS, even though it had one of the best editors I had ever seen. And that is saying something ^^ Documentation and workflow are absolutely vital for modders. If I can't figure out how to do something then how would someone fare who just wants to experiment with a few minor changes and gets addicted from there?

Modding is not complicated ;) DLL modding however is far beyond my abilities ^^

And to give you an example, just a few weeks ago I modded ARCC for Anno 2070 for myself (changes a few values that I found weird). What did that take? RDA explorer, unpack, edit xml files, repack -> replace -> done. Despite that, there are only 2 mods for Anno 2070, and that is because while this action (modifying) is easy, creating something new was not explained, detailed, and for modding we still have to use the hacktool (annocookie) to even play the game properly... ^^
Title: Re: Sandboxing code in C# (for AIW2 dll modding)
Post by: Aklyon on December 30, 2016, 11:26:01 AM
Documentation and workflow are absolutely vital for modders. If I can't figure out how to do something then how would someone fare who just wants to experiment with a few minor changes and gets addicted from there?
This is (most of) the problem I've got with Starsector, actually. Its neat, it has a lot of neat mods, and I want to mess with a couple ship ideas, but I can't figure out where to start.
The rest isn't actually a problem on the game end, just me being rubbish at art.