Any good use for machine learning/'real' AI?

[NichG]

So, I can't really volunteer to directly work on AIW2, but given that I find myself currently working on machine learning and AI professionally due to bizarre twists of fate, I'd be happy to discuss anything related to machine learning techniques in case there's any possible benefit or use of that knowledge to be gained here.

For the record, yes, I know that trying to design AI War's AI to work via machine learning techniques is a terrible idea both for dev time and also difficulty of making it fun. I was thinking indirect stuff, not even included in the executable players would receive - stuff like collecting data on beta testers' layouts of turrets, choices of targets, etc, and making predictive models to help understand and solve balance issues.

Anyhow, its mostly that I thought this might be interesting to talk about, not as a 'feature suggestion' or anything like that. Though there is that tiny part of me that wonders if AIW2 could capitalize on the deep learning hype to sell a few more copies ('now with Q networks!')

[Chthon]

So, I can't really volunteer to directly work on AIW2, but given that I find myself currently working on machine learning and AI professionally due to bizarre twists of fate, I'd be happy to discuss anything related to machine learning techniques in case there's any possible benefit or use of that knowledge to be gained here.

For the record, yes, I know that trying to design AI War's AI to work via machine learning techniques is a terrible idea both for dev time and also difficulty of making it fun. I was thinking indirect stuff, not even included in the executable players would receive - stuff like collecting data on beta testers' layouts of turrets, choices of targets, etc, and making predictive models to help understand and solve balance issues.

Anyhow, its mostly that I thought this might be interesting to talk about, not as a 'feature suggestion' or anything like that. Though there is that tiny part of me that wonders if AIW2 could capitalize on the deep learning hype to sell a few more copies ('now with Q networks!')

Machine learning is just one "real AI." The AI they used in AI War is another real AI. Neither one is fake.

As for Machine Learning, it is processor intensive. Very much so. We're talking about thousands of floating point calculations a second. You would likely need a second video card to run just the AI. That being said, it would be interesting if your game play transcript (a series of decision events for the computer based on your situation) was fed back into a neural network somewhere, so that it could update the AI, and they release patches based on it's updates to improve the AI. That could be a thing.

However what Keith already does might actually be better for improving the AI, even though it's less automatic. He looks at what players did, and comes up with ways to counter it using logic. This is something that is difficult to teach a computer to do.

[NichG]

Machine learning is just one "real AI." The AI they used in AI War is another real AI. Neither one is fake.

As for Machine Learning, it is processor intensive. Very much so. We're talking about thousands of floating point calculations a second. You would likely need a second video card to run just the AI. That being said, it would be interesting if your game play transcript (a series of decision events for the computer based on your situation) was fed back into a neural network somewhere, so that it could update the AI, and they release patches based on it's updates to improve the AI. That could be a thing.

However what Keith already does might actually be better for improving the AI, even though it's less automatic. He looks at what players did, and comes up with ways to counter it using logic. This is something that is difficult to teach a computer to do.

It doesn't have to be so processor intensive, though it can be if you want it to be. I mean, the really extreme end is people basically trying to replicate the human visual system in order to play Pac Man - its not because thats the efficient way to do it, its because that shows off that it's possible to do it that way. Also, training is always much more expensive, so if you're going to use something in the game where performance might matter, don't try to train it as you go - you'll want to ship a pre-trained model. Though, at a very simple ML level, genetic algorithms acting on stuff like fleet compositions would basically be free since you just have to record what fraction of each AI ship type died and adjust the ratios in future waves accordingly. You can also get a lot done with linear controllers, which are quite cheap, but even at that level you'll almost certainly need more data than a single player can provide (maybe ~300 examples before you get a noticeable effect). Mostly, you really need to train offline, using the entire player base as the data source.

But anyhow, in-game decision making stuff is going to be unnecessarily difficult to balance and test. Even if you can do it cheaply, its hard to say whether it would be noticeable, and if noticeable, whether it would be fun. But there's lots of things you can use AI for that aren't actually being the AI opposition.

For example, for people with a higher performance budget, you could use GPU vertex shaders to animate individual ships in swarms. The minimum update rule for ship positions is something that does a fixed function call to assign x,y,z(t) - something like x_i = x_group + a*sin(t+phi_i), y_i = y_group+vofs_i, z_i = z_group + a*cos(t+phi_i). To get swirly motion, you're looking at a couple of trig func evals, some multiplications, some additions. On the other hand, you could train a small recurrent network to produce interesting/meaningful motion patterns in a variety of ways (for example, train two ships to dogfight, where the one has to keep the other in front of it at all times). Lets say this is one or two sensor inputs, one or two hidden neuron, and vx,vy,vz. So that would cost ~49 adds and mults for a 7x7 matrix multiplication, and 7 evals of some nonlinear function (probably you want tanh, but you could do a crude piecewise linear approximation to that). It's a little more expensive, but not so much - you could probably still run a hundred thousand of those in real time on a decent GPU, especially if you used interpolation so you didn't have to actually evaluate the control network every frame.

Of course, you could also do this entire thing offline - train the networks, generate interesting motion patterns, and then just play those motion patterns back as a static animation within the game.

As far as 'look at what players do and come up with counters by hand', that's something that machine learning can help a lot with in terms of exposing things like that when there's a lot of data to sort through. If you've got ten thousand players, you can look at how _all_ of them are playing, and for example train an XGBoost model to predict how likely it is for the player to win or lose or give up on the campaign given what they're currently doing. That kind of model produces a list of feature importances - it will tell you 'if they buy lightning turrets by the third planet they take, it's 5% predictive of a win; if they don't buy a command station upgrade by the 5th planet, its 3% predictive of a loss, etc; losing any world on a 5/5 game is 15% predictive of whether the player will quit soon' etc. So that kind of thing can be used as a tool to find what might be worth paying attention to, but which none of the players are actually saying. At a more zoomed out level, you could even cluster different play styles and identify if there are subsets of the player base who just play the game in fundamentally different ways. Something simple like learning how to order different units in terms of the increased/decreased chance of victory in an engagement could help in fine-grained balancing of unit caps and unit costs. You can in principle tell if some units are just plain better than others, or if the majority effect is the rock-paper-scissors dynamic. You can also learn to cluster ship types or building types to detect design redundancies.

For very out-there stuff, there are people generating stuff like entire artificial Magic: the Gathering card sets with RNNs. With enough examples, the RNN can actually learn the structure of MtG's design space, and creates new cards consistent with those implicit rules (like red should deal damage, and about this much damage for this much mana, and so on). So you could go as far as to actually generate a balanced set of random new ship designs for every play. Neural net driven Hybrids AI Plot. That's at the extreme end though.

In terms of the actual AI components, you could train a predictor to try to figure out where the player is likely to attack next, and use that to have an AI that can anticipate the player. You can do all of the heavy-duty training elsewhere and just ship a pre-trained net which has inputs that can be used to condition the prediction (e.g. first it classifies the player, then it uses that classification and recent game state to predict what they'll do next). You only really need a single evaluation of that network every time the large-scale strategic situation changes (every time a planet is captured, for example), so even if one eval is relatively expensive, the overall performance hit would be negligible. You could then use that prediction in various ways - tie it in with the rule-based AI design and make a 'Scaredy Cat' personality that moves ships away from the place that will get hit next, or a 'Prophet' personality that tries to consistently have special forces or whatever patrolling through the most likely target planets, or a 'Surprise Attack' personality that anticipates when the player is about to move and throws a way just a moment before. The way the information from the network is used can be entirely hand-crafted.

Anyhow, these are just some example ideas. Again, I'm not seriously saying 'AI War should have this, please add it!' . But I love thinking about and talking about this kind of thing.

[jenya]

AI in AI War is not learning, it is emergent ("nondeterministic AI that was not dependent on having any historical knowledge"):
http://christophermpark.blogspot.ru/2009/07/designing-emergent-ai-part-4.html

[Chthon]

The problem you're missing is it has to be in real time. This means you have X factors that can result in Y consequences, and requires Z points connection in between that have to be checked N times every second. That's X*Y*Z*N floating point operations a second. This adds up fast.

I'm also not talking about the computer doing it visually. I don't know where you'd think that was the case. Maybe you got confused when I suggested the graphics cards help, but I'm talking more along the lines of CUDA. Even if the factors were something like "Strength at Planet X1, Strength at Planet X2" etc, you could have 80-120 planets. The results could be similarly "Attack Planet Y1, Attack Planet Y2" etc. The points Z would have be something along the lines of X*Y itself in order to have enough connections for it to learn. This makes it close to 80^4 operations every update at only 80 planets. If the AI only updated once a second this would mean 40,960,000 floating point operations a second. This also would be a very lean neural network, and it would ignore things like fleet composition and stuff.

While a modern video card can help with this, it is already busy doing floating point math just trying to put the image on your screen. You would have to significantly drop the frame rate to accommodate the AI. There aren't many who would like this, especially since they are trying to improve the game's performance. If we had a 120 planet scenario it's even worse at 207,360,000 operations a second. We're also still looking at just one update a second. Perhaps that could be exploitable by the player and more are needed. Then you may need even more updates further compounding the overhead.

Typically in cases where you use a neural network, the speed of the task is limited to the speed of the network. In this case the speed of the network must match the speed of the task. Unless the speed of the network can be made trivial, which right now it can't be without serious hardware, it is impossible.

That being said, there are alternatives, one of which I mentioned, where the game sends back a series of situations, decisions, and results of those decisions, for a neural network somewhere else to analyze.

One day we might have a computer commonly available to do this, but according to http://wccftech.com/ultimate-cpu-gpu-floating-point-performance-battle-amd-intel/ this is not now. Processors at the top of the line do 60-112 gflops, and the best video card does 737.3 gflops. This is assuming you can use it in 32-bit floating point mind you. They're much slower if you need 64-bit. While even 60 gflops sounds nice for a processor doing this, remember that it also has to run the game as well, and this is the total for all cores combined. We are trying to improve performance first. This means minimizing things that processors are not very good at like floating point operations.

Edit: Other things you might want to take into account:
AI ship strength and composition per planet
Neutral ship strength and composition per planet
Planet ownership

Other outcomes:
Ship movement for the AI (Posturing)
Reinforcement selection
Reconquest

All of these things increase the number of flops exponentially when included. Our meager network described above I realized wouldn't even be the tip of the iceberg if we want the network to play well.

[Lord Of Nothing]

The problem you're missing is it has to be in real time. This means you have X factors that can result in Y consequences, and requires Z points connection in between that have to be checked N times every second. That's X*Y*Z*N floating point operations a second. This adds up fast.

I'm also not talking about the computer doing it visually. I don't know where you'd think that was the case. Maybe you got confused when I suggested the graphics cards help, but I'm talking more along the lines of CUDA. Even if the factors were something like "Strength at Planet X1, Strength at Planet X2" etc, you could have 80-120 planets. The results could be similarly "Attack Planet Y1, Attack Planet Y2" etc. The points Z would have be something along the lines of X*Y itself in order to have enough connections for it to learn. This makes it close to 80^4 operations every update at only 80 planets. If the AI only updated once a second this would mean 40,960,000 floating point operations a second. This also would be a very lean neural network, and it would ignore things like fleet composition and stuff.

While a modern video card can help with this, it is already busy doing floating point math just trying to put the image on your screen. You would have to significantly drop the frame rate to accommodate the AI. There aren't many who would like this, especially since they are trying to improve the game's performance. If we had a 120 planet scenario it's even worse at 207,360,000 operations a second. We're also still looking at just one update a second. Perhaps that could be exploitable by the player and more are needed. Then you may need even more updates further compounding the overhead.

Typically in cases where you use a neural network, the speed of the task is limited to the speed of the network. In this case the speed of the network must match the speed of the task. Unless the speed of the network can be made trivial, which right now it can't be without serious hardware, it is impossible.

That being said, there are alternatives, one of which I mentioned, where the game sends back a series of situations, decisions, and results of those decisions, for a neural network somewhere else to analyze.

One day we might have a computer commonly available to do this, but according to http://wccftech.com/ultimate-cpu-gpu-floating-point-performance-battle-amd-intel/ this is not now. Processors at the top of the line do 60-112 gflops, and the best video card does 737.3 gflops. This is assuming you can use it in 32-bit floating point mind you. They're much slower if you need 64-bit. While even 60 gflops sounds nice for a processor doing this, remember that it also has to run the game as well, and this is the total for all cores combined. We are trying to improve performance first. This means minimizing things that processors are not very good at like floating point operations.

Edit: Other things you might want to take into account:
AI ship strength and composition per planet
Neutral ship strength and composition per planet
Planet ownership

Other outcomes:
Ship movement for the AI (Posturing)
Reinforcement selection
Reconquest

All of these things increase the number of flops exponentially when included. Our meager network described above I realized wouldn't even be the tip of the iceberg if we want the network to play well.

Um, not to wish to be rude, but he's making it VERY clear in his posts that he's not talking about something that would be done on the player's end while they're playing, but rather something done on arcen's end using data and to inform stuff. Whether it's a good idea, i don't pretend to know, but your objection, while technically fine (To my knowledge) is irrelevant.

[Chthon]

Um, not to wish to be rude, but he's making it VERY clear in his posts that he's not talking about something that would be done on the player's end while they're playing, but rather something done on arcen's end using data and to inform stuff. Whether it's a good idea, i don't pretend to know, but your objection, while technically fine (To my knowledge) is irrelevant.

Actually, if you read what I said again, I am talking about his idea of pre-trained network. I did not include any talk about back propagating the network to train it further. Such a task would at least double the processing power required. My point is either you end up with a simplistic AI that can't respond properly, or you end up with an AI that requires a whole new computer to run on and play against you.

Perhaps I wasn't making myself clear to those that don't know neural networks though. I hope this helps.

To use a pre-trained neural network, you have to calculate the weights of the connections between many nodes. These are floating point operations.

On the other hand, you could translate the neural network into an expert system, which if done correctly, could do the same takes in a fraction of the time. However I'm not sure myself how that would be done. I've only heard of it.

[eRe4s3r]

AI in AI War is not learning, it is emergent ("nondeterministic AI that was not dependent on having any historical knowledge"):
http://christophermpark.blogspot.ru/2009/07/designing-emergent-ai-part-4.html

The fact that it is producing emergent results (as in, not expected) does not make it an AI. If you throw 50 stones at a wall, you get an emergent hit pattern, 1 rock might even ricochet, 1 rock will land on the moon and the other will cause the end of the dinosaurs. But that is not a sign of your intelligence, it's just a sign that you got a lot of rocks to throw and really bad aim. Which means I could deduce just from that that you are human. An AI would not have bad aim. It would aim 50 times at the same spot, hitting the same spot. And thus create the "hole in the wall" problem. Any scripted AI falls prey to this, although some rts games managed to nearly entirely hide it.

Flowery analogies aside, to me this easy predictability of current pseudo AI scripts is the primary reason I get really quickly bored of rts games, and yes that includes AI War 1, though here with the many different scripts (AI personalities) I had a lot of fun after all, and a lot of suffering with certain combos... but my point is there isn't anything intelligent about anything the AI in AI War 1 does, it really just throws a lot of rocks at you and it doesn't just have bad aim, it flat out does not aim (as in, it does not care what you do, aside from reinforcement and the doom counter). That is producing emergent results, true, but is it a sign of intelligence? ;P


But all this is kinda moot, because I honestly do not believe machine learning could make an AI that would be a fun enemy AI personality in AI War .... how would you ever pre-train it.. you'd need to play through at least 30000 to 100000 generations. That is an unfathomable scale of work. And at the end, it would be heavily debatable whether what you produced is actually capable of playing AI War against a human, and is instead only capable of playing against a CERTAIN human playstyle (And even that is questionable, because I would bring up the notion that you can machine learn an AI , but you would spoiling the results by shortsighted or bad definition of goals, and this is why there may be a GO AI, but there ain't any AI that passed the Turing test.

And if you now say, "but ERE you unfathomable beautiful being that types a lot of text, there was an AI in the news that passed the Turing test.."

Well, read the actual conversation
http://www.scottaaronson.com/blog/?p=1858

And then weep, weep hard. This passed as human response to an actual human being, which really tells us more about the human than the "pseudo" intelligent chatbot. 30% thought this chatbot was a human??

Which reminded me of a quote I read somewhere, probably on an off-link from that AI researchers page about the Turing Test

A Turing Test is a conversation with a goal: for the subject, to prove that it’s human; for the interrogator, to learn the truth. And both of them know that this is the goal, and both know that the other knows it. So what’s the use pretending otherwise?

Think hard how you would prove you are human just by text. This doesn't require a vastly sourced database, or huge intelligence, you could just ask "how do you feel about Trump" and an AI would give you a response, so would a human.

And then you kill the AI; by asking WHY?

And Whatson, it crawled the net for responses, you can imagine how that turned out (if you imagine porn and cat pics, that is exactly not how it turned out because imagine that, an AI has no notion what is a cat pic or porn, instead it became a hateful troll because it was sourced from twitter)

Anyway lots of text, it's a funny discussion that I enjoy thinking about to be honest, but machine learning does not lead to intelligent anything, it is just a more complicated way to script responses to situations you expect. Because let's face it, the crux of a machine learned AI is the teacher and how would you even begin to train an AI for AI War ?? You'd have to play against a godlike human player and play as AI so that the AI learns from that??? But the moment you encounter a player that plays totally different, your trained AI is totally borked... (anyone still remember transport ship raids? AI could never cope with that, so the devs nerfed transports removing player agency from the game and thus reduce the amount of weirdness the AI encounters ;P)

So yeah, machine learned AI for AI War.. would never work. Imo. I would be happy to learn how ye would envision it working though. Just think about AI in AI War, it's a lot of stuff an AI has to do and even a good human player would be terrible at managing 80 planets at the same time, so right there you would already have a problem that you as human couldn't teach a proper self-learning AI anything about how to play as AI. Unless I misunderstand on what level you want to employ machine learning.

When you say pre-trained, on what level would you employ pre-training, and how would your formulate the decision tree?

Ps.: I am also aware AI researches don't care about chatbots currently, as the fundamental technology is still so far away that it isn't even worth to contemplate. I just wanted to give these examples of things that the media called AI's, which were really nothing of the sort

[Draco18s]

And Whatson, it crawled the net for responses, you can imagine how that turned out (if you imagine porn and cat pics, that is exactly not how it turned out because imagine that, an AI has no notion what is a cat pic or porn, instead it became a hateful troll because it was sourced from twitter)

Watson got pointed at Urband Dictionary at one point, either intentionally or accidentally I don't remember, and its vocabulary expanded immensely.

Buuut it didn't have a concept of "polite company" and when certain terms were allowed to be used and the researches had to go "oh god" and wipe its memory and keep it from ever going to that site again.

[NichG]

Chthon, you're seriously overestimating the cost. Part of that is that you're assuming a kind of worst case 'throw everything at it including the kitchen sink' combined with 'the only component in the system must be a neural network and everything must be done by it' combined with 'oh, and we want updates every second'. Yes, if you design it badly, you'll have problems.

But 'I just want a neural network to control everything about the game' is a research problem, not a practical engineering approach. It sounds great in papers because you can say 'see, look, its possible', but its totally unnecessary for real deployment. You don't need, nor would you gain anything, from having a giant neural net control the motion of one of the ships exactly in response to what the player is doing 20 systems away.

To be concrete, here is an example of a system you could make, and exactly how much it would cost.

Inputs:
- Current AIP, HAP (2)
- Player ship average tech level (current in fleet, and unlocked) (2)
- Player available and total energy (2)
- Player available metal and rate of income (2)
- Number of player worlds; minimum distance from AI warp gate to a player world; minimum distance to AI core world 1 and 2; Graph radius of player's territory; Number of disconnected clusters (6)
- Number of 'neutered' worlds (guard posts, warp gates killed; command center left operational) (1)
- Shield generators left of each type (3)
- Number of border worlds (1)
- Player ownership numbers for: turret controllers, fabricators, advanced factory, advanced starship factory, Zenith power generators; Number of data centers and co-processors killed (7)
- Total player tech points spent and unspent (2)
- Number of planets of each class (see later) that the player has taken so far (11)

So that's 39 inputs, but it covers most of the large-scale strategic situation. 39 is cheap, so we could even quadruple this without much issue, but lets go with this for concreteness for now.

Prediction target: what type of planet will the player attack next (independent probability for each class, not mutually exclusive)? Types could be...
- 'Empty' (no particular strategic value)
- 'Approach' (planet adjacent to a strategically important target, such as AI Home world or one of the other types)
- 'Connect' (planet which connects two disconnected or distant collections of player's worlds, defined as anything which reduces graph radius by >threshold)
- 'Hack' (player will send a Hacker rather than actually wipe out a planet; planets with Backup servers indicated)
- 'Neuter' (player will wipe a system's defenses but not knock out command center)
- 'Fabricator'
- 'Factory'
- 'Turret'
- 'ZPG'
- 'AIP reduction'
- 'Core attack'

Add as many as you're interested in, but 11 for now.

Lets do something like 2 hidden layers, 128 neurons each, which is generally reasonable for this kind of thing. So the network looks like 39->128->128->11, which is about 50000 floating point operations. There's also whatever you need for the nonlinearities, so lets call it 60000. Now, we don't need to update this data every second, only when the player does something that significantly changes it. So we could update maybe once a minute.

A cost of 60000 floating point operations every 60 seconds is 1000 FLOPS. You could have run this network back in the 70s. It's not terribly different from the kinds of networks that contribute to winning Kaggle solutions in 2016. For example, in the Otto competition a year ago, the highest performing single neural network in our 6th place solution was just 94->200->300->80; a bit bigger, but not all that much.

For unstructured data, it can be really, really cheap to get reasonable predictions.

Incidentally, recent results show that you don't need very much precision for neural networks. You can replace all the floating point numbers with binary and get almost the same accuracy.

[Chthon]

AI in AI War is not learning, it is emergent ("nondeterministic AI that was not dependent on having any historical knowledge"):
http://christophermpark.blogspot.ru/2009/07/designing-emergent-ai-part-4.html

The fact that it is producing emergent results (as in, not expected) does not make it an AI. If you throw 50 stones at a wall, you get an emergent hit pattern, 1 rock might even ricochet, 1 rock will land on the moon and the other will cause the end of the dinosaurs. But that is not a sign of your intelligence, it's just a sign that you got a lot of rocks to throw and really bad aim. Which means I could deduce just from that that you are human. An AI would not have bad aim. It would aim 50 times at the same spot, hitting the same spot. And thus create the "hole in the wall" problem. Any scripted AI falls prey to this, although some rts games managed to nearly entirely hide it.

Flowery analogies aside, to me this easy predictability of current pseudo AI scripts is the primary reason I get really quickly bored of rts games, and yes that includes AI War 1, though here with the many different scripts (AI personalities) I had a lot of fun after all, and a lot of suffering with certain combos... but my point is there isn't anything intelligent about anything the AI in AI War 1 does, it really just throws a lot of rocks at you and it doesn't just have bad aim, it flat out does not aim (as in, it does not care what you do, aside from reinforcement and the doom counter). That is producing emergent results, true, but is it a sign of intelligence? ;P


But all this is kinda moot, because I honestly do not believe machine learning could make an AI that would be a fun enemy AI personality in AI War .... how would you ever pre-train it.. you'd need to play through at least 30000 to 100000 generations. That is an unfathomable scale of work. And at the end, it would be heavily debatable whether what you produced is actually capable of playing AI War against a human, and is instead only capable of playing against a CERTAIN human playstyle (And even that is questionable, because I would bring up the notion that you can machine learn an AI , but you would spoiling the results by shortsighted or bad definition of goals, and this is why there may be a GO AI, but there ain't any AI that passed the Turing test.

And if you now say, "but ERE you unfathomable beautiful being that types a lot of text, there was an AI in the news that passed the Turing test.."

Well, read the actual conversation
http://www.scottaaronson.com/blog/?p=1858

And then weep, weep hard. This passed as human response to an actual human being, which really tells us more about the human than the "pseudo" intelligent chatbot. 30% thought this chatbot was a human??

Which reminded me of a quote I read somewhere, probably on an off-link from that AI researchers page about the Turing Test

A Turing Test is a conversation with a goal: for the subject, to prove that it’s human; for the interrogator, to learn the truth. And both of them know that this is the goal, and both know that the other knows it. So what’s the use pretending otherwise?

Think hard how you would prove you are human just by text. This doesn't require a vastly sourced database, or huge intelligence, you could just ask "how do you feel about Trump" and an AI would give you a response, so would a human.

And then you kill the AI; by asking WHY?

And Whatson, it crawled the net for responses, you can imagine how that turned out (if you imagine porn and cat pics, that is exactly not how it turned out because imagine that, an AI has no notion what is a cat pic or porn, instead it became a hateful troll because it was sourced from twitter)

Anyway lots of text, it's a funny discussion that I enjoy thinking about to be honest, but machine learning does not lead to intelligent anything, it is just a more complicated way to script responses to situations you expect. Because let's face it, the crux of a machine learned AI is the teacher and how would you even begin to train an AI for AI War ?? You'd have to play against a godlike human player and play as AI so that the AI learns from that??? But the moment you encounter a player that plays totally different, your trained AI is totally borked... (anyone still remember transport ship raids? AI could never cope with that, so the devs nerfed transports removing player agency from the game and thus reduce the amount of weirdness the AI encounters ;P)

So yeah, machine learned AI for AI War.. would never work. Imo. I would be happy to learn how ye would envision it working though. Just think about AI in AI War, it's a lot of stuff an AI has to do and even a good human player would be terrible at managing 80 planets at the same time, so right there you would already have a problem that you as human couldn't teach a proper self-learning AI anything about how to play as AI. Unless I misunderstand on what level you want to employ machine learning.

When you say pre-trained, on what level would you employ pre-training, and how would your formulate the decision tree?

Ps.: I am also aware AI researches don't care about chatbots currently, as the fundamental technology is still so far away that it isn't even worth to contemplate. I just wanted to give these examples of things that the media called AI's, which were really nothing of the sort

The fact is there is nothing we can make a computer do that will make it "intelligent" at the moment. There is much we do not understand about intelligence. Artificial intelligence is a different story. That is where we make the computer appear to have some intelligence. However no matter what it is, whether it be emergent, neural network, or otherwise, it is simply following a set of rules provided to it by a human. Yes, even a neural network follows a set of rules from a person, though we try to take ourselves as much out of it as possible.

All of the things you mention as not being AI are in fact AI, just not very smart AI. Just because it cannot pass the Turing Test means nothing in that regards. The Turing Test is not to test to see if something is an AI, it is to test to see if it is indistinguishable from human intelligence. Yes, any of our AIs available are a long ways from passing that, unless you compare computer poetry to certain poets. (shots fired)

Chthon, you're seriously overestimating the cost. Part of that is that you're assuming a kind of worst case 'throw everything at it including the kitchen sink' combined with 'the only component in the system must be a neural network and everything must be done by it' combined with 'oh, and we want updates every second'. Yes, if you design it badly, you'll have problems.

But 'I just want a neural network to control everything about the game' is a research problem, not a practical engineering approach. It sounds great in papers because you can say 'see, look, its possible', but its totally unnecessary for real deployment. You don't need, nor would you gain anything, from having a giant neural net control the motion of one of the ships exactly in response to what the player is doing 20 systems away.

To be concrete, here is an example of a system you could make, and exactly how much it would cost.

Inputs:
- Current AIP, HAP (2)
- Player ship average tech level (current in fleet, and unlocked) (2)
- Player available and total energy (2)
- Player available metal and rate of income (2)
- Number of player worlds; minimum distance from AI warp gate to a player world; minimum distance to AI core world 1 and 2; Graph radius of player's territory; Number of disconnected clusters (6)
- Number of 'neutered' worlds (guard posts, warp gates killed; command center left operational) (1)
- Shield generators left of each type (3)
- Number of border worlds (1)
- Player ownership numbers for: turret controllers, fabricators, advanced factory, advanced starship factory, Zenith power generators; Number of data centers and co-processors killed (7)
- Total player tech points spent and unspent (2)
- Number of planets of each class (see later) that the player has taken so far (11)

So that's 39 inputs, but it covers most of the large-scale strategic situation. 39 is cheap, so we could even quadruple this without much issue, but lets go with this for concreteness for now.

Prediction target: what type of planet will the player attack next (independent probability for each class, not mutually exclusive)? Types could be...
- 'Empty' (no particular strategic value)
- 'Approach' (planet adjacent to a strategically important target, such as AI Home world or one of the other types)
- 'Connect' (planet which connects two disconnected or distant collections of player's worlds, defined as anything which reduces graph radius by >threshold)
- 'Hack' (player will send a Hacker rather than actually wipe out a planet; planets with Backup servers indicated)
- 'Neuter' (player will wipe a system's defenses but not knock out command center)
- 'Fabricator'
- 'Factory'
- 'Turret'
- 'ZPG'
- 'AIP reduction'
- 'Core attack'

Add as many as you're interested in, but 11 for now.

Lets do something like 2 hidden layers, 128 neurons each, which is generally reasonable for this kind of thing. So the network looks like 39->128->128->11, which is about 50000 floating point operations. There's also whatever you need for the nonlinearities, so lets call it 60000. Now, we don't need to update this data every second, only when the player does something that significantly changes it. So we could update maybe once a minute.

A cost of 60000 floating point operations every 60 seconds is 1000 FLOPS. You could have run this network back in the 70s. It's not terribly different from the kinds of networks that contribute to winning Kaggle solutions in 2016. For example, in the Otto competition a year ago, the highest performing single neural network in our 6th place solution was just 94->200->300->80; a bit bigger, but not all that much.

For unstructured data, it can be really, really cheap to get reasonable predictions.

Incidentally, recent results show that you don't need very much precision for neural networks. You can replace all the floating point numbers with binary and get almost the same accuracy.

The problem I have with your simplification is how does the computer know where it should attack. Will you have rules in that regard? Will it go for the closest target? Will it go for the most expensive target? Your proposed AI is pretty dumb in that regard (no offense) as it only seems to choose when and if it should attack. I still hold that an AI best suited for this using neural networks would be one that trains an expert system that plays against you. The expert system uses everything the neural network has learned from you, and can operate much more quickly. The price you pay is how fast it can adapt to you.

However even if you do this, it can be done on the same machine in theory. If the neural network takes updates every few minutes from the game position, and determines whether or not it made a good set of moves in that time or not, it can adjust, and then readjust the expert system. Once again, we're dividing the number of operations accordingly to something that I even admit you should be able to do. After the game is done, the network can then be offered a chance to retrain on all of the data from second to second, deciding what it would have done, and looking at what your response might have been. Then when you come back it's even better.

As for needing to do floating point vs. binary, I've heard similar things, but I haven't heard that it was completely proven yet. It's more of a situational basis. Maybe it'd work well for this, maybe not.

Finally, I would like to see AI War continue to have the best computer AI in the genre. If this means taking points from different AI styles, so be it. Finding ways around the problems, and solutions for bottlenecks while still keeping the same excellence in AI is what I imagine we need here. Keith has done an excellent job in the past, the only reason I would want to replace him in this is so he can spend more time making other great games.

[NichG]

The problem I have with your simplification is how does the computer know where it should attack. Will you have rules in that regard? Will it go for the closest target? Will it go for the most expensive target? Your proposed AI is pretty dumb in that regard (no offense) as it only seems to choose when and if it should attack. I still hold that an AI best suited for this using neural networks would be one that trains an expert system that plays against you. The expert system uses everything the neural network has learned from you, and can operate much more quickly. The price you pay is how fast it can adapt to you.

The point of the AI I proposed isn't to replace the game's AI, it's to provide inputs which the programmer can use to express new kinds of AI personalities or inform the decisions made by fixed rules. For a game like AI War, its actually not good if the AI is too smart because the game is already stacked in its favor. Instead, the point is to make something which has a distinct feeling of presence and personality. So the point in doing something like predictive modeling of the player is that playing against an agent who seems to understand you and respond to what you do is going to feel more like there's a real mind behind the opposition than something which follows static policies. In a sense, the idea would be to make the player feel like the AI is actually paying attention to them.

In a broader sense, AI techniques in general let you ask questions based on the ends, rather than the means. For instance, asking something like 'what will the player do next?' is easy to learn an answer to since you just accumulate data on situations and what the player did next in that situation, and try to be as accurate as possible. But it'd be really hard to ask using a rule-based system since you as the programmer would have to personally understand the player enough to know how the player makes their decisions, and then would have to essentially hand-craft a copy of their decision process. Similarly, things like 'at the rate things are going, will the AI win?' - easy to estimate from data, hard to hand-code rules for. At some cost and increase in the amount of data needed, that can be extended to questions about how to achieve outcomes, for example 'how do I provoke the player to attack this particular planet?' - you collect data on cases where the AI acts randomly, build a predictive model to determine what planet will be attacked based on the actions, and then do a search (gradient descent, Monte Carlo tree search, whatever) to find the action pattern that increases the probability of the outcome you want.

But it really works best (and cheapest) when you keep the questions you ask very specific, and very local to a given moment, rather than broad things like 'what should I do to win the game?'. The point is, don't treat it like a miracle solution that can (and must) do everything on its own, treat it as another tool in your toolkit.

However even if you do this, it can be done on the same machine in theory. If the neural network takes updates every few minutes from the game position, and determines whether or not it made a good set of moves in that time or not, it can adjust, and then readjust the expert system. Once again, we're dividing the number of operations accordingly to something that I even admit you should be able to do. After the game is done, the network can then be offered a chance to retrain on all of the data from second to second, deciding what it would have done, and looking at what your response might have been. Then when you come back it's even better.

I wouldn't use a neural network for this - data requirements are too high. You can use a genetic algorithm in this situation though. Still, for AI War I'd be hesitant, because it would tend to make the different AI personalities more generic (because the learning process is always focusing on the goal you give it, so its harder to make things still feel different despite all converging towards that one goal). If you have a bunch of different objective functions you could aim at personality that way, but it feels like now we've gone from the 'I can do this in 3 hours of hacking' type of thing to '1-2 month research project' type of thing.

As for needing to do floating point vs. binary, I've heard similar things, but I haven't heard that it was completely proven yet. It's more of a situational basis. Maybe it'd work well for this, maybe not.

Here's an example paper on the subject: https://arxiv.org/pdf/1602.02830v3.pdf
Gets 10.15% error on CIFAR-10, compared to 7.62% which was state of the art at that time. There's a difference, but if you need a lot of speed you don't have to sacrifice too much accuracy to get it - they were getting ~23x speedup for that accuracy price.

Finally, I would like to see AI War continue to have the best computer AI in the genre. If this means taking points from different AI styles, so be it. Finding ways around the problems, and solutions for bottlenecks while still keeping the same excellence in AI is what I imagine we need here. Keith has done an excellent job in the past, the only reason I would want to replace him in this is so he can spend more time making other great games.

No one is talking about replacing anyone here. Its more like 'hey, there's this really cool set of tools that came a long way in the last 2 years, they're really accessible, and there's neat stuff you can do with them if you think a little outside the box and don't try to make AlphaGo: AI War edition'.