Arcen Games
General Category => AI War II => Topic started by: x4000 on January 23, 2018, 12:13:28 pm
-
I figured I'd make this discussion on the forums so that it's not just a discussion between Keith, Badger, and myself via email.
Badger notes that we really should have NAT punchthrough prior to starting the beta, because that's a big pain in the rear for getting multiplayer set up otherwise. I've noted in the past (prior to the kickstarter) that this was a possibility, but not one we've fully investigated yet, and it's something potentially very time-consuming as well as something with potential ongoing costs.
Basically, the way that NAT punchthrough works, to my knowledge, is via a few methods, which are similar to the private VPNs like Hamachi and whatnot:
1. There's some sort of server hosted somewhere, by Arcen or otherwise, and it keeps track of "game hosts" that say they want other people to be able to connect to them. So, for instance, Badger fires up the game and says he wants to connect.
2. Badger's friends then try to connect to him, via that relay server. At that point the relay server tries to broker a direct connection between Badger and his friends. If it's successful, then that's like getting a green dot in Hamachi. Basically there's no middleman, and because of the respective firewall and router configurations on both ends, things are all happy.
3. But it's also possible that even with the central server, the routers are not going to allow for this type of direct connection. There all manner of router and firewall setups, and there's no real centralized way to keep track of that stuff. So what happens in this case? Well, Badger can talk to the central relay server, and so can his friends, so they can pass messages to one another by talking to the relay and having it... well, relay... those messages on. This is the "yellow dot" situation in Hamachi.
As you can imagine, this introduces a lot of lag, but it ALSO means that all the bandwidth of the gameplay for Badger and his friends is now costing Arcen money directly while he's playing. We can't afford that sort of thing, and this is also the sort of thing that leads to game servers being shut down a few years down the road by AAA companies, and then everyone complaining.
The really large publishers maintain somewhat game-agnostic relay servers, as I understand it, so if they have 2 people playing a game they published 10 years ago, that's just a drop in the bucket on top of the 200k people who are playing their latest titles. Wheee. Steam also maintains relay servers and similar, but we don't use Steam networking, nor do we have any intention of doing so -- that locks us to Steam.
Remember my commentary about the "yellow dot" scenario, above? Well, in those circumstances ALL of the traffic is being routed through Steam instead of going through Forge Networking or similar on our end. This is... suboptimal. What about DRM-free copies of the game, for one thing? Also, Steam's networking is... not my first choice, for a variety of reasons.
Right now we're using a customized version of Forge Networking Remastered (https://github.com/BeardedManStudios/ForgeNetworkingRemastered), which we bought (https://www.assetstore.unity3d.com/en/#!/content/38344) but which is now apparently free and open source. They have a version of NAT punchthrough, but I don't know how robust it is: http://docs.forgepowered.com/nat-hole-punching/
There are other NAT Traversal solutions, such as this one, that are generalized, but mainly geared toward UNET: https://www.assetstore.unity3d.com/en/#!/content/58948
UNET is something we've found to be generally bloated and unreliable, so that wasn't something we wanted to go with. I think that the solution above could be made to work with FORGE, but I'm not sure it would be any better than the stuff built into FORGE. And in ALL of those cases, I'm not sure that this would be zero-cost to Arcen, which is the only way we can guarantee that 20 years from now the networking will still work. We're small!
With that in mind, I'm open to solutions and suggestions. There are various personal VPNs that might have APIs that could be integrated, for instance. Ideally whatever is going on is invisible to the FORGE networking layer (it just thinks it's talking directly to the clients, in other words), like was the case for Hamachi and Wippien and similar.
Next we come to what is essentially a manpower issue:
Now that FORGE is open source, if there are any programmers in the audience here who would like to take a stab at seeing what they can get working with it, and if they can improve things over what Keith has had time to do, then it's certainly worth an attempt at least. We're shortstaffed in this area, and I think that this particular feature is pretty much just not going to happen (I have a gut feel) if someone external isn't able to volunteer their time and expertise. Personally I'd rather have Keith focusing on the game itself, and the AI and whatnot, and other things he's more directly suited to.
-
Hi,
I found this thread through the kickstarter project update message and I'm very interested in doing networked games so I thought I'd reply.
I haven't actually done a full networked game, yet, but I've done basic object synchronization and messaging. I've been more working on the game engine side more ,
and I've been distracted by other projects.
But as I've been interested in networking I've also read about NAT quite a bit.
So I haven't actually tried this approach in practice, but from what I've read the best chance of success for NAT punchthrough (without needing a server for constantly relaying messages) is to have a server where the game hosts register and clients can then ask the relay to relay a message to the server to initiate the punchthrough. Then the client and the server use the same socket (so that the port is the same, this increases the chance of success as some firewalls are really strict) that they used to send a message to the relay server and send each other 10 UDP packets with 1 second intervals. I'm not 100% sure that this would also work with TCP connections, if both sides try to open a connection simultaneously, but it has a good chance to work. This way the firewalls on both sides should see incoming and outgoing packets to the same peer and allow the communication. This would be very light on server resources as it would only need to receive registrations (and optionally keepalive messages) from the game host and relay one request from a (verified) client to the server per connection attempt.
The approach of ForgeNetworkingRemastered from a quick look seemed that they won't use the same socket (or that they don't have to, not sure which one is the case as I didn't dig into the NAT client side) and basically the server is told that client from their specified port wants to connect (in a json message) and it then handles that somehow (perhaps by sending some sort of message to the client). At least from this quick look it seems that depending on the server implementation this may be the exact same approach.
The approach I described should work for basically any type of firewall (even ones that only allow return packets from the exact ip and port combination) to establish connection between two clients. Of course this needs some additional tweaking if all clients need to communicate with each other and some of them are behind the same NAT so that they see each others' private address instead of the public address, but this shouldn't be impossible to get working.
-
OK, so some random musings on this from a netops guy who's done more than his share of raging at poorly implemented network multiplayer systems, with the disclaimer that I've not looked very far beyond Forge's public API documentation (which conveniently leaves out a lot of the actually useful info on what it's actually doing):
There is no single silver bullet that will make this work, period. Luckily, you appear to already understand that given the content of your post. There are however a handful of things you can do to make this work in almost all cases and make it easy for users to fix when it doesn't:
- STUN (https://en.wikipedia.org/wiki/STUN (https://en.wikipedia.org/wiki/STUN)) and TURN (https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT (https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT)) will cover a vast majority of cases. They work pretty much like you described in your explanation of NAT hole-punching. Most likely, Forge uses some variant of these.
- Provide a way for the user to specify an arbitrary, fixed port to use for the listening socket. This sounds stupid and rather useless to a lot of people at first,
until you realize that a lot of consumer routers make you jump through hoops to expose ports externally, and it's often easier to just map one and do all your multiplayer stuff through that regardless of game. Minecraft is actually a great example of this being done right, when you go to host something locally, you can manually specify a port to host it on. This bit is largely independent of the networking layer you use under the hood. - UPnP and NAT-PMP will help cover a vast majority of cases where STUN and TURN do not work. While those of us who are tech savvy and care about security over convenience turn these functions off on our routers, most people don't, and supporting them will make a direct connection just work in a vast majority of cases. I'm not certain from looking at the Forge documentation whether they have this or not.
- Please, for the love of all things holy, allow local multiplayer. This will allow people to reduce the load on any kind of central server, allow multiplayer even if you eventually have to shut down the central server, and probably most significantly, let people have real LAN parties with the game.
- Make the central server component (relaying if you're going to actually do that, or just discovery if you won't do relaying) available for regular users to run. This serves two purposes, first you remove the absolute dependency on you hosting a server for multiplayer, and second you open up the option of a community supported relay pool (which will save you money). Take a look at https://syncthing.net/ (https://syncthing.net/) for an example of this particular aspect done right, pretty much their entire relay network is community hosted, and they provide an easy option for users to run their own private relays.
- Don't use Hamachi as a model for how to do things under the hood. They did a great job on the end-user experience, but did a lot of things below that wonderful UX that made those of us who have to handle actual network infrastructure hate them with a fiery passion. In particular, do not try to pull crap like they did for indirect relaying (they essentially hijacked a publicly routable but infrequently used block of IPv4 addresses that they did not own).
- Make sure you have proper IPv6 support. Like UPnP and NAT-PMP, this will get you a lot closer to things just working in most circumstances (if two players are on real IPv6 networks, they are likely to just need the central server for discovery and nothing more).
Now, if you're just going for the bare minimum to get NAT traversal working, then I would suggest specifically focusing on something similar to STUN and/or TURN, plus UPnP and NAT-PMP and IPv6 support. Combined, that should make things work in at least 90% of cases, and in most of the cases it doesn't it should just require a trivial manual port-mapping on the user's NAT router.
-
Holy smokes, this is more complex than I'd even expected. We're really not network programmers in that particular depth. We also don't have any central servers that would be good for this, at least not at the moment. This has to be a relatively common problem to the point that just doing the STUN-style initial brokering of a connection is available in some sort of organized middleware?
I believe we're using just a single port (UDP) with Forge, but I don't know for sure; we were with Lidgren, which was my project to implement. But Keith implemented FORGE and customized it a bunch when he did, so I don't know where we stand on that precisely. And yes, definitely local multiplayer is always something that we prefer above all else!
The sort of work that you're describing definitely sounds out of scope for what we currently have on our plate in terms of time and funding, so I'd be very interested in anything that can help us cut those hours down. Whether that's some form of middleware or hosted server system for this, or whether that's someone willing to donate some time showing us how it's done from a programming level.
-
As long as the option to swap port (for those who know how to setup a router), UPnP (for those who don't) and direct connect by IP exists, you don't really have to bother too much with this, just add the port, local IP and outwards IP to the hosting screen (in a copyable fashion) and most people should be able to host and connect with that information.
I'm not really seeing AI War as a game where you just pick a random game in a list and play for hours on end with randoms, and while I do realise there is a demand for that kind of thing too, that's where communities and other social interactables comes into the picture.
-
With the original AI War, we had the ability to choose your port, but it was set to something specific to start with. I'm not sure what you have in mind with UPnP; I've not heard that term used in this specific context before. Being able to get their local IP or their global IP is definitely something I want to put in directly, which we didn't have previously. The one problem with local IPs is that usually people have multiple adapters these days, and those can give strange results.
People have infinitely been asking for a global place you can go play with random people, but I'm not inclined to do a lobby like that. It would perpetually be a ghost town, because people are either already playing or not, for the most part; the matches are too long. But for people who don't know how to set up their router, the ability for them to connect to a friend without having to set up their router is really all I'm looking for. Generally my understanding is that that's NAT punchthrough -- but is there something related to UPnP that serves a similar function...?
-
I believe we're using just a single port (UDP) with Forge, but I don't know for sure; we were with Lidgren, which was my project to implement. But Keith implemented FORGE and customized it a bunch when he did, so I don't know where we stand on that precisely.
Single UDP port :)
-
I have a lot of experience coding TCP sockets on linux and windows for 20+ years in C++/Java/C#. Never really needed UDP.
Forge seems pretty standard. You can send a binary frame. The connection stuff looks the same, and "Setting the correct NAT host in the Connect method of the ServerUDP and ClientUDP is all you will need to do in order to enable NAT hole punching in your application." So if the hardware is setup properly, it appears you can use this library like a normal socket library. Just have to figure out the specific of server and client specification. I am at work, so I cannot watch the NAT Hole Punching specific video, and they did not list the code.
With that said, looking briefly at the Forge documentation, I am not sure NetworkObject will handle the data load given the immense number of ships this game has without serious performance issues. Considering AI War has thousands of ships, and the network object seems to be used to sync between clients, including RPC calls for things like exploding, I think the first step is to forget the hardware and NAT Hole Punching, and attempt a proof of concept project.
Check out how it performs with all the bells and whistles in the best situation, on a Lan. Then introduce the NAT hole punching, then full remote. If you find an issue with that, then you are going to have to go full binary or a mixture of the two.
These are my thoughts with 15-20 minutes looking at the docs, so take them with a grain of salt. ;)
-
You can send a binary frame.
Yep, that's what I'm doing. As raw as I could get it.
We're not using any network-object synchronization. Sync is achieved by the simulations running identically and deterministically on each machine independently of each other, and everybody being sent a copy of each player's commands along with instructions on when to apply each command (so they all do it at the same point in the simulation).
-
Keith, it appears I have thrown you into a thread with a bunch of people who know a lot more about this than I do. Hopefully this yields something constructive, and in the meantime I'll yield the thread to you. ;) Back to the GPU I scurry!
-
As long as the option to swap port (for those who know how to setup a router)
Already in settings, but can also expose on MP screen.
UPnP (for those who don't)
How do we enable this?
and direct connect by IP
That's already in.
just add the port, local IP
Easily done.
and outwards IP
Would a quick request to icanhazip.com, sanitization of the result, and output of same suffice?
to the hosting screen (in a copyable fashion)
Copy the whole chunk, or just the outwards IP, or what?
Thanks very much for the advice :)
-
Sorry I haven't been around for a while college and life suddenly got serious.
In my work and in my classes I've had the opportunity to become familiar with some networking, but I'm not exactly sure how to do NAT punch-through myself. Instead I've been working using websockets, which act like a file you just write to another computer somewhere out on the web. Websockets literally handle all of that for me.
Is this a possible protocol to use in AI War 2, or are you guys still using REST or SOAP?
Implementing websockets for you would be a little bit tough for me right now, as I'm mostly using Jetty or TomCat to perform them. Jetty is the Eclipse native libarary for Java, and TomCat is the Java native library. I don't know the library I would use in C#.
One of the features I believe that Websockets takes advantage of is IPv6 addressing, which would eliminate the entire need for NAT punch-through. After all, if we can address every atom in our galaxy, what's the chances that 2 computers are going to have the same address, right?
-
We can't use REST, SOAP, or even WebSockets -- our data has to transfer far faster than that. Basically we need data to go out and come back within about 100-200ms at most, and none of those protocols are set up for doing that. We also need the data to go as leanly as possible, because the more packets that are sent, the more you get data loss and resends, etc.
So it has to be a protocol built above UDP (because TCP stinks), where it keeps messages even if they are out of order or one gets dropped, and just requests the missing ones, etc. FORGE and Lidgren both basically provide a framework like that, as do things like UNET and most other game-based systems. TCP is a good 4x to 8x to slow at minimum, and the more packet loss there is, the worse it gets, as well as the larger the data, the worse it gets.
But it was a thought! Just letting you know the parameters of where we are.
-
Well, I'm familiar with the concept of UDP, but not any UDP protocols. Thought websockets was leaner than that though, as I said, it's like writing to a file. It's you who determines what information goes into the file. Other than the headers on the packets of course.
What about my suggestion of IPv6? Is that possible over UDP? If so then you are addressing computers directly behind routers.
-
Well, in terms of IPv6, support for that is somewhat implicit in the modern libraries (to my knowledge), but we don't get to decide how clients are set up. I wish! ;) Clients and hosts have their IPs set by their ISPs. We're talking about your computer talking to your friend's computer. I don't know what your setup is, or who you are, or what year it is, and I certainly don't get to touch your configuration. This isn't a server I'm setting up, or new drivers I'm installing on your machine, etc.
UDP and TCP are basically just the two lowest-level protocols that you can get to. Websockets and all the rest of those are implementations on top of TCP, IIRC. There's a whole stack of stuff that has to be stripped away for games to work, basically.
-
I came by to contribute to this, but noticed that ahferroin7 already mentioned many of the things I was going to say, especially as regards UPnP and NAT-PMP. Download the source code to a BitTorrent client (e.g., Transmission) to see practical and extensive use of these protocols. Of course, I have to imagine there are almost certainly C# libraries that you can use in Unity for these purposes.
That said, I also once built my own "MRUDP" protocol on top of UDP in the past; a "More Reliable UDP." This was back in the 90s for streaming financial data over (extraordinarily slow) CDPD links to mobile devices more reliably (in the face of packet loss and latency) than TCP could do. There have been numerous published protocols as well in the interim, including Google's recent QUIC. Here's an article from some years ago which provides an overview of several. Targeted searching will of course reveal others. http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/Reliable-UDP-(RUDP)-The-Next-Big-Streaming-Protocol-85316.aspx
Here's one that targets Mono/.NET and also includes setting up holes in NAT routers: https://github.com/RevenantX/LiteNetLib
Anyway, now that you know what to look for, I'm sure you will find lots of suitable things without having to reinvent this wheel.
Cheers,
Doug
-
You can send a binary frame.
Yep, that's what I'm doing. As raw as I could get it.
We're not using any network-object synchronization. Sync is achieved by the simulations running identically and deterministically on each machine independently of each other, and everybody being sent a copy of each player's commands along with instructions on when to apply each command (so they all do it at the same point in the simulation).
Excellent choice. Layering the data I see it like this
Game Data -> Raw Data -> Send over Socket -> Receive Raw Data -> Unpack/Decipher -> Update UI in multithreaded manner
Then you have to consider fault tolerance. You can't expect every packet to reach destination. But some data must always reach destination like commands to move object to x, y or attack, etc... This is a difficult problem.
Are you going to have the host PC be the hub and control everything? Like client user interacts with object, then command is sent to host PC, then game data is updated by host and sent out to everyone? I see you answered this somewhat in quote, but seems like there could be situations where client is acting on objects that do not exist.
From what I googled, I don't think you have to worry about big endian vs little endian, and using host/network byte order.
I think you have a good grasp of all this already, and your question or request for help wasn't really in this area :)
-
We're not using any network-object synchronization. Sync is achieved by the simulations running identically and deterministically on each machine independently of each other, and everybody being sent a copy of each player's commands along with instructions on when to apply each command (so they all do it at the same point in the simulation).
As I recall, this is how AI War (the First) worked.
Cheers,
Doug
-
Well, I'm currently learning database access at work through JDBC and TomCat, and likely will be learning antagonistic neural networking for my Senior Design project. I'd love to learn how the low level stuff actually works from a programmatic standpoint, and unfortunately I can't learn from either TomCat or Jetty because they don't provide the source code for me to peruse. It's endlessly infuriating. :(
-
Doug! Always great to see you. :D
Here's one that targets Mono/.NET and also includes setting up holes in NAT routers: https://github.com/RevenantX/LiteNetLib
Well... holy smokes on this one. This one is basically implementing everything we'd need, from the looks of it. It's very much like what I recall of Lidgren, except this looks a bit more evolved (to say the least). I'm quite happy to see the multicasting on there for LAN discovery. Dual mode for IPV6 is also of course very nice, since that's likely to (any year now, sigh) become more common. Basically running in "reliable mode" over UDP and then passing data of our own choosing is what we're doing. To some extent, that makes the network library itself a bit interchangeable.
Keith -- what might be nice is to migrate all the FORGE stuff and our networking code that goes with it out into the open-source moddable part of our code. All of that is open source now, anyhow. Depending on how similar the system above is, perhaps you or someone else could program that in as an option.
What might ultimately be a nice thing is a bit of an abstraction layer where we can swap out the underlying networking layer by just having it implement various interfaces or similar. That way in theory we could have a FORGE implementation and a LiteNetLib implementation, and the user can switch between them. Not useful in the long-term, probably, but in the next few months it would allow for some A:B performance testing in an easy fashion. Lidgren is also open source and has evolved in the past years, so it might also work as a third option to check on.
I imagine we have some folks here who would happily profile the transport layer for us if we had some options on that, while the actual "what we're sending via the transport layer" stays in its own logic area.
Basically kind of like how we can easily switch between DirectX and OpenGL and Vulkan and nobody notices a difference in actual functionality. Those in turn are running on top of variable drivers, etc. In our case, FORGE or LiteNetLib or Lidgren would be analogous to DirectX or OpenGL. The data we're passing in and getting out should be identical in all the cases, and these libraries tend to have a pretty similar setup for the bare-metal type approach that we take to things.
@Qoojo: Yep, endianness doesn't matter to us, partly because we're all in managed code and that abstracts that bit away from us. And yes, the host is the global manager for all things, and the clients talk to the host, which is also a client of itself. Clients request for things to happen, and then the host says "do this in future simulation step X" to all of them, and it happens on that step.
All of that logic is well above the transport layer, and pretty transport-layer agnostic. Even how Keith is serializing those commands into bytes is transport-layer-agnostic, because he has his own way of encoding things that is a lot more efficient than ascii, or even something like just raw int32s. We did a writeup with some table comparisons in an earlier design doc, and it's referred to as "BuzzSawBinary." It's actually smaller than gzip compressed data in a lot of cases, surprisingly.
Our problems are solely of the "hey I need to talk to you and I need you to talk back to me" variety with clients and hosts, not about knowing what to say after that's established. For customers, one of the big problems was always "who am I? who are you? why is there barbed wire between us??" ;)
-
We're not using any network-object synchronization. Sync is achieved by the simulations running identically and deterministically on each machine independently of each other, and everybody being sent a copy of each player's commands along with instructions on when to apply each command (so they all do it at the same point in the simulation).
As I recall, this is how AI War (the First) worked.
Yep! And our current networking already works fine, just like that, but with a different transport layer library. The NAT punchthrough and potentially multicasting are the main hiccups.
My original implementation was based on this paper: https://www.scribd.com/document/43559444/1500-archers-on-28-8k
The same principles are in place in most every RTS game with any substantial number of units to this day. Action games are a whole other beast, and vastly more complicated.
-
Well, the cool thing is that the AI is all open-source, so anyone is free to add this sort of thing as a mod. And if it works well and people prefer it, we can bake it right on into the official builds. As an AI type or otherwise.
As it stands, right now our priority is polishing what we have and reducing our workload, not increasing it if we can avoid it. We're already massively overbudget and over schedule, and we'd like for this to release in as pristine a state as we can. Keeping away from scope creep at this point is the big lesson for us in terms of trying for any kind of cleanness at 1.0.
But the good news is that we've built an overall engine that is so darn flexible that we can start contemplating things like this post-1.0 if the game is selling well enough; and if it's not, modders and similar can contemplate it if they would like to.
Well that is definately interesting. It would only be part of the AI, and I think it'd be cool to have it learn your habits and how you play, and surprise you as you play.
Only problem is I found out my computer is far too old to run Tensor Flow on writing recognition. It ends up exiting out while training with no message as to why. I expect I'm missing some instructions needed for that build of Tensor Flow.
Maybe I'll get on it if I take an AI course next Fall, or if I get the project I want in Senior Design.
-
@Qoojo: Yep, endianness doesn't matter to us, partly because we're all in managed code and that abstracts that bit away from us. And yes, the host is the global manager for all things, and the clients talk to the host, which is also a client of itself. Clients request for things to happen, and then the host says "do this in future simulation step X" to all of them, and it happens on that step.
All of that logic is well above the transport layer, and pretty transport-layer agnostic. Even how Keith is serializing those commands into bytes is transport-layer-agnostic, because he has his own way of encoding things that is a lot more efficient than ascii, or even something like just raw int32s. We did a writeup with some table comparisons in an earlier design doc, and it's referred to as "BuzzSawBinary." It's actually smaller than gzip compressed data in a lot of cases, surprisingly.
Our problems are solely of the "hey I need to talk to you and I need you to talk back to me" variety with clients and hosts, not about knowing what to say after that's established. For customers, one of the big problems was always "who am I? who are you? why is there barbed wire between us??" ;)
Oh nice. Yea, if you don't repeat data, then compression algorithms can have a difficult time. ;)
Looks like you have the barbed wire solved with these libraries.
-
My original implementation was based on this paper: https://www.scribd.com/document/43559444/1500-archers-on-28-8k
I guess we know the min specs now, "Support a target platform of: 16Mb Pentium 90 with a 28.8 modem".
I will over that link later. Interesting stuff.
-
I can't learn from either TomCat or Jetty because they don't provide the source code for me to peruse.
Tomcat has been an Apache project for... decades, and open source: https://github.com/apache/tomcat . I used it for the Space Empires IV Play By Web in the late '90s.
(https://web.archive.org/web/20010626023644fw_/http://seiv.pbw.cc/)
Likewise, Jetty is also an open source project. I use it to this day (as part of my Clojure web stack). Code: https://github.com/eclipse/jetty.project
Enjoy,
Doug
-
I can't learn from either TomCat or Jetty because they don't provide the source code for me to peruse.
Tomcat has been an Apache project for... decades, and open source: https://github.com/apache/tomcat . I used it for the Space Empires IV Play By Web in the late '90s.
(https://web.archive.org/web/20010626023644fw_/http://seiv.pbw.cc/)
Likewise, Jetty is also an open source project. I use it to this day (as part of my Clojure web stack). Code: https://github.com/eclipse/jetty.project
Enjoy,
Doug
Thanks, but when I have maven pull in the project, it doesn't have the source files with it, which won't let me F3 through them in Eclipse.
Maybe if I find time to download the binaries and then tell Eclipse where to find the source.
Also, I swore TomCat was Oracle, as every time i'm getting the Oracle license agreement splash like on every other Java file.
-
My original implementation was based on this paper: https://www.scribd.com/document/43559444/1500-archers-on-28-8k
I guess we know the min specs now, "Support a target platform of: 16Mb Pentium 90 with a 28.8 modem".
I will over that link later. Interesting stuff.
Ahaha. That was Age of Empires 1 -- we're a bit beyond that, at this point. :D