DWM: I've got a lot of ideas swimming in my head from the many articles I've been browsing on game development. A lot of it is influenced by some of the more advanced ideas on distributed worlds. I really don't know how to put it together into a concrete architecture yet, so let me try to describe some technical scenarios that illustrate some of the elements. This will be kind of rambling, because I'll go off in detail on certain sidepaths that interest me, then return to the scenario.
The basic network environment & initial connection
First, the basic environment. I'll assume for now that code running on player's machines can't be trusted -- maybe we've open-sourced the code, which means that the only things you can rely on are the things you can deduce from the network traffic. Assume that SpookyDistance runs a network of trusted game servers. They use some sort of public/private key authentication. As a player, you fire up your client program. It contacts one of our servers, sends a challenge using our public key, and the server verifies for you that it is in fact a SpookyDistance game server. You now trust this server to simulate the game accurately, because you trust SpookyDistance not to cheat. (Using this method, given access to the code, other people could set up their own server networks with their own keys and reputation. Just a thought. JDH: See /OtherServerNetworks)
The game servers keep a large database of objects and their capabilities. Only trusted game servers have the ability to create object and modify their caps. Your client program first authenticates itself with a game server (password-based), and requests control of your player's persona (but not the ship, yet). Ther server finds out which node currently owns the persona, gets a lease on control of it, and sub-lets control to your client. This is a basic aspect of the distributed nature of the game; at any given time, any simulated object is owned by one node. When a player is not connected, the persona owner will most likely be a trusted game server. If nobody is currently concerned with actively simulating an object, it still needs to be "parked" somewhere - maybe at a central top-level server with the definitive database of objects, assuming a hierarchy of servers. Perhaps there are many independent hierarchies for different categories of objects, such as personas, physical objects in space, organization entities (corporations, governments).
Your client will also tell the server something about its node characteristics; most importantly, probably, an estimate of processing power and network connection performance. This information gets used later to distribute control of objects intelligently.
The player gets ready
Let's assume that you start in-station. This is a totally different part of the game from the 3D spaceflight simulation. I won't go into detail on this part of the game, but if you use your imagination, many of the same concepts might apply here as with the physical simulation, which I'll go into in more detai. Some of the ideas will be easier to understand when applied to physical objects.
Let's further assume that you're a relative newbie, and you don't own your own ship yet. So you look up pilot contract work being offered at the station, pick one, and get a ship assigned to you. you do some planning: studying the ship's stats, the cargo, the destinations, etc. Maybe you file a flight plan, contract some mercenary escorts, or pay protection money to a racket contact so you can safely take a shortcut through a system that's controlled by an arm of the Mafia. (Maybe you contract specifies a bonus for swift completion, or you may just want to move on to the next contract as quickly as possible.) Maybe you do a little research on the NPC crew that comes with the ship (or was assigned to you, or maybe you have to find them yourself) to make sure there are no suspicious characters there. If the roster includes that guy you suspect of sleeping with your (simulated?) wife, you replace him from personnel available on the station. Whatever. When you're ready to go, you request permission to launch.
"We have ignition"
At this point, your client asks the server for ownership of the ship. As before for the persona, ownership is leased to the client.
From here on, I'm going to assume that we're using a distributed model in which each client simulates the environment of interest to it, and gets occasional state updates from nodes that have ownership of the objects being simulated. Our first problem is to initialize this environment.
Based on the capabilities of the ship's sensors, your client builds a list of "Interests" and registers them with the game server. (Since ship caps cannot be changed without the server network's knowledge, perhaps it asks the server to build the list itself by just telling it what sensor systems are online.) The server may not have control over all the objects of interest; but it will know about ones that it is controlling, and about ones that it leased to child nodes (because it will also be simulating these, even if it's not controlling them). It may have to pass the interests up the hierarchy, possibly causing additional objects to be activated from storage. Whenever a node determines that it has control of an object that matches an Interest, it starts sending state updates for those objects to the node owning that Interest. Based on the client's node characteristics and degree of interest (like, for instance, asteroid GX1000 is ten meters off my bow, and nobody else is within a thousand kilometers of it), the network may rearrange leases to push control of some objects to your client. (Do the clients participate in these decisions, or are they controlled by the servers?)
In general, each node will be simulating an area of space. Each node will hold a list of Interests owned by nodes that are concerned with nearby or overlapping regions of space, and will send state updates for matching objects to the Interest owners. How refined are the Interest queries? I'm not sure. Perhaps we ignore sensor caps and have queries like "all objects within 4000 kilometers of the position of object static object PLANET20875". However, if the queries are this broad, then there's a hacker vulnerability, because a client will be getting information about lots of things that he shouldn't actually be able to detect. OTOH, more specific queries like "all objects detectable by a class 3 electromagnetic sensor at location of SHIP3567" requires the Interest responder to simulate the ship's position, and generally do the work associated with determining sensibility of an object for the client, so that it never sends anything that client shouldn't know about. (This could also be sensitive to level of detail vs. distance, e.g. the client may receive position information about an enemy ship that is far away, but no details -- it's just a blip on the radar.) Also, there's always the possibility of a specific query that returns objects that you need to know about due to collision possibilities, yet you should be unable to sense (because, e.g., your sensor systems are damaged). I'm not sure how to deal with these security issues in an open source environment.
The initial information the client gets for each object of interest may include capabilities (or perhaps these always come from the authoratative server network), rendering information, and ... script code! In order to avoid having to update client (and server) software every time new object types are introduced, it must be possible to update incrementally, which in turn is most easily (and probably most compactly) done using a compiled, intermediate representation of script code. This would be most important for objects that are being controlled by the local node, but may also play a role for non-controlled simulated objects.
As your ship flies around, some objects become interesting that weren't before, and your client may be notified of new objects of interest. Other objects stop being interesting, and you drop them. Certainly, after a "jump" (or squirt, or swallow, or whatever), your whole relationship with other network nodes might need to be completely renegotiated.
Ongoing simulation and dead reckoning
While your ship is flying in "normal space", all physical objects of interest are being simulated based on last-known kinetic and behavioral information (scripts again -- think for instance of fire-and-forget missiles), and user input is modifying the controlled ship's behavior. Your client node has a list of Interests from other nodes that it must attend to, comparing its controlled objects to the queries.
Occasionally, your client will receive updates from other nodes that have your Interests at heart, er, I mean registered. These will be time-stamped, and will provide corrections to the kinematics of one of your non-owned objects. For instance, another player ship is flying nearby. You're simulating its behavior as best we can, but in the meantime the other player yanks his joystick (ahem) and noses over towards you for a missile run. When the client gets an update, it has to correct the position of the player's ship, which no longer matches what was expected. In the simplest case, "warping" occurs, and hopefully it won't be large enough to be too disturbing. Worst case, the simulator might need to be able to make the correction look more smooth. Only testing will tell if this is necessary or feasible.
In the meantime, your client is similarly sending out updates of its owned objects state. How does it know when to send them? Well, it knows when it last sent out an update, and it calculates the behavior of each controlled object based on that, in addition to calculating its actual state (which is affected by user input, or by the response of behavioral script code to the environment). When the two states diverge by more than a predetermined error tolerance, the client sends an update to all interested nodes. So instead of sending updates on a heartbeat, we only send updates when we think they're needed.
This technique is called [dead reckoning], and it's pretty frugal of network bandwidth, but puts a greater processing load on each node than more traditional techniques.
To be continued... resolving collisions and other important events