Technology
SDxWiki

Basic Development Tools

DWM Here's my proposal. Anyone reading along?

Guidelines

Rationale

I think it's likely that we will eventually want to run server software on Linux, which continues to have a major presence on Internet servers. Having worked on platform ports before, I want to avoid having to deal with one down the road. Instead, let's make sure from the start that server software will build on that platform.

I realize that not everyone wants to bother learning a new toolset right now, so for client-only software, we could certainly use MSVC. Personally, I'd be just as happy using open source tools for the client software. But if the people with an interest in the front-end work prefer MSVC, I can deal with it.

I would be really pleased, though, if we could avoid VC6 in favor of VC7. I tend to use a lot of advanced C++ techniques when building libraries, and VC6 definitely falls frustratingly short.

Consequences

There are a lot of subsystems that need to run on both the server and the client -- physics modelling and networking come to mind immediately, for instance. We will probably use cross-platform libraries for common services in order to target the code for both environments -- for example, there are cross-platform multithreading libraries. As a consequence, even the front-end code will need to work with some of these libraries in order to interoperate with lower-level SDx components, foregoing direct access to Win32 APIs sometimes.

For the time being, each developer will have to supply his own tools. This may pose some dilemmas with respect to commercial tools like MSVC, which ain't cheap if it has to be bought. Eventually, it will have to be bought, regardless of what cheaper arrangements you might be able to make in the short term, and probably before SDx has any income. Or, the software will need to be ported to a more affordable platform.

Suggestions

Client

Consider avoiding commercial tools like MSVC. Many critical low-level Microsoft components, like DX8, are available for free, or alternatives are available.

Possible compiler toolsets:

Possible IDEs:

If DevStudio/MSVC is desired, someone should look into licensing issues.

If MSVC is used, front-end developers may want to avoid becoming dependent on components that are specific to the DevStudio environment, like the MFC. (BTW, the entire MFC is a really bad example of C++ API design.) There are a variety of cross-platform UI libraries available that might be of interest. If you choose to stick with Microsoft, then at least look at the ATL as an alternative to MFC.

Server

For the Linux builds, choose a single common distribution, rather than trying to be portable across multiple distributions. An eventual port to additional Linux distributions shouldn't be too difficult. Use the most recent stable and commonly-available version of GCC and related tools.

Both

Scripting Languages

JDH: What about scripting technology?

DWM: Well, I was going to start a list of possible criteria, but they're all the kinds of things you'd normally look for in a language. I guess the main thing is that there be no visible compile/link/run cycle, since you'd want to use this for rapid development of game content, and expose (part of) it to end users. It should also be reasonably easy to learn and use, again for the sake of end users.

The two favorites for new game development, as far as I can tell, are Lua and Python. (See links on Interesting Links page.) I know that you're partial to Ruby, which I would guess is on par with Python in terms of size, overhead, and syntax. Lua has a slight edge in my mind because of its conciseness and extreme flexibility for syntax extensions. Inventing something from scratch would just seem silly in the current environment -- you could get a lot of leverage out of existing documentation, tutorials, examples, and libraries. (I imagine that Python probably has the edge over both Ruby and Lua here.)

Original Discussion

DirectX anyone?

Dan Muller: Took a quick look at DirectPlay, a component of DirectX. Interesting, it manages a lot of network conneciton issues for you. I'd love it if the game server were portable to Linux, but DirectPlay would make it easier to get something up and running.

Jim Horn: Hopefully, a lot of the JumpGate problems are caused by the programmers and not DirectPlay. If DPlay really performs that bad, we're in trouble...

John Harding: I thought DPlay sucked - or is the current version better? There are quite a few open source, cross platform libraries around - not that I care about anyone but myself and I guess I'll be running Windows 2100 when it comes out.

John Harding: I'm not that interested in low level graphics programming (it soon starts to make my head hurt). From a quick perusal of the DirectX family it's all fairly low-level (from my perspective anyhow).

Check out Crystal Space which sits on top of DirectX (when run on Win32): http://sourceforge.net/projects/crystal/ and http://crystal.sourceforge.net/docs/online/manual/cs_toc.html#SEC_Contents This seems like a fairly consistently active library (last time I had my "let's write a game" hat on it was also showing up high in sourceforge's activity stats) and it seems to offer a higher level of abstraction. For example check out the "physics" library: http://crystal.sourceforge.net/docs/online/manual/cs_140.html#SEC253

This seems like a decent level of abstraction to start with. I don't know what other libraries are out there but something to get us up and running sounds like a good idea - and I'm sure we want something that's open source.

Istvan: From a marketing standpoint it would be optimal if the server code runs on something reasonably stable, such as Linux or, potentially WinXP (jury still out on that?). However, the eventual public (and media) response might be fairly dramatic if there were multiplatform clients available (Linux, possibly even Mac). I might need to research if any of the MMOG development efforts out there are daring to touch the Mac market.

This page also looks like a good place to ask about the Development Environment. What does/will the preferred toolkit look like?

DWM Another one of those embarassing questions that we don't have an answer for. :) Not easy to answer, either. What part of the development environment are you asking about? Main programming language, physics engine, database system, ...?

I suppose it's very likely that we'll do most of our coding in C++ since all of us up here in Michigan are experienced C++ programmers. Although for AI work I was investigating a language called OZ, which is quite good for constraint-solving tasks, which is one way in which AI problems can be cast. The problem with Oz is that it's unfamiliar, it's a bit unusual compared to the imperative programming languages that we know and love, it's interpreted, and it can't (at the moment) be embedded in a program written in another language. However, you can embed C++ code in Oz programs, which might not be all bad. Oz has some interesting tools for managing distributed programming, so it might in fact make sense to have Oz at the top of the program. Worth investigating, anyway.

Istvan Question from a position of little knowledge: Would it be possible to make a system call from C++ to run the Oz interpreter at some regular interval, and thereby batch at least one level of Oz AI processing? This wouldn't be much good for very active AI agents, but it might be effective in allowing the use Oz for maintenance-level AI.

DWM Yes, if you run it as a separate, stand-alone process. That's an entirely reasonable approach. I'm all for using a language well-suited to each task, and with the goal of a distributed architecture, separating some tasks into separate programs should be feasible.

I expect that we'll custom-code the gross aspects of simulation object management. For a database package, I'd be partial to using PostgreSQL, which is free and is portable to UNIX and NT systems -- but not to Win9x systems. I'm hoping that client code, although it may participate in managing live objects, won't have to manage stored objects in a database, so this shouldn't be an issue. (Besides, by the time we share this with the wider world, Win9x should all have been replaced by at least WinXP, which is NT-class.)

I think there are Open Source libraries available to help with physics modelling, and we might look into those. For the main client-side game engine, there are lots of choices, ranging from using naked DirectX, through other commercial libraries with varying degrees of portability, to various Open Source libraries. I have a strong 'portability' ethic in my coding, but I fear that making client front-end code (graphics etc.) portable among different operating systems would probably be a nightmare unless we use a very good cross-platform library. In any case, I don't expect to be one of the primary front-end coders, so maybe someone else will provide opinions on that topic.

For network communications, I understand that there are libraries to help with this, too, but I'm inclined to think that we will end up doing much of this ourselves at a lower level in order to achieve our desires for anti-hacking security. As I understand it, intercepting the network data stream is one of the primary methods of hacking games. OTOH, we have some other ideas for hack prevention that might make this less important than I'm making out here.

These are all preliminary thoughts, though. Sketching an architecture and breaking the system down along gross lines should probably come first, then divvy up responsibilities, and let each subgroup recommend tools. In any case, cheap is good. Personally, I'm happy without spiff-o development environments, and I'm almost as comfy inside a Makefile as inside a C++ program. Since we probably won't be doing a whole lot of traditional UI programming, I doubt that environments like MS Visual Studio have all that much to offer us. Perhaps we can look elsewhere for compilers, and find less expensive alternatives.

Anyway, that's some rambling on the subject. To me, 'development environment' encompasses a large assemblage of special-purpose tools, usually added incrementally as needed.

DWM (Much later.) Both John and Frank have expressed an interest in CrystalSpace. I took a closer look at it, and it is indeed interesting, but does not appear to include a physics simulator. (That's OK, few of these libraries do.) I was also a little put off (emphasis on 'a little') by a few aspects of the programming style. The library uses a single monolothic header file with #define's to select subparts of it, a technique that I really dislike because my experiences with Windows and the Banyan Vines SDK show it to be fragile and to cause far too many unnecessary compilation dependencies. (Header files should be task-specific and idempotent.) The harping on the 'Shared Class Facility' seems unwarranted; it seems like MFC's CObject in its style and the way that the library wants it to permeate your entire application.

Those are some objections based on things that I understand (language, API design, and build management issues). Since I don't know a whole lot about the whole graphical rendering stuff, I can't really say that I understand the benefits of the library very well; they probably far outweigh these minor objections.

One thing that concerns me a bit with the free libraries in general is that none seem to make good use of C++'s power yet. There are advanced template-based techniques for complex math (like the kind involved in coordinate transformations and physics modelling) that have real benefit for expressiveness and performance which don't seem to have trickled into the consciousness of Open Source developers yet. They seem to be mostly relative newbies to C++. In fact, many libraries are written in C and have thin C++ wrappers to 'objectify' them, which is great for portability but really prevents exploitation of C++'s best features.

The Open Dynamics Engine 1 might be a good choice. It's LGPL'ed, so any changes we make to it will have to be made available to the public. This complicates code management a bit.