Ok, the resource system needs to be something where new resources can be added with a very minimum amount of work. It also needs to be flexible, since there's no point in adding resources if the resource system itself is going to restrict what types of resources can exist in the first place. We will hit a soft upper bound on useful resource types, hopefully before Bacchus gets out the door. But that won't mean we'll be done creating resource types.
Resources should be able to exist in script. So when we add scripting, we need it to have access to resources, and it needs to be able to create new resources.
A resource consists of several things. There are binary files of arbitrary format, some of which we'll support on our own, some of which we'll expect people that really want them to write scripts to support them. Several examples of binary files are jpg, png, ogg, mpg, wav, flac, tiff. The list goes on.
So we have these xml files that provide all the meta-data needed for the resource system to manage them. That's part of the resource.
In order to give the resource system the lowest overhead for people to use it, it needs to have a simple interface to the rest of the game. I'm envisioning something where you do tResource* theResource = tResourceManager::getResource(myresource); and you get the xml file loaded into memory, you get the binary files all loaded that can be loaded (streamed files are an exception by their nature). Basically after this line, you can send textures from the resource to openGL for rendering with no intermediate steps required.
To support such a simple interface to the rest of the game, the resource system needs to be able to load any arbitrary binary file, and it needs to be able to do it in a generic manner. So you can't have, for example, LoadTiff, LoadJpg, LoadOgg, and so forth for methods. You need one method with one return type that is capable of loading any arbitrary binary file. Obviously that return type will have to be a class pointer, this is a problem solved in the cockpit already that can be copied into the resource system.
Normally, when you need one method with one return type in a bunch of classes you use inheritance to get that method there. So for gMap and gCockpitParser and gTheOthers they'll need to subclass tResource in some way or other, and tResource contains the LoadFile method (or whatever it winds up being called). So what does this LoadFile do?
The simplest way for LoadFile to load any arbitrary format is for it to have a list of sorts, and it takes the file its given (needs a string, not a file pointer, to do it right) and scans its list of loaders to determine which one to use. If none is found, it throws an exception. The loaders themselves should be marked what they're loading for the game and what the format is, for example (texture, tiff), (music, mp3). So if it sees, say, (texture, mp3), it can say "I don't have a loader that loads mp3s as textures". It might say "I have an mp3 loader, but it thinks mp3 is music and I can't use it". The loader itself needs to verify that the file is the right kind and the match has been made and so check the integrity of the file. If it fails, it also needs to throw an exception.
By making this neato LoadFile method work like this, it's possible to implement a new resource with 3 steps:
1. Create the xml file and subclass tXmlParser
2. Create special binary file loaders
3. Register binary file loaders *and* the new subclass created in step 1
Some of us already know from experience that step 1 is hard enough as it is and would like it to be simpler. How simple it will be depends on how much we load up the base classes and how useful the stuff in the base classes will be. But step 1 involves creating a dtd, which by itself isn't hard. The hard part about making a dtd is designing the xml format in the first place.
Step 2 is actually very simple. Just write a function that loads a file and returns a pointer to the data. For known media types, like textures, sound effects, and music, classes that already exist can be used to store the data. The reason for this is simple. Textures will be stored in a known format in memory, and it will be a format that is supported by OpenGL. So the class that contains a texture will need to hold all the information that describes how the data is stored and the data itself, and all texture loaders will be responsible for filling it in. Music and sound are very similar in that regard and so we'll have a strong base class to handle it. I think audio data is probably the easiest format for us to standardize internally. So you're writing a function to a spec, that's all.
Step 3 is the easiest step of all, but may be the one that's hardest to understand to a new guy that doesn't know what he's doing. Documentation is the way to address that.
So after all this, we need tResourceManager to provide a single API that allows the rest of the game to retrieve, create, sort, delete, and do other things to resources. (Mind you, tResourceManager might ultimately find a vacation home as a shared object in a python library for other applications to use in resource creation) tResource needs to provide an interface that makes it possible for tResourceManager to do everything it needs to do, which is moving files around, expiring its cache, and so forth. The subclasses of tResource can extend the interface for their specific components, and they should do so. There's no reason gCockpit should be using a gMap to set itself up, after all. So gCockpitParser would have stuff in its interface that gMap doesn't have, and vice versa. But tResourceManager doesn't know about them, nor does it care for its work. (Except that it needs to be able to create instances of the subclass to give when asked, but it can do that and then dynamic_cast them for internal use. The cockpit/arena/whtever will have to dynamic_cast it back to the subclass for its own use)
That's some part of my thinking behind all of this. There is probably a fair amount of overlap with what wrtlprnft is saying, I don't really know.
I'm hving a little trouble of my own parsing his first post, not sure why. Some of the details being tossed around aren't fundamentally important to the overall design I'm offering. Yeah, they need to be done one way or another, but how they're done won't affect any of the rest of this stuff.
For example, I really like the elegance of wrtl's <Files> section as far as the code goes. But as an xml author I'd rather not have to do that, I've dealt with writing xml files for vcdimager by hand, which use that convention, and it's a pain. But either way it doesn't affect the design I'm talking about, and I'll take it either way or both because I don't care *that* much.
Also, I object to tResourceManager having its primary access to a file a method that returns a file pointer. The caller needs to know more about the file than a file pointer can give it. This is actually important to the design, and expecting the primary file accessor to get file pointers instead of opening the file itself will significantly restrict the design of the system as a whole.
The next topic is how a resource will be used. So how will it be used? Right now, it's done like this:
Code: Select all
FILE *fp = tResourceManager::getResource(somefilepath);
parser = new gCockpitParser(fp);
parser->Parse();
... or some similar idiom. After this, all you've done is load the xml file and parse it. You haven't loaded any binary files it depends on, like textures and the like. You still have to do that.
I don't like this idiom at all. I think it should be more like this:
Code: Select all
gCockpitParser* theCockpit;
theCockpit = dynamic_cast<gCockpitParser*> tResourceManager::getResource(someresourceindex);
After this piece of code, all binary files for the resource are loaded and can be retrieved through the tResource interface. Actual data structures specific to the cockpit would be retrieved through the interface implemented in gCockpitParser, but they should be initialized already. Maybe that isn't possible and we'll need a third line here to call an initializer, but that's fine. It's still a lot simpler than the first way, which is the way it's done now.
Then there's managing the resource system itself. At various strategic places in the code, more than likely outside the game loop, tResourceManager will need to be called to do things, such as "install this resource", or "expire your cache", or "clear all of your cache". Some of these things will be in response to user input, some of them will be things that are done automatically. Some of them will be in response to command line flags that are designed to allow the game to manage resources in a cronjob. There's a good list of things that have to be done, I'm sure we can brainstorm a pretty complete list, so I'm sure y'all can each come up with at least 1 thing that tResourceManager needs to do that fall under the heading of "resource management".
Finally, it should be obvious at each step where scripting hooks will exist. tResourceManager will need to be available to scripts, scripts should be able to provide file loaders, subclass tXmlResource to make new resource types, and the interfaces created in scripts should be available somehow to the rest of the game to use. tResource's interface, at a bare minimum, must be available to scripts. Like I say, where scripts hook in should be pretty obvious.
See why I tried to keep my first post on the subject short?
I've got more where this came from, too. Should I move this to the wiki now? We can keep discussing it here and then modifying the document on the wiki. Is this a vision we can all share to at least the extent that I can turn it into a working draft design document instead of a forum post?
