vValue and DCT

[Tank Program]

*wakes up after a small book*

[Z-Man]

If you're bored by the code specs, just read the usage examples

[wrtlprnft]

I love it.

One thing that's still bothering me is settings for resources.
I'd like to be able to have seperate setting subtrees for all resource files, but with some logic connecting them. So you could have a tree for all resources by “wrtlprnft”, a tree for all resources of the type “aacockpit” and a tree for all resources that are from “wrtlprnft”, of the type “aacockpit”, in the category “” and have the name “testfile”. Then all the settings would be merged into one tree for that resource. Is that possible?

[Z-Man]

I think this could be possible with the overlays. We'd have to allow overlays to have a whole set of fallback paths. Memory consumption may be a problem; overlays don't need to consume the same amount of memory than the full tree, but every treenode that gets touched (reading it is enough) in the overlay needs to be created and kept.

I don't think automatically creating the virtual resource trees for all cockpit files and all wrtl files would be a good idea. It would be better to do this in the resource file. Let it specify a set of nodes in the config tree that, for the purpose of this resource, should be virtual root. If they don't exist, they are created as overlays of the real root. Then, an overlay specifically for the resource is created, combining the other overlays.

[dlh]

Don't forget the old style configuration file format -> new style configuration format converter. Or maybe this will be one of the breaking changes for 1.0. Would be possible/easier to write the converter in Io, too, if we go that route.

[Lucifer]

Aha, thank you wrtlprnft for jogging my memory.

Didn't we talk about basically taking all of the current settings and turning them into resources? The idea was that every piece and part of a resource is part of one large whole, and the game assembles all the parts into that whole, filling in what it doesn't know with default values. So we wouldn't have a config system so much as we'd have something like a resource tree which contains all of the currently active resources. At the start of every round, we'd basically copy the tree to make the new grid and apply physics simulation to it, accept keyboard/network input, and render it. So what passes for a config system would be all the directives needed to load resources, and that's it. (Well, network configuration, machine-specific stuff)

[Z-Man]

I forgot in the code outline: the ConfigItemProxy constructor also takes the desired default value as an argument.

Nemo: I'd guess the old system would need to exist a while in parallel to the new system. At the very least, the setting change network messages need to stay compatible with 0.2.8 and older for now, and keeping the old system around will be the easiest way for that. We'd drop it in 1.0, of course.

Didn't we talk about basically taking all of the current settings and turning them into resources?

I don't remember any of that, maybe it was in IRC? This would make the config files XML files, and they'd look a bit like the config section of maps right now, right? If so, I don't think I'd like it. It's just too verbose. Config files should have little more format than "key = value". What does this approach give us?

A problem about config items defined in resources or script (I mean: a map where the zone size of the blue team is configurable via "blue zone size"): how do we detect and inform the user about errors in the configuration, especially setting name typos? By the time the configuration files are loaded (right at program start), the resources haven't been parsed yet and can't be parsed because the configuration determines which resources will be in use in the first place. Two possible ways out(Note: below is my thought process. You can skip directly to the long dashed line ---------, the stuff before it is just ramblings.):

1. Accept all configuration items in config files as valid, but mark them as preliminary; once a script or resource declares it's using an item with a given type and sees just a preliminary item, that item is transformed into a real config item of the right type. If nothing is there, a new config item is created with a default value. At program end, inform the user about all preliminary config items, those are potential typos. Problem here: the error report comes really late.

2. At the moment it is known a resource is about to get used (because a resource reference config item is set), it is parsed and the settings it declares are created. Afterwards, the user can use these settings like builtin settings in the configuration file.

Problem with both approaches: resources referencing other resources. Say we get really far and have a campaign consisting of missions which are a map that references meshes and materials and assorted scripts. During each run, most of that isn't even touched, it takes many game sessions to complete the campaign. So what should we do with the material that defines a config item? How do we know a unknown config item used in a configuration file is a typo or just used in a resource that was not loaded?

I think 1 with a modification would be the best approach. The modification, which is only sensible with a configuration tree: You only get informed about left preliminary config items in tree nodes that were actually used by resources. So if my config file sets "crazyMap zone color = 0xff00ff"and "wrtlMap zone sixe = 10" and I only load a map by wrtlprnft that uses the setting "wrtlMap zone size", I'll get a warning later (or maybe already after the map is loaded) about "wrtlMap zone sixe" probably being a typo, but nothing about crazyMap. Hrmm. Of course, if my config says "wrltMap zone size = 10", I'd get no warning. Or I get a warning about both crazyMap and wrltMap not being used. It's tricky

Ok, approach 3, getting us closer to what Lucifer said: the map (or other resource) specifies a config file to load. The config file is independent of the map. Only when the map is first parsed, the config file is loaded. So in the above example, the config file containing crazyMap settings wouldn't even be loaded, and there would be no problem. Moreover, the config file could be loaded AFTER the section of the map that defines the used setting items is parsed (or after an initial parsing run that just scans for setting item use), then we'd know which settings are legal and which aren't and can warn the user immediately. Of course, the settings loaded by a resource should apply to that resource only. No problem there: before they're loaded, an overlay over the whole settings tree is created, and the map specific settings are loaded into that, and only the map has access to that overlay. A map can load multiple config files of course.

Ok, I think I know what I want to say with that A resource should have a section defining which config items it uses and which config files it wants loaded extra:

Code: Select all

<ConfigItems>
<Node name="wrtlMap"><Node name="zone">
<Item name="size" type="float" default="20" />
</Node></Node>
</ConfigItems>

<ConfigFiles>
<Load name="maps.io">
<Load name="wrtl.io">
<Load name="wrtlmaps.io">
</ConfigFiles>

Getting us even closer, the search path for the config files should be determined by the resource path; otherwise there would be tons of name collisions for files with totally incompatible content requirements. That basically forces us to search for the files in the resource tree itself. Hrrmm.

---------------------------------------------------------

Ok, coming from the other direction. Configuration items should be defined in XML resources, let's call them config headers. The content would look similar to the ConfigItem secion in the code block above. Resources specify the config headers they use items from. Likewise, your config files also specify the config headers with the settings they are modifying.

Yes, I think that's it. Say I want to create a series of maps in Z-Man/maps/asymmetric with fortress maps where the two teams have different settings. I'd write a config header containing (using shell comments because they're easier)

Code: Select all

<ConfigItems>
<Node name="common">
<Node name="zone">
<Item name="radius" type="float" default="40" /> # defines item "common zone radius" for zone radii
</Node>
<Node name="cycle" overlayFrom="/ cycle" /> # lazy copy of root cycle physics settings
</Node>
<Node name="blue" overlayFrom="common" /> # lazy copy of all custom settings for team blue
<Node name="gold" overlayFrom="common" /> # same for team gold
</ConfigItems>

Of course, the config header may just be a lean XML wrapper around a big text section containing raw script code. Here, I'd prefer XML: it's the mapper that writes that file, and mappers are already familiar with XML and its workings.
With that header, provided the setting "cycle speed" exists in the root tree, it's now legal to write a config file

Code: Select all

ConfigHeader include("Z-Man/asymmetric/configitems-1.0.aacfg.xml") # include the above header
blue zone radius = 20 # define different settings for the two teams
gold zone radius = 10
blue cycle speed = 100
gold cycle speed = 50

Due to the included specification, the config system knows which settings are legal and which aren't and can give meaningful error messages. A map could contain:

Code: Select all

<ConfigHeader include="Z-Man/asymmetric/configitems-1.0.aacfg.xml" /> # include the above header
...
<Zone pos="<gold spawn point>" shape="circle" radius="gold zone radius"/> # simplified, of course: the zone of team gold
<Spawn pos="<gold spawn point>" settings="gold cycle" /> # again simplified: the spawn point could tell the game code which physics settings apply for the cycles spawned there.
... (same for team blue)

You'll notice: I did not put special markers there to indicate the usage of config items. I'm assuming here that script download will be safe and that we'll simply evaluate all values from XML that are supposed to be numbers, but aren't plain numbers, in the safe script environment.

[Luke-Jr]

Does vValue actually work now? Can it parse expressions and return the result?

It works more or less as much as it can without DCT.

Right now we need an expression parser, so we can do what Lucifer has cited:

Code: Select all

<Zone …>
    <Point x="x(t)" y="y(t)" />
</Zone>

What are 'x' and 'y' functions? If vValue had variable support (which isn't planned except for DCT), and 't' were defined, and maps supported vValues there, that would work.

Does Io provide efficient caching of expressions?

Provide? No, Io is a minimalistic language, it provides only the basics. But it allows you to implement caching (not going into details about implementation possibilities). And the cached value can reside in some C++ wrapped object and can even be referenced freely in C++ for extremely fast access.

But can it guarantee a non-stale cache and only update it when it would otherwise be stale?

Still trying to understand this discussion. I gather vValue has been designed to permit expression evaluation in arena settings, but what kind of elementary values are permitted in such expressions? Can we refer to config items? To current game state? To past game state / user-defined variables?

DCT = config items, with support for arbitrary "user-defined" variables. Provided the config items are based on 't'/time, they can be asked for their value at a past or future time.

Or, to ask bluntly: What can vValue do what scripting can not?

- Speed and efficiency
- Better prediction
- No scripting overheads

Back to DCT, resources would be loaded directly into the tree. So we potentially already have the tree structure for maps and cockpits. The organizational work for DCT mostly involved turning the existing configuration settings into an organization of objects-- cycles, players, etc already have an OO layout, so they would only need minor tweaking.

Someone brought up the point about adding new game objects-- perhaps we should consider abstracting a bit more at the same time: make all game objects use a common subclass defining size, texture, colour, etc and get more specific from there.

[Z-Man]

Caches:

But can it guarantee a non-stale cache and only update it when it would otherwise be stale?

Sure, otherwise I wouldn't talk about tracking dependencies. And I wouldn't call it cache, I'd call it, umm, approximation of functions by piecewise constants

I think you should explain a bit more. For example, HOW can resources be loaded into a tree that basically holds just values? How should game objects be represented in the tree? Objects aren't just a bunch of numbers. So far, all I know about DCT is that "It's a Tree!". Details, please Of course, you can do what you want and implement what you want (as long as you don't expect it to be used just because it's there), but it is always better to let others cross check your stuff.

[Luke-Jr]

I think you should explain a bit more. For example, HOW can resources be loaded into a tree that basically holds just values? How should game objects be represented in the tree? Objects aren't just a bunch of numbers. So far, all I know about DCT is that "It's a Tree!". Details, please

Ok, I'm going to try rephrasing this all :p

DCT would be our fixed high-level API. It consists of objects and "attributes", represented in a tree form. Attributes are always either unset (reading will pass through to the parent object's value, generally for defaults) or contain a vValue. Unlike C++, "classes" are objects themselves and have default values as their attributes (this is what replaces CYCLE_ACCEL and such). The tree representation should be based on a "Tree" object with attributes that act as arrays (either functions returning a different object based on an int argument, or actual arrays) for Objects, Players, Vehicles, Cycles, etc.

vValues are limited to simple math, specific builtin functions we define, and referencing attributes via the tree representation of DCT. Anything beyond that will need scripting, possibly including current C++ functionality such as preventing enemies from the same client/IP, preventing imposters, voting, etc...

[Z-Man]

That clears some things up, but others are now more confusing.
You say a failed lookup is directed to the parent, and that classes are objects. I assume that means there is a "cycle" object in your tree representing the cycle class in C++, and that when a cycle gets created in C++, it gets instantiated as a child of said cycle object. So when I ask that cycle instance what its acceleration value is, it would look it up it its parent, which is the root cylce object in the tee. Right?

Why does the Tree object take an int argument for lookup? Isn't it a string traditionally?

And how do you load XML files into that?

How would it be used? How can a user create a new variable? How can anything (resource, script, C++ code) reach that variable? How would it be possible, in this system, to give two teams different physics settings? How would object creation work? It probably needs to interact with this, and it also needs to interact with the script (to create those modified C++ objects with virtual functions overridable in script).

How do you think what vValue provides will be enough? It can only be numbers and strings (and functions returning numbers and strings)? What about cross-references, arrays of things, coordinates? All of that is found in the game code somewhere.

Most importantly, if this is just about making vValue useful: why not use normal C++ means? With pointers to the input data? We need to notify some system anyway whenever a vValue input changes, so why not make it simple and define a vValueInput class which does this automatically in the assignment operator and that also informs the vValues it is used by when it gets destroyed? This would also be much faster than going over a tree every time a vValue function needs reevaluation. And the initialization of the vValues to functions could be handled by the script, to which we need an interface exporting all kinds of things anyway. I don't see a reason to have TWO interfaces exporting object properties.

And why is it called DCT if is goes so far beyond configuration?

[Luke-Jr]

You say a failed lookup is directed to the parent, and that classes are objects. I assume that means there is a "cycle" object in your tree representing the cycle class in C++, and that when a cycle gets created in C++, it gets instantiated as a child of said cycle object. So when I ask that cycle instance what its acceleration value is, it would look it up it its parent, which is the root cylce object in the tree. Right?

Unless that individual cycle object has had its acceleration changed, or has been reparented (eg, via a script to add cycle classes).

Why does the Tree object take an int argument for lookup? Isn't it a string traditionally?

The Tree object is the root object. Its "Cycles" attribute-function takes an int argument and acts like an array. so, /Cycles(0) would return the first cycle, which is likely the class object.

And how do you load XML files into that?

Each node would become an object, etc.

How would it be used?

at the start of a round, you would execute "/Grid << Anonymous/original/classic-0.1.1.aamap.xml" or such

How can a user create a new variable?

Syntatically? Undefined at this point. I was thinking a new variable would be created by setting it.

How can anything (resource, script, C++ code) reach that variable?

Just like it reaches any other variable, for scripts and vValues. C++ code would need to use a DCT API.

How would it be possible, in this system, to give two teams different physics settings?

Have a script create new subclasses of the cycle object-class, modify its settings, and change the parent of all players based on their team.

How would object creation work?

Either creating a new one from scratch (from a parent object), or loading a resource into a new one.

How do you think what vValue provides will be enough? It can only be numbers and strings (and functions returning numbers and strings)? What about cross-references, arrays of things, coordinates? All of that is found in the game code somewhere.

ph has some basic array support in vValue, I think. Cross-references would simply be either a shared vValue or a vValue pointing to another attribute. Coordinates would be based on an eCoord object.

Most importantly, if this is just about making vValue useful: why not use normal C++ means? With pointers to the input data? We need to notify some system anyway whenever a vValue input changes, so why not make it simple and define a vValueInput class which does this automatically in the assignment operator and that also informs the vValues it is used by when it gets destroyed? This would also be much faster than going over a tree every time a vValue function needs reevaluation. And the initialization of the vValues to functions could be handled by the script, to which we need an interface exporting all kinds of things anyway. I don't see a reason to have TWO interfaces exporting object properties.

Who said the code would go over the DCT for evaluation at all? While certain things might benefit from relative DCT references, the most common would be a direct reference, which would be used as a pointer in the C++ implementation.

And why is it called DCT if is goes so far beyond configuration?

Shrug, you have a better name? DCT came from back when it was simply the idea of having configs in a tree, that could be switched/rotated around.

[Lucifer]

Right now we need an expression parser, so we can do what Lucifer has cited:

Code: Select all

<Zone …>
    <Point x="x(t)" y="y(t)" />
</Zone>

What are 'x' and 'y' functions? If vValue had variable support (which isn't planned except for DCT), and 't' were defined, and maps supported vValues there, that would work.

Well, when I go to class in a few minutes, my teacher is going to write "x(t)" on the chalkboard and expect me to understand that what will usually be put there is the definition of the function, not the name of it....

[Z-Man]

Why does the Tree object take an int argument for lookup? Isn't it a string traditionally?

The Tree object is the root object. Its "Cycles" attribute-function takes an int argument and acts like an array. so, /Cycles(0) would return the first cycle, which is likely the class object.

Hmm, what about classes derived from gCycle? Would they appear in the cycles array? And wouldn't then /Cycles(0) be the parent of /Cycles(1) for the purpose of property lookup fallback? That's not exactly a tree like I'm used to Will the number of each object be constant? I suppose that is needed for references.

And how do you load XML files into that?

Each node would become an object, etc.

Ah, so not by magic? We'd still need, for each resource type, loading code that knows how to create the right kinds of objects in the right way?

I was thinking a new variable would be created by setting it.

That would fail to catch typos, see my last long post on resource defined setting items where I'm struggling precisely with that problem.

How can anything (resource, script, C++ code) reach that variable?

Just like it reaches any other variable, for scripts and vValues. C++ code would need to use a DCT API.

I noticed the C++ quesion is a bit silly here, because C++ code, by definition, would not know what to do with the value retrieved. So probably the only way custom setting items would be accessed would be via vValues depending on them. Nevertheless, any plan on how this interface would look like? Regular tree navigation stuff? And how would the regular lookup from script be?

How would it be possible, in this system, to give two teams different physics settings?

Have a script create new subclasses of the cycle object-class, modify its settings, and change the parent of all players based on their team.

Shouldn't it be possible to just pass each cycle the settings to use at creation time (if the code handling that is adapted to that possibility, of course)? I mean, what settings the cycle uses will be determined by the config system, and from where it gets them should not be fixed.

How do you think what vValue provides will be enough? It can only be numbers and strings (and functions returning numbers and strings)? What about cross-references, arrays of things, coordinates? All of that is found in the game code somewhere.

ph has some basic array support in vValue, I think. Cross-references would simply be either a shared vValue or a vValue pointing to another attribute. Coordinates would be based on an eCoord object.

Ah, so sub-objects are allowed. What about pointers to objects? What I'm getting at here is this: either vValue can't handle everything it may be needed for in this scheme, or it needs to get so bloated that the speed benefit over the script will evaporate.

Who said the code would go over the DCT for evaluation at all?

I was exrapolating from the little information available. You said something about relative references in the tree, and I interpreted it as that. Because if it doesn't go over the tree for evaluation, what if the tree gets rearranged?

Shrug, you have a better name?

Dunno. DAS: Data Access System, GSH: Game State Hierarchy, DST, Data Storage Tree, UGH: Universal Group Hierarchy, LRS: Luke's Replacement for Script It's not a scenegraph, that's for sure, you'd model world coordinate relationships in one.

[Luke-Jr]

Why does the Tree object take an int argument for lookup? Isn't it a string traditionally?

The Tree object is the root object. Its "Cycles" attribute-function takes an int argument and acts like an array. so, /Cycles(0) would return the first cycle, which is likely the class object.

Hmm, what about classes derived from gCycle? Would they appear in the cycles array? And wouldn't then /Cycles(0) be the parent of /Cycles(1) for the purpose of property lookup fallback? That's not exactly a tree like I'm used to Will the number of each object be constant? I suppose that is needed for references.

/Cycles would be a pseudo-array of anything derived from a generic cycle, including subclasses and their children. Subclasses might also have their own more specific arrays, though.
Assuming /Cycles(0) is the generic cycle class, it would be the parent of all cycles. Numbering is arbitrary, and only for interators, really. References would need to be relative or pointer-ified.

And how do you load XML files into that?

Each node would become an object, etc.

Ah, so not by magic? We'd still need, for each resource type, loading code that knows how to create the right kinds of objects in the right way?

No, because we'd have generic code for creating objects and setting attributes. So we could have a XML loader that used that API.

I was thinking a new variable would be created by setting it.

That would fail to catch typos, see my last long post on resource defined setting items where I'm struggling precisely with that problem.

True, perhaps using := like in Io, then-- so long as we don't throw an error if it already exists.

How can anything (resource, script, C++ code) reach that variable?

Just like it reaches any other variable, for scripts and vValues. C++ code would need to use a DCT API.

I noticed the C++ quesion is a bit silly here, because C++ code, by definition, would not know what to do with the value retrieved. So probably the only way custom setting items would be accessed would be via vValues depending on them. Nevertheless, any plan on how this interface would look like? Regular tree navigation stuff? And how would the regular lookup from script be?

From C++/low-level-SWIG, I'd imagine maybe DCT.get("/path/to/value") returning an object or vValue.

How would it be possible, in this system, to give two teams different physics settings?

Have a script create new subclasses of the cycle object-class, modify its settings, and change the parent of all players based on their team.

Shouldn't it be possible to just pass each cycle the settings to use at creation time (if the code handling that is adapted to that possibility, of course)? I mean, what settings the cycle uses will be determined by the config system, and from where it gets them should not be fixed.

That requires configuring each cycle independently, which would be a hassle for an admin to manually change. Using subclasses, the admin can simply change the attribute on the subclass to affect all cycles parented from it.

How do you think what vValue provides will be enough? It can only be numbers and strings (and functions returning numbers and strings)? What about cross-references, arrays of things, coordinates? All of that is found in the game code somewhere.

ph has some basic array support in vValue, I think. Cross-references would simply be either a shared vValue or a vValue pointing to another attribute. Coordinates would be based on an eCoord object.

Ah, so sub-objects are allowed. What about pointers to objects? What I'm getting at here is this: either vValue can't handle everything it may be needed for in this scheme, or it needs to get so bloated that the speed benefit over the script will evaporate.

Pointers to objects are supported in certain cases, such as traversing the tree. References to vValues or DCT locations are also.

Shrug, you have a better name?

Dunno. DAS: Data Access System, GSH: Game State Hierarchy, DST, Data Storage Tree, UGH: Universal Group Hierarchy, LRS: Luke's Replacement for Script It's not a scenegraph, that's for sure, you'd model world coordinate relationships in one.

So pick one and rename it. I'm not stuck on DCT. :p