vValue and DCT

[wrtlprnft]

It does work, as far as I tried it.

Code: Select all

tValue::BasePtr myValue = tValueParser::parse(tString("1+2"));
std::cerr << myValue->GetInt() << std::endl;

This should actually work. There's a way to add functions with or without parameters, it's done somewhere with the sin function in stc/tools/values/.

[Z-Man]

Meriton: Yes, that is one of the possibilities. Scripts may also be installed purely clientside (for fancy new effects, new intelligent HUD displays, new camera modes) or be sent directly over the network for small extrapolation stuff (although that would be my imagined ideal biotope for vValue). Then there is the standalone game mode; scripting would make it far easier to create varied gameplay with missions, objectives, tutorials, all that.

Does Io provide efficient caching of expressions?

Provide? No, Io is a minimalistic language, it provides only the basics. But it allows you to implement caching (not going into details about implementation possibilities). And the cached value can reside in some C++ wrapped object and can even be referenced freely in C++ for extremely fast access.

[meriton]

Still trying to understand this discussion. I gather vValue has been designed to permit expression evaluation in arena settings, but what kind of elementary values are permitted in such expressions? Can we refer to config items? To current game state? To past game state / user-defined variables?

Or, to ask bluntly: What can vValue do what scripting can not?

[Z-Man]

Or, to ask bluntly: What can vValue do what scripting can not?

It already exists, and it probably is faster than scripting will be. I'm unsure about the other questions as well, but it seems vValues can be strings, ints and floats.

[dlh]

Or, to ask bluntly: What can vValue do what scripting can not?

It already exists, and it probably is faster than scripting will be. I'm unsure about the other questions as well, but it seems vValues can be strings, ints and floats.

Scripting exists and is working fine for me. ;)

[meriton]

Existence is something scripting will probably achieve as well, which leaves performance. Has anyone benchmarked or otherwise assessed the performance drop that would be caused by scripting? I mean, if all the script does is simulate sty-ball, I can't imagine how it could add significantly to arma's total system requirements.

[wrtlprnft]

Note that vValue (or tValue or cValue or gHudValue; it changed names over time) was originally intended only for the cockpit. There you'd want to have the user define some value in any way he wishes, and then have the gauge decide what type it expects.
It was a simple base class with virtual getter functions for int, tString and float, and derived functions that held the actual values. Later I used the concept to implement dynamic values (ie callbacks), and even later I created subclasses that could hold multiple other values and perform some operation on them when getting queried and return the result.
ph wanted me to move it to tools/ because he wanted to use it for his zones, and then I lost track of all of it. Ph added stuff, and luke moved everything around again and created his parser, and noone knows how it all works now.

[meriton]

Oh, rereading this thread my last post is horribly redundant

Sorry about that.

I agree with Lucifer:

So, like I say, if Luke wants to still do it, that's his prerogative, so long as he doesn't mind terribly if we later discover it's not useful in light of where scripting went. If I were he, I'd wait and see how the Io thread works out.

[Lucifer]

Ok, until a few months ago, I had an eagle's eye view of the vValue thing. Basically I took luke's advice and "****** off" when he told me to.

Anyway, wrtl's history is mostly accurate. Philippe went to get it from the cockpit because I told him to. He needed to store values that could be any type and gotten from any place, and I told him to check out the cockpit stuff (recall that the cockpit rewrite of the hud was originally done with me sitting watch and merging code until wrtlprnft got svn access). That was the point where it was decided it should be a general tool. wrtlprnft and I sharply disagree on what the expressions were originally intended for, I say he was narrowly focused on the cockpit and I was thinking about other stuff, he says it was solely for the cockpit. Regardless, it came out of an irc discussion he and I were having. We spent several months looking for code and/or joking about "when wrtlprnft finally writes an expression parser". Then luke jumped in with vValue (very late to the discussion, therefore his "**** off" to me was extremely rude considering I had been a part of it pretty much the whole time, but we expect luke to be rude and not know what's going on), and I still think vValue is NIH syndrome. In any case, until luke showed up, we were looking for third party code to do it.

So somewhere before luke started in with vValue (and I think in part because I suggested we could just use Python to parse expressions), I suggested to philippe that the scripting engine could parse the expressions for us, and it was worth thinking about. At that time, scripting was still faaaar away. Last word on it was that z-man was going to play with swig sometime soon, but afaik he didn't get to it, and nemo hadn't started his ruby branch yet. So, at that time, an expression parser was thought of (by me, and I believe wrtlprnft) as a little piece of code that was only worthwhile if we could find one or implement one very cheaply, and was a stopgap until we had scripting. Then we'd reevaluate whether or not keeping the expression parser as it was was good or if we should look at scripting.

There's been a lot of scope creep since then. Obviously because Luke put his hand in.

vValue is separate from how to organize config items. I agree with the basic premise of organizing config items in a tree, but I question the usefulness of the tree. I usually use trees in my own stuff, where a config item looks like a filepath, but there are limitations to that approach. So my own take is I have to see what someone writes. The api to access it and the file format associated are both far more important to me than what it looks like in memory, and maybe I missed it, but it doesn't look like either has been discussed yet.

On scripting, we all agreed that python scripting would be optional. That's it. And the reason we did was because of luke's objections that python scripting would make it impossible for him to play the game on his PDA and it was easier than fighting over it. Luke suggested that MOO scripting could be required because it was so small, therefore, there is obviously a size of scripting environment where luke agrees that it's ok to make it required. I suspect he only approves of MOO being required, more NIH. But if it comes to a vote of some sort, I'm for requiring scripting and luke can fork if he doesn't like it, regardless of what language we go with. I'm done with appeasal, I want to see the work done and all this appeasal is just stopping it.

About using the scripting engine for the expression parser. You wouldn't have to learn a scripting environment. The ideal scripting engine that can be used as an expression parser uses essentially algebraic expressions, and that's all we're talking about supporting there. Maybe it'll make sense to embed real scripts in there, maybe not. In any case, the expression parser that is most useful to us is the one that requires minimal study to write expressions, and that pretty much means algebra. Python uses algebraic expressions, I don't know about Io and Ruby. Ruby appears to from what I've seen. Remember, we're only talking about stuff like (x^2 + y^2). One line with a series of terms connected by arithmetic operations with regular functions mixed in, you know, trig functions, logarithmic functions, that stuff. What you'd put in a calculator.

The variables available for the expression would be other vValues, game constants, state variables, that sort of thing. Gametime is the one that strikes me as the most useful, wrtlprnft has some cockpit-based examples that use other game data. Luke hasn't produced any examples. PHilippe is the one most dependent on it, he needed it in his last period of coding, and he's got a whole slew of examples of his own for zones v2. In any case, if you make the radius of a zone = 5sin(t), where t is gametime, what's the radius of the zone? It's oscillating, obviously, between 5 meters and 0. So a person ideally just needs a good math background, some understanding of parametric systems, trig (of course), and a few other things. Even if all someone has is basic algebra, they can probably reach some pretty good stuff just by reading the docs we'll write. But no programming should be required. In fact, if we wind up with python, we can use pythonica for the expression parser, which gives us mathematica-like syntax.

[Z-Man]

For the basic stuff, Io uses normal expressions; x*x+y is jus as valid as everywhere else. For special functions, Io is a bit special (which was my main gripe): you write "2 sin" instead of "sin(2)". Seen from a different angle, that's what you type in your regular calculator as well, so it's not that bad.

Right, let's discuss the actual configuration system. I'm off to my early morning lecture, I'll write something on that after that.

[Z-Man]

So this is what I've come up with during the boring parts of the lecture. I'm assuming we'll be using Io and the Io parser for the configuration system, but things should translate easily to the other scripting languages.
For the user, if she wants to do the things she's doing with the current configuration system, not much will change. She'll have to throw in some equal signs and replace some underscores with spaces. For example, cycle_rubber 200 could become cycle rubber = 200. She'll be able to save some typing using

Code: Select all

cycle do(
 rubber = 200
 speed = 10
 brake  = -10
)

to access multiple settings in one node. To base one setting on another, it's a simple matter of writing

Code: Select all

cycle do(
  speed = 10
  relativeRubber := 20  // this introduces a new config item
  rubber = speed * relativeRubber
)

And from then on, changing speed or relativeRubber will also change rubber. If that's undesired and the formula should just be evaluated at the time of its first evaluation, writing "now" after it should suffice. Note: I only have some rough ideas how this could be possible. It should work with basic expressions, but I don't know about custom functions. Maybe it would be easier to turn the logic around, make the default the evaluation now and require to write "rubber = autoUpdate( relativeRubber * speed )" for auto-updating settings. This, and whether it will be indeed this easy to add new config items (looks possible in Io), strongly depends on the scripting language's capabilities.

There should be two ways of creating copies of settings; one would be cloning, that copies all values of the original over to the new sets of settings and makes the new settings independent of the old. The other would be overlaying; the overlay would not copy any settings from the original from the start, but relay requests for settings to the original instead. So changes to the original still affect the overlay. As soon as the user assigns a setting in an overlay a value, the overlay will use that instead of the original

Code: Select all

cycle speed = 10
cycle brake = -10
cycle rubber = 200
cycleClone := cycle clone
cycleOverlay := cycle overlay
cycle rubber = 0
cycleOverlay brake = 0
cycleClone brake = 0

brake would be zero then in both the overlay and the clone, speed would be 10 in both, and rubber would be zero in the overlay and still 200 in the clone.

Now to the C++ side. Every element of the factual configuration tree is a subclass of

Code: Select all

class ConfigTreeNode{
public: // standard tree navigation, throws exceptions when nodes don't exist
    ConfigTreeNode & GetParent();
    ConfigTreeNode & GetChild( char const * name );
};

The things you access with the Io commands above are ConfigItems in C++ which work a bit like they do now, but they'll hold the value themselves instead of a reference:

Code: Select all

template<class T> class ConfigItem: public ConfigTreeNode{
public:
    T const & Get();
    void Set( T const & );
private:
    T value_;
};

To allow the overlays to actually work, you won't access them directly in C++. Instead, you'll be handling ConfigItemProxies:

Code: Select all

template<class T> class ConfigItemProxy{
public:
    ConfigItemProxy( char const * name );

    // accessors
    T const & Get();
    operator T const &(){ return Get();}
    void Set( T const & );

    void Attach( ConfigTreeNode & node ); // attaches to the value in the config tree, provided it is a value of the right type. Exception thrown on failure.
private:
    ConfigItem< T > * value_;
};

The set and get method will be relayed to the config item. ConfigItemProxies will be arranged into structures for organization:

Code: Select all

struct CycleConfigItems: public ConfigItemProxySet<CycleConfigItems>{
    ConfigItemProxy<float> rubber_, speed_, brake_;
    CycleConfigItems(): rubber_("rubber"), speed_("speed"), brake_("brake"){ SetParent<RootConfigItems>("cycle"); }
};

Each cycle will be given a pointer to such a struct by the game logic and use the proxy accessors to get the settings. That's just two indirections without virtual calls or conversions, I think we can live with the performance hit of that. The thing in the body of the constructor tells the system that the cycle settings node initially sits directly in the root configuration node (clones and overlays can be everywhere); in practice, it'll probably rather be the a GameSettings node. It should be possible to only require the class to be forward declared at that point. If you want multiple copies of a node as the child (like for the four local players), just call the registration multiple times with different names.

The mysterious base class ConfigItemProxySet will handle all the gluing. It looks like this:

Code: Select all

class ConfigItemProxySetBase{
public:
    static ConfigTreeNode * CreatePrototypes( IoState * state, char const * rootNameInIo ); // initialize the config item tree, returns the root node

    template<class P> void SetParent( char const * name ); // register with parent node under that name

    void Attach( ConfigTreeNode & node ); // attaches to a node of the config tree; all setting proxies contained will be attached, too

    // stuff used by the ConfigItemProxies to register
};

template<class DERIVED> class ConfigItemProxySet: public ConfigItemProxySetBase{
// only private stuff that guarantees that the use of a DERIVED object anywhere in C++ triggers the generation of the glue code
};

Using it;

Code: Select all

// somewhere in the main program. Don't worry about the state argument, that's an Io internal.
ConfigTreeNode & rootNode = *ConfigItemProxySetBase::CreatePrototypes( state, "configuration" );

// in the game code:
ConfigTreeNode & cycleNode = rootNode.GetChild( "cycle" ); // fetch the node of the cylce settings

CycleConfigItems cycleConfig;
cycleConfig.Attach( cycleNode ); // attach the cycle settings to it

gCycle * cycle = SpawnCycle();
cycle->SetSettings( &cycleConfig );  // spawn cycle and give it its settings

// in gCycle:
CycleConfigItems * config; // passed from outside
float speed = config->speed_;

Game modifications would simply look elsewhere in the tree for the cycle settings to pass to the cycles.

You should also be able to auto-register subtrees; since there are many rubber settings, they'd probably go into its own rubber setting set, which will be a child set of the cycle settings. This could work like

Code: Select all

struct CycleRubberConfigItems: public ConfigItemProxySet<CycleRubberConfigItems>
{
    ConfigItemProxy<float> rubber_, speed_, minDistance_;
    CycleRubberConfigItems(): rubber_(""), speed_("speed"), minDistance_("mindistance"){}
};

struct CycleConfigItems: public ConfigItemProxySet<CycleConfigItems>{
    ConfigItemSubSet<CycleRubberConfigItems> rubber_;
    ConfigItemProxy<float> speed_, brake_;
    CycleConfigItems(): rubber_("rubber"), speed_("speed"), brake_("brake"){ SetParent<RootConfigItems>("cycle"); }
};

Attaching the cycle settings should automatically attach the rubber settings, which will be available via the rubber_ element of CycleConfigItems, a smart pointer to a CycleRubberConfigItems object. Note that the first rubber_ element gets an empty name as a constructor; I *hope* it'll be possible to make a treenode hold an item and a subtree simultaneously. It'd suck to have to write

Code: Select all

cycle rubber value = 200
cycle rubber speed 40

To set CYCLE_RUBBER to 200.

An interesting side node, the bits I remember about MOO are strikingly similar to Io. The crucial differences are that Io's documentation is way better (and, if you saw the Io docs, that tells you a lot about MOO) and I was actually able to get Io to run.

[Lucifer]

Hmmm, this is going to take a little while to sink in. I'm still questioning the usefulness of a tree itself. Does anybody feel like throwing together a diagram of all current config items as a tree? Something tells me this conversation would be a lot easier to have if we could refer to such a thing.

Ok, can we back off a hair for a bit while I let this sink in? At least for my part of the participation, y'all rip away at what's there.

Is there anything inherently right or wrong with having a config file as a script instead? I know there are other programs that do it, like GNU MailMan for example. Something's always rubbed me wrong about it, but I have to admit I haven't figured out what, exactly. Maybe some vestigial conservativism or something.

Ok, I went back to the beginning of the thread. I jumped in in the middle trying to moderate, so now I have to get ontopic to start with.

What luke's describing looks to me more like a scenegraph. That's how FlightGear organizes itself internally, and config items in FlightGear are just attributes of objects, where what you specify in the config file is used to initialize new instances of the objects. I think. I haven't looked at the actual code, this is based on limited observations of its behavior. But that's what it looks like to me, a treed hierarchy of all game data. Throw on a few more objects at the parent level and you add the UI to the tree. Is this what we want? My main gripe with how flightgear does it is that it's pretty tricky to specify config items in the config files, but that's not necessarily a failure of the system, it could just be a failure in the config parser or the documentation.

How would we propogate setting changes, then? I.e. if the cycle owns its own rubber, and we want to change rubber for all cycles in this grid, but not that one over there, how? And who owns which settings? Does the cycle own rubber? I realize we associate rubber with a cycle, but what if we create a tank object. Tank object will need rubber too, does it get the same rubber as a cycle, or does it get different rubber? Do zones have rubber? They currently don't, obviously, but it's reasonable I think for someone to create a zone that does. It could be argued that pig's styball zone should have rubber. Should rubber be the same everywhere? I think it should, and we should create a new object-specific setting to replace rubber (or create a new lag compensator device that sits underneath rubber, trivializing rubber to some extent).

One more question. How does the console interface look? We've talked about doing away with it completely, and maybe the event system we've tossed around as the replacement for the console fits here (and also provides a solution to the setting change propogation problem).

Ok, now I'd like to see a tree of current config items that shows some proposed and proof-of-concept config items.

[meriton]

I humbly suggest we should agree on the scope (and therefore, the requirements for) the config / scripting system before we discuss implementation details, because if the entire game-logic is modeled in a script, the script will access configuration items, but the C++ code won't.

Moreover, in such a situation, it would make sense to organize the configuration as variables of the script (to avoid writing a cross-language API). It appears possible that self-updating values could be expressed in the scripting lanuage with little effort.

[dlh]

So this is what I've come up with during the boring parts of the lecture. I'm assuming we'll be using Io and the Io parser for the configuration system, but things should translate easily to the other scripting languages.
For the user, if she wants to do the things she's doing with the current configuration system, not much will change. She'll have to throw in some equal signs and replace some underscores with spaces. For example, cycle_rubber 200 could become cycle rubber = 200. She'll be able to save some typing using

Code: Select all

cycle do(
 rubber = 200
 speed = 10
 brake  = -10
)

to access multiple settings in one node. To base one setting on another, it's a simple matter of writing

Code: Select all

cycle do(
  speed = 10
  relativeRubber := 20  // this introduces a new config item
  rubber = speed * relativeRubber
)

And from then on, changing speed or relativeRubber will also change rubber. If that's undesired and the formula should just be evaluated at the time of its first evaluation, writing "now" after it should suffice. Note: I only have some rough ideas how this could be possible. It should work with basic expressions, but I don't know about custom functions. Maybe it would be easier to turn the logic around, make the default the evaluation now and require to write "rubber = autoUpdate( relativeRubber * speed )" for auto-updating settings. This, and whether it will be indeed this easy to add new config items (looks possible in Io), strongly depends on the scripting language's capabilities.

autoUpdate looks easier to do and makes more sense to me:

Code: Select all

autoUpdate := getSlot("block")
extractValue := method(o, if(o getSlot("call"), o call, o))
cycle := Object clone do(
    speed := 10
    relativeRubber := 20
    rubber := autoUpdate(speed * relativeRubber)
)
extractValue(cycle rubber) println
cycle speed = 15
extractValue(cycle rubber) println
cycle rubber = 3
extractValue(cycle rubber) println

[Z-Man]

Meriton: No, not the whole game logic will be handled in the script. That would be a big chunk of work to be done at once before the game gets usable again. Many things will be overridable in script, but the core and default implementation for most things will stay in C++. That's why we need a C++ interface to the configuration system. But don't worry, the C++/script interface needs to be written anyway, and it's pretty much done for Ruby and in good progress for Io (in fact, the bits needed for the outlined system here are already in place). It won't be additional work for the configuration system beyond the single line calls doing "give me that script object, see if it is actually a float, and if yes, return it".

Nemo: wow. I had something way more complicated in mind. I see how dependency tracking and caching can be added to that.

Lucifer: we have time. I'm certainly not in a rush to implement any of this as long as the question which script we'll actually use is settled (Nemo's bit just gave Io a big bonus, I have to say).

The one thing I don't like about config files getting parsed by the script is this: they're basically free from scripts then, and any error will be handled as a scripting error. Correct me if I'm wrong, but if a scripting language finds an error, it usually just gives up completely. We could parse the config files line by line, but that loses us the do() things and the flexibility.

About Trees vs other structures: I think Luke is trying to develop the tree structure in this DCT.txt file in the trunk. For my part, I'd just sort the settings alphabetically and throw everything that begins with the same sequence (limited by the underscores) into one tree node. That already gives the root -> cycle -> rubber two layer hierarchy. The advantage of the tree over flatter structures, like from INI files, comes when you clone/overlay settings around, it makes it easier to access logical groups of settings. With a flat structure, it's the structure's designer's decision whether it's easy to copy all cycle settings or all cycle_rubber settings, with a tree structure, the one doing the copying can decide. That said, I've never used a tree like configuration system before.

It sounds a bit like Luke tries to establish that the configuration tree IS the whole gamestate, yes. I think that would be silly; for one, it only makes sense if you really get all of the gamestate into that, which includes the grid tesselation datastructures, which are just too huge and absolutely useless for direct access. And abstracting a gamestate by a tree of numbers and strings and links between them is about two levels of abstraction over the limit where there is so much abstraction around that you can't get your stuff done. I'd like the system limited to configuration. Actually, I'd also like to keep the game time out of it. The config system should be considered static unless the admin or a map rotation script or, in rare exceptions, a mission management script (mission: touch all zones. Each zone touch increases your speed) changes config items. All important bits of the real gamestate will be wrapped in script, and the script console will be just as easy to reach as the config console (possibly be even identical). If you want a zone to do sine waves, program it into the zone directly.

About setting propagation: no, a cycle does not own its own rubber. It references the rubber setting the game code gave it. If it's hardcoded into the game code that all cycles use the same rubber setting, then all cycles will have the same rubber. If the game code creates overlays of the cycle configuration (empty) for every cycle, then, by default, all cycles would still have the same rubber and changing the global rubber setting would affect all cycles; if you change the rubber setting in one of the overlays, only one cycle would be affected. Other game objects can have rubber and inherit the cycle rubber setting if they want; the game code just needs to give them the cycle settings. They can, of course, get a class-specific overlay or copy of the cycle settings so you can give tanks more rubber than cycles. You can also create one overlay of the settings per team, say to give the attackers in an asymmetric game mode higher speed, but less rubber.

The console interface would look like the config file interface, just like now. You write

Code: Select all

cycle accell rim = 1000

to kill those boring rimhuggers. In fact, and that's true for the config files, I think with a little magic (everything will be executed in a special configuration context, so we have some control over how script commands are interpreted), we may be able to restore the underscores. I'm not sure about the =. Commands that take a single argument could keep their current form:

Code: Select all

admin kick "Player 1"

Multi-argument commands would require parentheses. We could save the poor admin some typing work by adding stuff to the symbol search path, like

Code: Select all

pathAppend cycle
pathPrepend admin
kick "Admin" // found in admin node
rubber = -1  // found in cycle node
pathClear