The official Ladle Server stress testing thread.

[compguygene]

I have tried on several occasions in-game to get players to help me stress test my new German server by either going to ID MegaFortress Germany, it supports 9v9, or the new Ladle 44 TheServerPharmDE Ladle server. I can easily get 3 to 6 players to join, but not enough are willing to leave G5's or whatever to actually test this. Perhaps it's just me, I just want this server tested! I also think we really need to organize stress testing of all the existing Ladle Servers before this next Ladle. I am also going to be putting up one based in Florida that has really nice specs, including a 1000mbit (1gbit) connection, instead of the usual 100mbit connection that good servers have.
But, I really need the community to test this stuff. So, what I am asking, in the self-organizing spirit of the Ladle, is that all server owners and players please organize some tests. I am running webmin on my servers so I can get stats without being there. I just need people to play in the servers so we don't go into this next Ladle with great servers being done in one or 2 rounds, and servers that perhaps are not optimal being used for the semis and beyond.

[dariv]

Can't a single user open 16 instances of the client and connect to the server with each one? The ping would be the same but if each is running a chatbot, the bandwidth should be equivalent to 16 real players.

Someone who can write scripts could even test all 8 servers automatically this way, from a signle machine. Input the server IP, and output the mean/variance of ping over a 5 minute period for each server.

[compguygene]

none of that truly simulates what it is like when a user is under the load of actual gameplay.

[Lucifer]

Where's Jonathan and his army of bots when you need him?

[compguygene]

I disagree that a bot can simulate this. For example, two years ago, I played a part in trying to duplicate a bug that in CTF servers, a very well skilled player, in a full server, doing a lot of "wiggling", would cause the server the lock up and reset sometimes. Only a few players could "wiggle" enough to cause this. I found in my own testing, that by hosting the server with better network settings, and on a more powerful hosting environment "fixed" the problem. No bots were able to duplicate the problem.
I have spent a significant amount of time in servers that I host watching a live server monitor of resources via webmin. There is quite a difference between the resources 12 noobish players use in a fortress server and 12 pros. This only makes sense with tighter grinds, faster turns, etc. Bots are not going to reproduce that specific behavior that can really stress a server's resources.

[Lucifer]

I disagree, the arma AIs can make all those maneuvers just fine, you just have to tweak the settings.

But I don't know what Jonathan's army of bots were, I only know that he'd login to the server with them. If you can only drum up 6 players, then you can focus the test on those 6 players, but fill up the server with connected bots.

One of the rules of war is that you have to fight with the weapons you have, not the weapons you wish you had.

Anyway, good luck finding enough players to give the stress test you want. I still have "develop AI clients for stress testing" on my todo list, I'll let you know when I do it.

[INW]

I think a bunch of players would grab a better test than a bunch of bots. With a bunch of players, you can ask them how the server felt. Jumpy? High pings? (Bots have a ping of 0 so that wouldn't work with AI's at all) Smooth playing or multiple freezes? I think a real humane testing would benefit best but which is easier- a quick test with a bunch of bots that can;t produce much information or trying to find a bunch of players to join a server and play with viable output?

Hmm.

[Jonathan]

My bots were really just modified chatbots. Stupid impulsive critters, but able to survive. I don't think they generally turned as often as real players. And as I kept them in splitscreen on a single client, the bandwidth hit was only linear. Perhaps most importantly, I don't have them anymore.

#define RLBOT should prove that bots can stress-test, albeit that might be overdoing it.

[compguygene]

I disagree, the arma AIs can make all those maneuvers just fine, you just have to tweak the settings.

But I don't know what Jonathan's army of bots were, I only know that he'd login to the server with them. If you can only drum up 6 players, then you can focus the test on those 6 players, but fill up the server with connected bots.

One of the rules of war is that you have to fight with the weapons you have, not the weapons you wish you had.

Anyway, good luck finding enough players to give the stress test you want. I still have "develop AI clients for stress testing" on my todo list, I'll let you know when I do it.

I stand corrected! Hope to see this done soon!

[Lucifer]

@INW: I'm not talking about server-controlled bots, I'm talking about bots that connect as clients and act like human players (with chat and everything). Since it's difficult to drum up enough players to really stress-test a server on demand (although popular servers certainly get enough players, it's not on demand), CGG could run the 6 players he gets, and add 12 client-bots and get the same test he wants, but limited to the feedback of 6 players rather than the 18 he wants.

@Jonathan: Didn't know they were dumb chat-bots.

[sinewav]

I was really hoping this thread would be about a reliable, objective stress test we could run on all ladle server as a way of benchmarking and reporting the results. Is there some sort of script that can be written and distributed to volunteers to provide server load fo such a test? I'd much rater run a little app on my comouter while I'm working on stuff rather than trying to meet up with a bunch of people at a specified time. Speaking of which, why not just get a group together an hour before the next Ladle starts and test the server then?

[Jip]

#define LRBOT should prove that bots can stress-test, albeit that might be overdoing it.

What would that do?

[Jonathan]

Oops. Actually it's RLBOT. It makes the chatbot triple bind incessantly. Its name stems from one of my own bots that zigzagged (which I think was the real LRBOT); unsurprisingly, it's short for right-left.

[Lucifer]

I was really hoping this thread would be about a reliable, objective stress test we could run on all ladle server as a way of benchmarking and reporting the results. Is there some sort of script that can be written and distributed to volunteers to provide server load fo such a test? I'd much rater run a little app on my comouter while I'm working on stuff rather than trying to meet up with a bunch of people at a specified time. Speaking of which, why not just get a group together an hour before the next Ladle starts and test the server then?

Well, the problem is that we don't currently have client-bots that can be used for the testing. If someone is willing to volunteer the programming time, it would really rock, and I'd be happy to mentor the work. Best way to contact me about it is probably irc, but if you hit me up on facebook, that works too.

[compguygene]

I was really hoping this thread would be about a reliable, objective stress test we could run on all ladle server as a way of benchmarking and reporting the results. Is there some sort of script that can be written and distributed to volunteers to provide server load fo such a test? I'd much rater run a little app on my comouter while I'm working on stuff rather than trying to meet up with a bunch of people at a specified time. Speaking of which, why not just get a group together an hour before the next Ladle starts and test the server then?

Well, the problem is that we don't currently have client-bots that can be used for the testing. If someone is willing to volunteer the programming time, it would really rock, and I'd be happy to mentor the work. Best way to contact me about it is probably irc, but if you hit me up on facebook, that works too.

I am really just hoping to get a good discussion going about the topic. I also hope to report results of the testing we can get done here.