Suddenly high CPU usage of the server?

What do you want to see in Armagetron soon? Any new feature ideas? Let's ponder these ground breaking ideas...
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Suddenly high CPU usage of the server?

Post by danger89 »

Hi,

I got a big problem. Suddenly my servers->both takes (random?) at lot of CPU usage!

The server has a tripple core 2,7GHz AMD processor but the server takes 20% CPU with only 2 players......

3 days ago armagetronad-dedicated was using 100% CPU!

I didn't changed anything in the config files and did not else?

What to do? Somebody know this problem? If this isn't solved quick enough I will close all my Wildcat servers, it's ruining my complete server.
arma_cpu.png
If I remember it right, before it was always 1% maybe 2% or 3% CPU, not really much more.

Ps.
Armagetron server reboot didn't solve the issue.

Kind regards,
Melroy
Last edited by danger89 on Sun Feb 06, 2011 2:19 pm, edited 1 time in total.
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
User avatar
Tank Program
Forum & Project Admin, PhD
Posts: 6712
Joined: Thu Dec 18, 2003 7:03 pm

Re: Suddenly high CPU useage of the server?

Post by Tank Program »

Can't say I remember a particular instance of this happening before. Maybe a debug recording? Look at what files it's got open with lsof? Check out netstat?
Image
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Re: Suddenly high CPU useage of the server?

Post by danger89 »

Ok, how do I start recording debugging? And what is netstat?
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
User avatar
Z-Man
God & Project Admin
Posts: 11740
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: Suddenly high CPU useage of the server?

Post by Z-Man »

danger89 wrote:Ok, how do I start recording debugging?
You pass the extra command line argument "--record highcpuusage.aarec" to the main armagetronad executable. How are you starting the servers right now?
danger89 wrote: And what is netstat?
A program telling you which program has network connections open with what. Dunno how it can be useful here.
User avatar
Tank Program
Forum & Project Admin, PhD
Posts: 6712
Joined: Thu Dec 18, 2003 7:03 pm

Re: Suddenly high CPU useage of the server?

Post by Tank Program »

Well I'll admit netstat is a bit of a long shot, but if the server's getting flooded over the network causing high CPU usage, it'd give some indication maybe.
Image
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Re: Suddenly high CPU usage of the server?

Post by danger89 »

Ok, I have some information except the debugging record (I will do that as soon as possible!).

LSof:
http://pastebin.com/rJmKzQb9

Start-up script (shell):
http://pastebin.com/vae80T4s

IPTraf monitor tool (log file):
http://pastebin.com/Kf0wAzDG (will be destroyed after 1 month)
(6 player of Armagetron active)
NOTE:
I also run a Call of Duty 4 server (5 players active) & MySQL server at the same server as Armagetron, just that you know.

Ignore port 28960, 28961, 28962 & 3306.

Netstat:
http://pastebin.com/jQVAU2j8

Var directory:
http://pastebin.com/70y1dKxL

Note:
It seems if the server is empty the CPU usage will go drastic down.
Attachments
current_load.png
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
User avatar
kyle
Reverse Outside Corner Grinder
Posts: 1987
Joined: Thu Jun 08, 2006 3:33 pm
Location: Indiana, USA, Earth, Milky Way Galaxy, Universe, Multiverse
Contact:

Re: Suddenly high CPU usage of the server?

Post by kyle »

also what version of armagetron are you using?
Image
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Re: Suddenly high CPU usage of the server?

Post by danger89 »

kyle wrote:also what version of armagetron are you using?
armagetronad-0.2.8.3.1 met ZThread-2.3.2 configured.

Note: the server worked well for a long time, but just suddenly about 1 week ago it went wrong :|
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
User avatar
Tank Program
Forum & Project Admin, PhD
Posts: 6712
Joined: Thu Dec 18, 2003 7:03 pm

Re: Suddenly high CPU usage of the server?

Post by Tank Program »

Another question then. What changed 1 week ago? Software upgrades? Some other sort of system or configuration change?
Image
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Re: Suddenly high CPU usage of the server?

Post by danger89 »

Thats the problem, nothing as far as i know.
EDIT:
The only what I can think of is I added 1 user via USER_LEVEL blabla@forums 2. But that is not related to this problem I guess.
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Re: Suddenly high CPU usage of the server?

Post by danger89 »

OK I got the debug recording. I joined the server (so 1 player) and watched via htop command it says the server has 25% CPU usage.

Please download the recording debug log file here:
http://www.joinmyserver.com/library/dow ... sage.aarec

With zero players it's near 0% CPU usage.
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
User avatar
Z-Man
God & Project Admin
Posts: 11740
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: Suddenly high CPU usage of the server?

Post by Z-Man »

Without actually playing it back, here's one sticky bit from your config:

Code: Select all

DEDICATED_FPS 500
That one lets the server run the simulation loop 500 times per second. First thing I'd do is try and reduce that, there shouldn't be any ill effects.
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Re: Suddenly high CPU usage of the server?

Post by danger89 »

Z-Man wrote:Without actually playing it back, here's one sticky bit from your config:

Code: Select all

DEDICATED_FPS 500
That one lets the server run the simulation loop 500 times per second. First thing I'd do is try and reduce that, there shouldn't be any ill effects.
As you can see in the same log file above I set it to 40 FPS.
CONFIG LDEDICATED_FPS 40 # maximum simulation steps per second the dedicated server will perform

CONFIG LDEDICATED_FPS_IDLE_FACTOR 2.0 # number of times per frame the server should check whether simulation can be done if no network input is coming
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
User avatar
Z-Man
God & Project Admin
Posts: 11740
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: Suddenly high CPU usage of the server?

Post by Z-Man »

Yes, but there's a line later somewhere that sets it to 500, overriding the previous setting. And nothing after that.
User avatar
danger89
Average Program
Posts: 54
Joined: Thu May 14, 2009 9:12 pm
Location: The Netherlands
Contact:

Re: Suddenly high CPU usage of the server?

Post by danger89 »

Yes, but I dont set the second one. I cant find it in any cfg file.
WILD CAT EX-Owner
Website: http://www.joinmyserver.com
Post Reply