Pings and Lag, suggestions?

[Swampy]

There are serveral points of discussion within this thread, so I'll start by using some bullet points:

- I'm new to Linux, anything is possible (to be set up wrong)
- My router's wireless capabilities are disabled (to rule out war driving)
- My router's password is changed from default
- I've had the same message N54 said he's had pop up happen to me as well
- I have a single external IP and 4 internal addresses. The teams server port and internal address has changed a few times. My external address hardly ever changes, and Lucifer has pointed out.
- As for lag and pings with the Teams server, I really think my issue is with the networking setup on the box. Pings to external addresses (pinging google, for example) is slower than any other machine in the house. Causal web browsing is painfully slow. Downloads (from sourceforge) are quick. The 2.7.1 download came down around 385kbs to the machine. So it could be resolutions and image loading that makes it appear to be slow.
- The teams server stated out with Mandrake 10.1 Official, later changed to Fedora and then Changed to 10.2 Mandrake 64.
- I ran the 2.7.1 dedicated version on each OS.
- I replaced the drive 10.2 is running on this morning with a spare IDE drive I had laying around. I'm back to 10.1 Official now.

I have 2.7.1 installed now, and I'm going to get the team’s config up and running this morning. We'll see how it goes.

Z-man, I'd be happy to provide you with any log information you'd like. I've browsed through the logs and the only thing that sticks out to me is a message indicated that the host name for the server isn't a fully qualified domain name.

Is there anything I can do to troubleshoot the NIC (and/or) router config to rule out the networking side of the box?

One thing I noticed this morning is what the ping charity reports when I launch the dedicated server from the consol. Ping charity changed from 100 to 100100. Is that normal?

[Z-Man]

Z-man, I'd be happy to provide you with any log information you'd like.

You could PM or mail me ( z-man at users.sf.net ) the logs of the last tree days or so, that would help. I think n54 may want a copy, too.

I've browsed through the logs and the only thing that sticks out to me is a message indicated that the host name for the server isn't a fully qualified domain name.

One of the "ANET_INIT: unable to determine... " messages? This points to a misconfiguration, but should not bother AA operation or anything else. You may want to fix it anyway, though: Determine your hostname via the "hostname" command and enter the line

Code: Select all

IP HOSTNAME

( replacing your internal IP and the hostname you found ) in /etc/hosts. Afterwards,

Code: Select all

ping $(hostname)

on the commandline should work if you did not disable ping packets in your firewall.

Is there anything I can do to troubleshoot the NIC (and/or) router config to rule out the networking side of the box?

Sorry, I would not know. I only know the basics of setting up a network, I'm not really capable of solving problems. You may want to try the program "traceroute" if you have not done so yet; it will ping a remote host and list the hops the packets take on the way, giving the time each hop takes.

One thing I noticed this morning is what the ping charity reports when I launch the dedicated server from the consol. Ping charity changed from 100 to 100100. Is that normal?

Yes, perfectly normal. When nobody is online, the ping charity is essentially undefined and just defaults to a high value.

[Swampy]

It might be too soon to say for sure, but installing Mandrake 32 bit might have been the fix. I've been playing on there for a while this morning and I haven't heard one complaint about lag. Pings are reduced locally and are pretty stable. They're running about the same as Swampland original and aren't jumping up like they've been in the past

Of course, this is all subject to change. It could also be that Internet usage in my area isn't as much as at night, too.

Also, the "painful" Web browsing I spoke of earlier is fixed. I had the DNS servers in the wrong order

It performs the same as my 3.0Ghz XP machine now

I'd like to give the server a couple of days of being beat on before I make any more changes.

Z-man, I'll still provide the logs for the earlier install if you'd like to see them. I don't want to put anything on your plate if you have other things you'd rather focus on.

(Lucifer, you're not off the hook so easy )

[Z-Man]

Well, let's hope this really solves the problem.

Z-man, I'll still provide the logs for the earlier install if you'd like to see them. I don't want to put anything on your plate if you have other things you'd rather focus on.

The reported incidents were a bit too strange to ignore, so I'd like to at least take a small look at the logs. Could you send them over?

[n54]

There are probably easier and more effective ways to launch attacks at either the server or its clients, just as you say.

Yes there are but that doesn't mean that somebody hasn't possibly found some weakness in armagetron and is trying to use or test it. We're not talking about DoS attacks here nor a brute force port scan but rather seeing if armagetron might have a weakness in what it allows. It's not a scenario we should dismiss imo.

What you call "normal" Armagetron port ( 4535, i assume ) is just the normal port on the server side. Only the server needs a socket with a known and published port number to be reachable. The client does not "bind" its socket to any specific port and gets a more or less random port number assigned by the OS. When the client initiates the connection, this port number gets automatically known tho the server so it knows how to send packets that find their way back.

I guess we're having a small misunderstanding here... I think we are in agreement on the following but maybe I said it in a way that was misunderstood:
If the server uses port 4534 to accept incoming connections then the client has no choice but to use port 4534 as the recipient address port (i.e. the server address port) in establishing a connection to the server. The client will use any previously unused local port that it wants to as it's own sending address port. Now after that initial connection from the client to the server the server will know how to address the client: it will simply use the senders ISP provided IP address and port (not the local ones behind NAT as the server wont get that info). The server and client will then continue to communicate using those now established addresses and ports. I guess we agree on this.

Next step:
In absolutely no way should the server be able to change the local port the client uses. The program uses UDP, a connectionless protocol, so if info is lost or a connection is lost well then it is lost and nothing else and whoever played will simply have to reconnect again manually (this is the meaning of the term "connectionless"). We have all experienced this when we are arbitrarely disconnected and it is the way it is supposed to work. If you want things to be connection-oriented (the opposite) then one uses TCP, simple as that.

I somewhat get the impression that you're saying the server tries to "blindly search" (what sending 0.0.0.0:2809 would be) when it looses a connection and
1. I haven't ever gotten such a situation/packet request before when I've simply lost connection, ever, and being disconnected this way isn't at all unusual
2. Noone with a tad of security on their network is going to accept such "blind searching".
3. Sending 0.0.0.0:2809 assumes that the client is running on an internal NATed lan and the server has absolutely no way of knowing wheter this is true or not so it doesn't make sense for the server to do it unless it's programmed to not only guess wildly about what ports the client has open but also to guess wildly about what environment the client is in.
I hope there is no code attempting blind searching in Armagetron if there is I think it should be removed as it will only cause trouble (basically UDP misuse).

Fixing the clientside port does not work for two reasons: you would not be able to run two clients on one computer, and connections through a NAT router will not work because the client port gets altered by it.

I'm not sure what you're talking about here, or why you are talking about this. Anyway: any and all ports get altered by NAT using encapsulation totally outside our control and this is a fundamental part of what makes NAT what it is and is absolutely no problem nor does it have anything to do with the client and/or whatever ports it uses and or whether several instances of the program are running and connecting to the internet at the same time (which should be sorted out by the clients using different sockets automatically and has nothing whatsoever to do with NAT).

Now, wait a sec. n54, your network setup is: you have a router connected to the internet, your pc connects to the router. The router uses NAT. On your PC, there is an additional personal firewall that triggered the alarm. Right?
Now, how the hell got the packet the firewall complained about past the router? When a NAT router gets a packet from the internet, it tries to match it to previous outgoing connections. Only if it finds one ( or you activated port forwarding for the destination port ), the packet gets transmitted into the LAN, otherwise, it just is ignored, and that is what should have happened here. I either don't understand your firewall or I don't understand the router. What firewall do you use?

It's simpler: you haven't fully understood NAT

I wrote a long explanation here but it wasn't good and clear enough and I would rather recommend reading RFC 1631 http://www.cse.ohio-state.edu/cgi-bin/rfc/rfc1631.html (I hope you are familiar with Request For Comments they're the whitepapers of the internet.

The packet got to me because as long as it knows where it want's to go (my ISP IP:port which is all the server will ever see as my sender address) it will get there (or since we're talking UDP: it will try to get there).

NAT does not equal a firewall! I've seen other people who think NAT protects them like a firewall and that is plain wrong, horribly wrong, NAT only protects somewhat against packets with nonvalid sender and addressing information (as in nonvalid IP's) and to a certain degree protects information about your local (behind the ISP connection) network (unless your ISP connecting router or your local machines are cracked).
A router using NAT does not stop or ignore any packet with an address not found in it's NAT memory/table/db, unless it is a packet without a correct local destination or local sender address.

Now I feel like I'm repeating myself but I'll say it once again: the request I got from - censored - port 4534 was addressed to me (obviously!) and there is absolutely no reason why it should be stopped by any NAT

I hope this clears up your misunderstandings about NAT (they were grave)

[Lucifer]

Z-man, I'll still provide the logs for the earlier install if you'd like to see them. I don't want to put anything on your plate if you have other things you'd rather focus on.

Just send him the latest log, it goes a week between rotations.

(Lucifer, you're not off the hook so easy )

Damn. *snap*

Umm, also, n54, yaza, swampy, and anybody else who's seen this, keep an eye out for it in the future. Now that swampland's been upgraded, it would be particularly useful to know if this doesn't happen anymore.

Swampy: Try tracerouting back and forth from each machine in your LAN to each machine in the LAN. (If you hadn't already figured that out from z-man's comment) That should give you an idea how fast the router is and if there are any real issues between how the router handles switching packets to each network interface, and also an idea of which network interfaces on each machine are faster than others. (I have had shitty network drivers bring an otherwise wonderful card down to a crawl, that was in Windows though. No issues with any Linux network drivers)

The reason I think running a 32-bit OS instead of a 64-bit OS is working for you now (and will continue to work for you) is because even though processors have hit 64-bit, we still have the memory bandwidth issues. I've seen numerous tests where variou speople benchmarked 32-bit and 64-bit applications on numerous processors (Opteron, Athlon, Lietanium, the PPCs, etc) including variations of the OS itself and there's not yet a clear advantage to running 64-bit applications on a 64-bit processor. It's not at all like the 16->32 bit change was, that change was awesome. We'd literally run out of bits. But for 99.9999999999% of applicatoins *now*, 32-bits is plenty. So in some specific applications, 64-bit is the clear winner. In others, 32-bit is the clear winner. But for general purpose operation on a 64-bit processor, there's no clear winner how many bits you need in your executables. And running 32-bit applications on an AMD 64-bit processor makes them scream in ways you can't get on even a Xeon, you know? (And AMD gave us a whole slew of registers in the Athlon64 line that really smooth things out)

/me steps off his soapbox

[Z-Man]

There are probably easier and more effective ways to launch attacks at either the server or its clients, just as you say.

Yes there are but that doesn't mean that somebody hasn't possibly found some weakness in armagetron and is trying to use or test it. We're not talking about DoS attacks here nor a brute force port scan but rather seeing if armagetron might have a weakness in what it allows. It's not a scenario we should dismiss imo.

Of course. But the packet your firewall blocked would not have been handled by Armagetron anyway since the destination port was wrong ( or else the firewall would not have blocked it ). I'm just weighting the chances that I did something stupid in the code against the chances a skilled hacker looking for an exploit is doing something stupid, and I think I win

...I guess we agree on this.

Absolutely.

Next step:
In absolutely no way should the server be able to change the local port the client uses. ... I somewhat get the impression that you're saying the server tries to "blindly search" (what sending 0.0.0.0:2809 would be) when it looses a connection.

The server never changes the port of the client, and it does not search blindly. But the client sometimes, very rarely, changes its source port all on its own, either when
1) the client socket breaks and it has to create a new one with a new port
2) the client is behind a NAT router and the router decides, since UDP is connectionless totally within specification, to change the source port it is maping AA packets to.
The packets sent by the client will still reach the server in case of such an incident, but unless something is done, packets from the server will not reach the client. Instead of terminating the connection, the server will modify the clients connection information so that the server's packets will reach the client: it changes the port it's sending to. It does not do portscans or even send to 0.0.0.0, simply because packets send to 0.0.0.0 don't get anywhere at all.

Fixing the clientside port does not work for two reasons: you would not be able to run two clients on one computer, and connections through a NAT router will not work because the client port gets altered by it.

I'm not sure what you're talking about here, or why you are talking about this. Anyway: any and all ports get altered by NAT using encapsulation totally outside our control and this is a fundamental part of what makes NAT what it is and is absolutely no problem nor does it have anything to do with the client and/or whatever ports it uses and or whether several instances of the program are running and connecting to the internet at the same time (which should be sorted out by the clients using different sockets automatically and has nothing whatsoever to do with NAT).

Sorry for not being clear, I was simply saying that the following does not work ( i.e. for exactly the reason you state ): Both server and client bind their socket to port 4535. Both server and client explicitly only send packets to port 4535. I was under the false impression that you thought the port 2809 from your firewall message should read 4535, which it would only if the client explicitly bound its socket to that port.

It's simpler: you haven't fully understood NAT

There are many forms of NAT. The one I ran across most of the time is called Masquerading by the linux kernel, Internet Connection Sharing by Microsoft and Overloading by Cisco ( see here ). The software solutions will usually handle packages not finding a match in the address translation table on the router PC itself. All standalone boxes I saw will drop unmappable packets.

I was just assuming your box does the same, but obviously it does not. It seems to forward unmappable packets to the LAN anyway and just broadcast them there. Probably perfectly valid behaviour, but I've never seen it before.

When a packet comes back from the destination computer, the router checks the destination address on the packet. It then looks in the address translation table to see which computer on the stub domain the packet belongs to. It changes the destination address to the one saved in the address translation table and sends it to that computer. If it doesn't find a match in the table, it drops the packet.

That's from the dynamic NAT section of the linked document, which is a bit different from overlapping NAT, because it multiplexes to many global IPs and does not multiplex ports. They don't say anything about packages not found in the ATT in Overloading NAT.

A router using NAT does not stop or ignore any packet with an address not found in it's NAT memory/table/db, unless it is a packet without a correct local destination or local sender address.

Sorry, but how can a packet coming from the Internet have a correct local destination? To the outside internet, all that is visible is the IP given to your router by your ISP. If my understanding is only a bit correct, then all packages coming to you from the outside should just have this IP as their destination. ( I'm aware of the existence of source routing where this assumption is not true, but think it is irrelevant here. )

[n54]

Well we're arguing on an empty stomach here, neither of us knows too much yet without the logs but at the danger of seeming to be bickering I'm going to reply because there seems to be a lot of misunderstandings we might as well clear up (don't reply before you've read it all, things fall somewhat into place).

Next step:
In absolutely no way should the server be able to change the local port the client uses. ... I somewhat get the impression that you're saying the server tries to "blindly search" (what sending 0.0.0.0:2809 would be) when it looses a connection.

The server never changes the port of the client, and it does not search blindly. But the client sometimes, very rarely, changes its source port all on its own, either when
1) the client socket breaks and it has to create a new one with a new port

Ok I don't see how this would result in a request from the server to open a new port... what you are describing here is the opposite and in this scenario my client would be initiating the switch not the server. If that was the case then it would already have been approved by my firewall on the first outgoing (from me to server) connection using the new port. The client should open the connection to the server and not the opposite. Now if it is programmed to happen the other way around we've found a bug there imo (but I doubt you've specifically programmed it so that the server should start connections).

2) the client is behind a NAT router and the router decides, since UDP is connectionless totally within specification, to change the source port it is maping AA packets to.

What makes you think this happened? Some info:
- NAT doesn't treat TCP and UDP differently
- NAT doesn't randomly decide on its own to change it's existing NAT table content this way (it would wreak havock on absolutely everything if it did).
- from the inside going out NAT will use the destination header to the server as supplied from the inside, it will only change the IP:port of the senders address to the address of it's outside interface. Now here is the crucial part: if it got a packet from the inside (client) to the server with the address - censored - port 2809 that would mean that the client used this destination address which in turn would mean that it was already seen and approved by the firewall because it was an outgoing connection from a running program approved to make any outgoing connections... this in turn means that there would never be a problem in the first place. This is how it works. Oh and btw - censored - port 2809 would be a new NAT table entry, it wouldn't automatically delete or replace the old one.

What actually had to happen to produce the warning message in the first place (there's no way around this):
- A packet with my ISP-provided IP arrived at my router << without doubt
- NAT translates the packets destination address (my ISP-provided IP) to a valid "inside" destination address (for example my local machine IP but in this case it used 0.0.0.0). << without doubt
- either way (wheter it was directly addressed to my local machine IP or using 0.0.0.0) my firewall (which intercepts all packets to my nic of course) recognised the packet as being a new incoming connection (i.e. unestablished/not a reply to an outgoing one) to a running program. << sorry to say it but this is without doubt as well

The packet that set of the warning was received through NAT in a perfectly normal way although I understand perfectly well why you though differently from your reply.

The packets sent by the client will still reach the server in case of such an incident, but unless something is done, packets from the server will not reach the client.

Why? This is not how networking works Any packets reaching the server will always contain the clients ISP-provided sender address and as long as the server uses this address when replying it will always be correctly (and transparently) translated by NAT to whatever it should be.

Now: if the server does not realize it has gotten a new connection (that the clients address port changed) then it will have trouble but I have difficulty seeing this as a likely scenario to happen as it should be no different from the server recognising an initial connection in the first place because the client changing port makes the whole thing a new seperate/additional connection. (How does it happen in the first place? Answered in RFC 768 http://www.faqs.org/rfcs/rfc768.html and RFC 760)

Instead of terminating the connection, the server will modify the clients connection information so that the server's packets will reach the client: it changes the port it's sending to. It does not do portscans or even send to 0.0.0.0, simply because packets send to 0.0.0.0 don't get anywhere at all.

First of all packets with 0.0.0.0 do get somewhere: it's the default "generic" network address. But yes it is true 0.0.0.0 would not be the destination address used to reach my client from the server as that indeed would not work (improper use). The destination address the server used would obviously be my ISP-provided IP. But since the sender of the packet (whether it was the server or somebody else) doesn't have my internal IP, and because this was an incoming connection request, and because my NAT allows it, and because the NAT did not find an existing "translation" for this request in its table, it used 0.0.0.0 as the local "destination".

Now for a small surprise: we actually have an explanation of the whole incident if what you say about the server "modifying the clients connection information" by "changing the port it is sending to" is true (and also something to fix imo).

Because what would happen if the server did this? It would end up with me getting a request from - censored - port 4534 to 0.0.0.0 port whatever

This is bad! This is the server trying to change the port of the client (contrary to what you said earlier). It should just disconnect me instead...

Now even if this is the case I think we should have confirmation of it from the logs or else it's not the reason imo.

Sorry for not being clear, I was simply saying that the following does not work ( i.e. for exactly the reason you state ): Both server and client bind their socket to port 4535. Both server and client explicitly only send packets to port 4535. I was under the false impression that you thought the port 2809 from your firewall message should read 4535, which it would only if the client explicitly bound its socket to that port.

Ah good No need to say sorry: it's always very easy to misunderstand each other when discussing details, and remember that I'm in kind of a "reduced state" so not only does my replies take forever to finish but also they're not perfectly formulated (it's been years since I taught networking).

It's simpler: you haven't fully understood NAT

There are many forms of NAT. The one I ran across most of the time is called Masquerading by the linux kernel, Internet Connection Sharing by Microsoft and Overloading by Cisco ( see here ). The software solutions will usually handle packages not finding a match in the address translation table on the router PC itself. All standalone boxes I saw will drop unmappable packets.

Yes, or rather: there are many implementations of NAT. One thing that is important to realize is that NAT is environment specific and configurable (in some situations you would never allow NAT to use 0.0.0.0, 255.255.255.255 or similar but in others you would).

I was just assuming your box does the same, but obviously it does not. It seems to forward unmappable packets to the LAN anyway and just broadcast them there. Probably perfectly valid behaviour, but I've never seen it before.

0.0.0.0 is not a broadcast (that would be 255.255.255.255) even though the effect is similar they differ in use. Anyway, yes it can be valid: a small consumer (as opposed to business) router (and NAT) does not know or remember what is going on and so if it get's a packet inbound that it doesn't find where should go in it's NAT table or routing table then it will try it's best at delivering the packet as it can't decide on its own wheter it is a valid connection or not (as long as it is validly formed IPs and unless the router, NAT or any packet filter is configured not to allow it of course).

When a packet comes back from the destination computer, the router checks the destination address on the packet. It then looks in the address translation table to see which computer on the stub domain the packet belongs to. It changes the destination address to the one saved in the address translation table and sends it to that computer. If it doesn't find a match in the table, it drops the packet.

That's from the dynamic NAT section of the linked document, which is a bit different from overlapping NAT, because it multiplexes to many global IPs and does not multiplex ports. They don't say anything about packages not found in the ATT in Overloading NAT.

This is NAT in the context of large business environments imo (large stub networks if you like) and in these situations it would be insane to not drop the packet, anyway this cleared a lot of stuff up: we've been talking about slightly different things

A router using NAT does not stop or ignore any packet with an address not found in it's NAT memory/table/db, unless it is a packet without a correct local destination or local sender address.

Sorry, but how can a packet coming from the Internet have a correct local destination? To the outside internet, all that is visible is the IP given to your router by your ISP. If my understanding is only a bit correct, then all packages coming to you from the outside should just have this IP as their destination. ( I'm aware of the existence of source routing where this assumption is not true, but think it is irrelevant here. )

Ah blarg lol I totally messed up trying to include mentioning the need for validly formed IPs in an incredibly cumbersome and confusing manner lol mea culpa I thought I was tired now but I must obviously have been worse writing that

It will be interesting to see wheter the logs indicate a server attempt at a new connection to the client or not. If it is we have some UDP fixing to do and although I'm no programmer there are meant to be transparent port switching abilities in UDP we could try to implement in a way that doesn't set off all the alarms

Edit: reading through my next last reply makes me shudder, you'ld think I was drunk writing it there are so many imprecise-to-the-point-of-becoming-wrong uses of words and sentences (and no I wasn't drunk) really sorry about that: I've got to take most of the blame for any misunderstandings

[Z-Man]

Ok, no more quoting. It just messes things up

I think you are absolutely right with the analysis what happened on your side: the packet came in and really was addressed to the wrong port. Your Router forwarded it to the generic address/broadcast ( to distinguish it, we'd need the netmask, no? ) because it did not belong to an already established connection, your personal firewall blocked it because it did not belong to a connection. The question is now, why was it sent?

I had a chance to look at the logs in the meantime, there were no indications of port changes ( AA spews out messages about it ). And I looked at the source again and noticed that the code handling the port change actually never gets called at all, so we may as well take it out.

Neither 1) nor 2) from my post can have happened here, both would be handled correctly by your router/firewall. I know 2) does not happen with well-behaved NAT routers, but it really did happen in early testing. I don't know with what router, though, it may have been an early version of the linux kernel masquerading or a hardware LAN-to-ISDN-router. Without the compensating code, AA would not have run in these tests. Perhaps I did something else wrong these days, who knows, but port-changing code itself is not a bug. It serves precisely the purpose of sending the packets back to the IP and port that they came from, even if the port changes for any reason.

So, now whe have to solve the riddle why the ominous packet was sent. The code that we now know was not called is the only place where an already existing connection gets a new port. An involontary memory overwrite could alter the port, too, but its unlikely that it would leave the IP unchanged. It could also be the server port that was changed, but that does not happen even if the server socket breaks. That leaves four possibilities I can think of right now:
- a provoked memory overwrite by a hacker. It would strike me as quite unusual why anyone would stop at changing a connection port when he can crash the server or alter scores, then.
- a packet sent by someone else with a spoofed source; that leaves the question how he got your IP and why he was not using the correct port to disturb your play. Also, since the real connection to the server broke down and it would not have in this scenario, I think we can safely exclude it.
- something really strange happening in swampy's network
- the network control socket.

I'll open another thread for the network control socket issue for developers only. Whoops. Swampy/Lucifer: is the router that connects the two servers with the net configured to let all incoming connections/packets through to the servers, like n54's router is, or do you disable connections by default and only forward the AA ports you actually use? I stronly suggest you use the latter possibility.

About the logs:
Yaza timed out several times in them, but I did not see n54 timing out. When did this happen?

The only other curious thing I saw was two massive simultaneous connection losses of all clients while 0.2.7.0 was running.

[n54]

Yes there seems to be some inherent universal quote limit where it confuses (at least me) more than it clarifies lol

0.0.0.0 and 255.255.255.255 are special in that they work totally independent of any netmasking or subnetting: "superips" (however only inside the network/subnetwork they're used in) RFC 3330 has a nice summary table at page 3 http://www.rfc-editor.org/rfc/rfc3330.txt

I got one of the strange packets at 3/19/2005 4:51:53 AM (iirc this was the arrival of the first one). Since then I've increased firewall logging so I'll have slightly more info if it happens to me again (like how much time passed between the arrival of each request, precise number of requests etc.)

We have at least removed some possible explanations, maybe we'll find new ones as Swampy and Lucifer work on the lag problem or if more people experience the strange packets. If nothing similar happens maybe we should put it aside and just keep it in mind for the future.

I haven't played much since this happened because of all of the forum posting (lol tells you something about how little I get done in a day) but I began fiddeling a bit with Ethereal because I wondered if I should let it run while playing in case something happened again on the extremely odd chance that something useful might be found by looking at any future strange packets contents (and to compare them to normal armagetron server packets, especially during connection establishing).

The likelyhood that anything valuable should be found this way on a client is microscopic but since I never have any other internet connections up while playing armagetron it makes for very readable logs and doesn't interfere with the game so it doesn't cost me anything (except remembering to turn it on). And I would trust Etheral to be more precise than my firewall since it "captures" the packets themselves

If any at swampys want to maybe they could run it there to see if they find anything strange? Their address is http://ethereal.com/ and they have both source, binaries, and well written compiling/dependency/installation documentation for just about any operating system. I haven't been using it in a while so I'm just reading the (somewhat small) user manual for now but they have searchable mail archives etc. and it's pretty easy to get started sniffing your nic. Afaik a few linux distributions includes it so you might already have it ready to go.

[Z-Man]

Plus, quoting increases ones inclination to nitpick

The last log entry I have here from you is from 03/18/22:07. You say "sorry this is unplayable for me cya all" and quit. The server was running 0.2.7.0 then on Mandrake 64. Can you remember whether this was the right server and before the incident? Taking the various possible US/Europe timeshifts into account, I cannot decide it.

[n54]

That is me leaving the Swampy Teams server. I got the packet later playing on the other Swampy (non-team) server.

Have you gotten the log from both servers? If you look through the log of the other server (the non-team one) I would expect that you at least can see me saying "hi" or similar sometime before 3/19/2005 4:51:53 AM (GMT +1)

I'm in GMT +1 and afaik Swampy's are in GMT -5 so normalized to GMT we get.
GMT 0 03/19/03:07 my log entry at Swampy Teams before I leave
GMT 0 03/19/03:51 the strange packet arrives while playing at Swampy's non-team

[Z-Man]

No, I only have the logs of the teams server. I asked Swampy by mail to send me the other logs, too. I'll report back when I get them.

[Z-Man]

Ok, got the right logs. They read:

Code: Select all

Mar 18 22:44:40: n54: i lost the numbers i used for the old one
( the last thing you said, after some rounds: )
Mar 18 22:50:25: Seleucus core dumped Squirrel for 1 points.
Mar 18 22:50:30: aLaGaN core dumped n54 for 1 points.
Mar 18 22:50:39: Seleucus core dumped Woota for 1 points.
Mar 18 22:50:41: Seleucus core dumped aLaGaN for 1 points.
Mar 18 22:50:43: Woota: uhhhh ok
Mar 18 22:50:44: FaTaLz core dumped Zerox- for 1 points.
Mar 18 22:50:46: Zerox-: ..
Mar 18 22:50:46: FaTaLz: lol
Mar 18 22:50:49: FaTaLz core dumped cotton candy for 1 points.
Mar 18 22:50:54: FaTaLz core dumped Seleucus for 1 points.
Mar 18 22:50:55: Winner:FaTaLz
Mar 18 22:51:06: Admin: Player Stats located at http://breakfast.servegam
e.com (combined stats there also)
Mar 18 22:51:11: Ping charity changed from 155 to 153.
Mar 18 22:51:11: Deleting objects...
Mar 18 22:51:16: Deleting grid...
Mar 18 22:51:16: http://breakfast.servegame.com
Mar 18 22:51:26: User 8 timed out.
Mar 18 22:51:26: Killing user 8, ping 0.234801.
Mar 18 22:51:26: cotton candy left the game.
Mar 18 22:51:26: User 2 timed out.
Mar 18 22:51:26: Killing user 2, ping 0.102762.
Mar 18 22:51:26: Squirrel left the game.
Mar 18 22:51:27: User 3 timed out.
Mar 18 22:51:27: Killing user 3, ping 0.0940358.
Mar 18 22:51:27: FaTaLz left the game.
Mar 18 22:51:27: User 4 timed out.
Mar 18 22:51:27: Killing user 4, ping 0.106498.
Mar 18 22:51:27: Seleucus left the game.
Mar 18 22:51:28: User 7 timed out.
Mar 18 22:51:28: Killing user 7, ping 0.327952.
Mar 18 22:51:28: n54 left the game.
Mar 18 22:51:28: User 1 timed out.
Mar 18 22:51:28: Killing user 1, ping 0.0862023.
Mar 18 22:51:28: Woota left the game.
Mar 18 22:51:28: User 5 timed out.
Mar 18 22:51:28: Killing user 5, ping 0.107917.
Mar 18 22:51:28: aLaGaN left the game.
Mar 18 22:51:28: User 6 timed out.
Mar 18 22:51:28: Killing user 6, ping 0.157359.
Mar 18 22:51:28: Zerox- left the game.
Mar 18 22:51:28: Time:     83.4474 seconds
Mar 18 22:51:28: Sent:     1009590 bytes in 13390 packets (12098.5 bytes/
s)
Mar 18 22:51:28: Received: 251438 bytes in 5594 packets (3013.13 bytes/s)
Mar 18 22:51:28: Timestamp: 2005/03/18 22:51:28
Mar 18 22:51:28: Creating grid...
Mar 18 22:51:28: Ping charity changed from 153 to 100100.
Mar 18 22:51:28: done!
Mar 18 22:51:29: go (round 8 of 40 )!
Mar 18 22:51:29: Killing user 0, ping 10.
Mar 18 22:51:29: Nobody there. Taking a nap...

This looks exactly like the other massive simultaneous connection losses I noticed in the other logs. They happen several times a day, every time right between rounds. From the logs, they may well be one of the exploits made public, but that would not explain the strange packets. The timing, I'd say, strongly suggests a bug. The logs from the other server, which was running 0.2.7.1 for some while and did not show these problems, let me hope that it magically went away. Otherwise, I'll have to see whether anything unusual may happen between rounds.

I'll post a request for the other server admins who might be running 0.2.7.1 to scan their logs in another thread.

[n54]

Ok that's the right log. Just a small timeline:

51:29 the server started napping
51:53 the weird packet from somewhere using the server IP:port as sender address

My ping is about 180ms real (icmp ping) and 250ms armagetron and the difference between the timestamps above are 240ms.

I can't think of anything else we can do unless someone gets & logs similar packets with port requests as the ones Yaza, Swampy and I got.