Wiki taken down for a bit.
Moderator: Lucifer
Wiki taken down for a bit.
Bad timing. Sorry. Looks like someone is using the server to farm bitcoins. Should be back up in an hour or so.
Re: Wiki taken down for a bit.
Wiki back up. The code injection came over my wife's website, the datestamp says it started just two days ago. The attacker was able to execute arbitrary code as the webserver's user www-data. In principle, they would have been able to peek around and steal the wiki user database with password hashes and email addresses. My educated guess is that they were not interested in that. If you want to sneakily steal passwords and make the most of them, you don't leave behind a process that causes 100% CPU load. Yeah, it was bitcoin mining.
I'll investigate further, obviously, and we will make a more thorough announcement. In the meantime, I'd say that only if your password on the Wiki is the same as your home banking password, you should change both. And reconsider your password reuse policy
Sorry this happened again. Does anyone know a good way to prevent one site jeopardizing data of another running on the same server?
I'll investigate further, obviously, and we will make a more thorough announcement. In the meantime, I'd say that only if your password on the Wiki is the same as your home banking password, you should change both. And reconsider your password reuse policy
Sorry this happened again. Does anyone know a good way to prevent one site jeopardizing data of another running on the same server?
Re: Wiki taken down for a bit.
If I make the assumption you're using Apache, you could use mod_privileges. That will let you run the virtual hosts under different users. Also, you shouldn't make the files globally available. You can probably chroot on top of that, locking them to their directory. I believe they could still screw with globally accessible files though. So, guest read permission might have to go away.Z-Man wrote:Sorry this happened again. Does anyone know a good way to prevent one site jeopardizing data of another running on the same server?
I would also suggest helping her realize why her site was vulnerable. It's not too hard to protect a site from injections. Keep the server and database safer that way.
Which brings another thought to mind. Don't use the same database user for each site. Might be fairly obvious, but I've come across a lot of stupid things people do, so I thought it might be worth mentioning. If there is anything they can inject some SQL, it will be open season on everything you've got. May as well limit them a little bit.
All that really comes to mind. Maybe limit what bash commands they can use as well. A user made specifically for a website doesn't really need to be able to do a lot with bash, unless you've got something special set up for it. Remember that exec() and shell_exec() is going to be limited with what you edit there, but that's the goal.
Re: Wiki taken down for a bit.
For that, I use a different user for each website and don't give them permission to access each other. The only way I can think of that might be better would be experimenting with chroots or running each server in a virtual machine on that main server.
Re: Wiki taken down for a bit.
*reading that later*
Apparently, this is now a sport. A different minerbot is active. They are using straight simple apache/php vulnerabilities as far as I read.
Apparently, this is now a sport. A different minerbot is active. They are using straight simple apache/php vulnerabilities as far as I read.
Re: Wiki taken down for a bit.
You could just knock her site offline until you get it taken care of? Also, make sure there's no cronjobs or startup scripts that are going to respawn all of it.Z-Man wrote:*reading that later*
Apparently, this is now a sport. A different minerbot is active. They are using straight simple apache/php vulnerabilities as far as I read.
Re: Wiki taken down for a bit.
Yes. Steps taken:
1. Shut down web server completely, investigate.
2. Shut down apparently affected site, turn on web server again.
3. Kill bot. Yes, it had a crontab going.
4. Find new bot at new path with new crontab.
5. Cleaned that. www-data can no loger edit its crontab, what appears to be the entry exploit disabled.
The injected code on my wife's site probably was not the initial exploit, rather it was left there as a backdoor for the future.
So, second bot gone, acces path shut with some boards and nails.
Urgh. Looks like there is no better time to get a bigger server than now, doesn't it?
Ladle recordings will suffer a bit this time, too.
1. Shut down web server completely, investigate.
2. Shut down apparently affected site, turn on web server again.
3. Kill bot. Yes, it had a crontab going.
4. Find new bot at new path with new crontab.
5. Cleaned that. www-data can no loger edit its crontab, what appears to be the entry exploit disabled.
The injected code on my wife's site probably was not the initial exploit, rather it was left there as a backdoor for the future.
So, second bot gone, acces path shut with some boards and nails.
Precisely the kind of thing I was looking for.Light wrote:If I make the assumption you're using Apache, you could use mod_privileges. That will let you run the virtual hosts under different users.Z-Man wrote:Sorry this happened again. Does anyone know a good way to prevent one site jeopardizing data of another running on the same server?
That's a given once it is actually possible.Light wrote:Also, you shouldn't make the files globally available.
Would that be possible inside the apache process? I mean, sure, one can chroot the whole server, but can the server chroot its instances responsible for the different sites?Light wrote:You can probably chroot on top of that, locking them to their directory. I believe they could still screw with globally accessible files though. So, guest read permission might have to go away.
I'm the admin, so any techical stuff is entirely my fault. After our last attack, all regular injection paths were eliminated by setting minimal rights: no script execution in data directories, no write permission to www-data in script directories. All of that does not help if one postpones OS distribution upgrades, though (security updates were running in a cronjob, but they only go so far). Problem: It's a VPS, and you can't upgrade the kernel without a full reinstallation. I already upgraded the OS anyway, so its kernel is one distribution version down... seems to work OK, but I should not stretch that further.Light wrote:I would also suggest helping her realize why her site was vulnerable. It's not too hard to protect a site from injections. Keep the server and database safer that way.
Yeah Only the wiki is using the database, so no problem there.Light wrote:Which brings another thought to mind. Don't use the same database user for each site. Might be fairly obvious, but I've come across a lot of stupid things people do, so I thought it might be worth mentioning. If there is anything they can inject some SQL, it will be open season on everything you've got. May as well limit them a little bit.
Not a bad idea, would have stopped the first bot. Maybe not the second one, that seemed to be purely using php to do its work. But the same principle applies: it was exploiting a system that was not in fact needed.Light wrote:All that really comes to mind. Maybe limit what bash commands they can use as well. A user made specifically for a website doesn't really need to be able to do a lot with bash, unless you've got something special set up for it. Remember that exec() and shell_exec() is going to be limited with what you edit there, but that's the goal.
Urgh. Looks like there is no better time to get a bigger server than now, doesn't it?
Ladle recordings will suffer a bit this time, too.
Re: Wiki taken down for a bit.
It depends on the possible code injections. If you can inject PHP, then it could be the source of just about anything given the right permissions.Z-Man wrote:The injected code on my wife's site probably was not the initial exploit, rather it was left there as a backdoor for the future.
Yes, but you have to be using a different user for the virtualhost. A quick Google search should be able to help you out with this.Z-Man wrote:Would that be possible inside the apache process? I mean, sure, one can chroot the whole server, but can the server chroot its instances responsible for the different sites?
That sounds like a huge pain in the ass. Running a normal server you can just reboot.Z-Man wrote:It's a VPS, and you can't upgrade the kernel without a full reinstallation.
I hate moving servers, so that never sounds like fun to me. lolZ-Man wrote:Urgh. Looks like there is no better time to get a bigger server than now, doesn't it?
You could always host them on any file sharing site until you're ready to host them yourself. Or maybe even as attachments here? Unless, maybe you were talking about you actually recording them, in which case I have no useful input.Z-Man wrote:Ladle recordings will suffer a bit this time, too.
Re: Wiki taken down for a bit.
That was the case. It was this one: http://www.exploit-db.com/exploits/29290/Light wrote:It depends on the possible code injections. If you can inject PHP, then it could be the source of just about anything given the right permissions.Z-Man wrote:The injected code on my wife's site probably was not the initial exploit, rather it was left there as a backdoor for the future.
Yep. I'm pondering getting a real server where I can really do as I please or falling back to shared web hosting where those issues are taken care of for me. Probably loses too much flexibility, though. I bet a shared hoster where you can insulate YOUR sites from one another will be hard to find.Light wrote:That sounds like a huge pain in the ass. Running a normal server you can just reboot.Z-Man wrote:It's a VPS, and you can't upgrade the kernel without a full reinstallation.
Hosting is fine They are on my home server. Making them for this ladle, however, collided a bit with the damage control work. Indirectly. Both collided with eating from opposite directions.Light wrote:You could always host them on any file sharing site until you're ready to host them yourself. Or maybe even as attachments here? Unless, maybe you were talking about you actually recording them, in which case I have no useful input.Z-Man wrote:Ladle recordings will suffer a bit this time, too.
-
- Round Winner
- Posts: 214
- Joined: Sat Jan 01, 2011 6:05 pm
Re: Wiki taken down for a bit.
your intelligence blows my mind
Re: Wiki taken down for a bit.
Don't be overly impressed. Google did most of the hard work (well, and I guess the guys discovering the vulnerability and the other guys writing the forum posts google turned up), I just plugged in the odd POST requests that showed up in the webserver logs. Took me a long time to find out where user specific crontabs are managed
Edit: Heh, and sometimes, you find this helpfully documented sploit script on your system:That one didn't work. Would have been a root right escalation. Phew.
In total, there were at least FOUR completely different miner bots active, each killing the one that came before. Definitely a sport. In addition to the exploit attempt above, the logs show one simple direct su attempt. In addition to CPU time, they used quite a bit of bandwidth, 150 gigs a day. Nothing I have seen so far has been even a tiny bit stealthy. Like, bot 3 brought a full copy of boost with it and compiled itself. Still, if I were to gain parital control over a system and make some noise by neccessity, but wanted to leave a backdoor anyway, I would also leave a second, easy to discover backdoor for the admin to find and feel smug about... I'll definitely compare all php scipts with what is supposed to be there.
Edit: Heh, and sometimes, you find this helpfully documented sploit script on your system:
Code: Select all
#!/bin/sh
# A cleaner implementation of: http://www.exploit-db.com/papers/15311/
# This script uses Perl as a setuid() provider, but any lang will do
...
In total, there were at least FOUR completely different miner bots active, each killing the one that came before. Definitely a sport. In addition to the exploit attempt above, the logs show one simple direct su attempt. In addition to CPU time, they used quite a bit of bandwidth, 150 gigs a day. Nothing I have seen so far has been even a tiny bit stealthy. Like, bot 3 brought a full copy of boost with it and compiled itself. Still, if I were to gain parital control over a system and make some noise by neccessity, but wanted to leave a backdoor anyway, I would also leave a second, easy to discover backdoor for the admin to find and feel smug about... I'll definitely compare all php scipts with what is supposed to be there.