Weeks ago, I came back from work at probably 18:30 P.M.
So far, it's all good. right?
Wrong!
Around 19:30 I got a call that says : "leave anything you do and come to ******* (Vendor name removed - will be called "Gas-Supplier" from now on) Headquarter, there's a worm spreading and it's rebooting any computer of their operational network"
Let me stop here for a second and explain what kind of type "Gas-Supplier" is, "Gas-Supplier" is one of the biggest fuel station in the country. It has more than 200 stations.
So, I've arrived quickly around (20:00 p.m) to see that everybody their is stress. Every person who tries to get fuel in one of these gas-stations, cannot do so, because the computer that responsible to it, gets rebooted every 1 minute.
The first step was checking which worm was spreading and preventing it from further spread. Firewall had been put between every station to each-other on ports 445/135.
Quick scan of the station from remote showed up that those computers are Windows 2000, SP4, Not fully patched. In-fact, they were vulnerable to almost any remote attack to SMB/RPC.
You don't have time to patch [it crushes every minute, till every services like workstation service or other services for remote control [RPC] it takes like 20 seconds, and crushes after 30-40 seconds], them from remote (and they are all around the country, so going single station at the time isn't practical).
That was the time I understood I'm going to be there all night. The CEO, a person who earns millions every-year, sat behind me for sometime and hold his head and I could have hear his self-worries, the fear in his eyes that this will be published the next day.
So, I've tried to figure out remotely what was the problem, it didn't work well because of the crashes I've written above and the slow connection between the VPNs.
I went with 2 workers of "Gas-Supplier", to the closest station, and we checked which virus is causing the computers to crash.
Quickly, an anti-virus on safe-mode showed that it's infected with Conficker (B/C), Kido/ DownUp or any other name that applies to this worm.
So, now we know which worm it is. Should be easier to patch, right? well.. not really.
The conditions to patch it from remote were extremely bad. I've built scripts that copy Kaspersky's Conficker removal tool, that needs to run for a few minutes, but the computer shuts after 1 minutes, because Conficker wasn't written good enough for Windows 2000 SP4 and makes the services.exe crash.
One of the programmer's for "Gas-Supplier" gave me a tool that message-pump the Shutdown event so it basically gave me 5:00 minutes before services.exe crashes the computer and reboots it.
This is a message to conficker writers, please next time, don't crash services.exe on Windows 2000. If it's not safe for you to inject code, don't! You ruined my night :)
So I've remotely put some start-up scripts do dis-infect the computers using the Kaspersky conficker removal tool (which is much better than Norton's removal tool in my opinion) and another one to patch the affected computer.
Did it all night long, on few computer simultaneously (I had to catch it when they were up and not rebooting).
done by 6.00 A.M.
Statistics :
0 News reports about it.
1,000,000$++ made since by that "Gas-Supplier"
2 coffee
1 double espresso
2 days off afterward
1 weird-nice experience.
Cheers.
So far, it's all good. right?
Wrong!
Around 19:30 I got a call that says : "leave anything you do and come to ******* (Vendor name removed - will be called "Gas-Supplier" from now on) Headquarter, there's a worm spreading and it's rebooting any computer of their operational network"
Let me stop here for a second and explain what kind of type "Gas-Supplier" is, "Gas-Supplier" is one of the biggest fuel station in the country. It has more than 200 stations.
So, I've arrived quickly around (20:00 p.m) to see that everybody their is stress. Every person who tries to get fuel in one of these gas-stations, cannot do so, because the computer that responsible to it, gets rebooted every 1 minute.
The first step was checking which worm was spreading and preventing it from further spread. Firewall had been put between every station to each-other on ports 445/135.
Quick scan of the station from remote showed up that those computers are Windows 2000, SP4, Not fully patched. In-fact, they were vulnerable to almost any remote attack to SMB/RPC.
You don't have time to patch [it crushes every minute, till every services like workstation service or other services for remote control [RPC] it takes like 20 seconds, and crushes after 30-40 seconds], them from remote (and they are all around the country, so going single station at the time isn't practical).
That was the time I understood I'm going to be there all night. The CEO, a person who earns millions every-year, sat behind me for sometime and hold his head and I could have hear his self-worries, the fear in his eyes that this will be published the next day.
So, I've tried to figure out remotely what was the problem, it didn't work well because of the crashes I've written above and the slow connection between the VPNs.
I went with 2 workers of "Gas-Supplier", to the closest station, and we checked which virus is causing the computers to crash.
Quickly, an anti-virus on safe-mode showed that it's infected with Conficker (B/C), Kido/ DownUp or any other name that applies to this worm.
So, now we know which worm it is. Should be easier to patch, right? well.. not really.
The conditions to patch it from remote were extremely bad. I've built scripts that copy Kaspersky's Conficker removal tool, that needs to run for a few minutes, but the computer shuts after 1 minutes, because Conficker wasn't written good enough for Windows 2000 SP4 and makes the services.exe crash.
One of the programmer's for "Gas-Supplier" gave me a tool that message-pump the Shutdown event so it basically gave me 5:00 minutes before services.exe crashes the computer and reboots it.
This is a message to conficker writers, please next time, don't crash services.exe on Windows 2000. If it's not safe for you to inject code, don't! You ruined my night :)
So I've remotely put some start-up scripts do dis-infect the computers using the Kaspersky conficker removal tool (which is much better than Norton's removal tool in my opinion) and another one to patch the affected computer.
Did it all night long, on few computer simultaneously (I had to catch it when they were up and not rebooting).
done by 6.00 A.M.
Statistics :
0 News reports about it.
1,000,000$++ made since by that "Gas-Supplier"
2 coffee
1 double espresso
2 days off afterward
1 weird-nice experience.
Cheers.


