Approximately 4 hours ago, Ozblogistan began to receive a very high level of traffic. During this period my performance tracking tools stopped working so it is difficult to tell how much traffic was received during the period.
A few minutes ago I kicked over our web server to reset all connections, which resolved this matter last time. This time too, apparently.
There are three possibilities:
- Misconfiguration of the web server software
- A badly behaved indexer program
- A denial of service attack
I am confident that the server is correctly configured, but I cannot eliminate the possibility that I am misinterpreting the available performance records. That kicking the server over “fixes” the problem might suggest this possibility.
If there was a badly behaved indexing program crawling all Ozblogistan sites, it should have turned up in my Cloudflare dashboard which, so far, it has not.
The odd man out in the Ozblogistan network is Catallaxy Files. The Cat is the only site currently not “behind” Cloudflare’s protective service. If we were receiving the loving ministrations of a badly behaved indexing program, then that program was exclusively crawling the Cat.
The unhappiest possibility is that this was a denial-of-service attack; again, directed against the Cat. Sometime in the next few days I expect that Catallaxy Files will be placed behind Cloudflare along with the other Ozblogistan sites and that this will prevent any such attacks from being successful.
Well, that’s it, folks. Please resume your usual blogging and commenting.
If you see this message, you’re on the new servers at the original providers.
Moses ain’t got nothing on me.
That you can see this post means you’re on the new server. Welcome.
There will be glitches — duplicates and the like.
Sorry about the disruption, but I hoped you all enjoyed the kittens and the dalek.
Last night the Ozblogistan network slowed down and then began to “yo-yo” — stop working and then reappear cyclically.
I have been becoming increasingly dissatisfied with WPEngine’s reliability and service. Tis was the straw that broke the camel’s back. So I began a crash move of the network to a new hosting service, Page.ly.
That is why the site was unavailable last night and this morning, and why you first saw kittens and then a little moving truck.
Page.ly’s service has already shown itself to be much better than WPEngine’s; however they have reported back that migration is going slowly because the WPEngine servers periodically drop the connection they’re using to download files.
Their advice was to switch back to WPEngine’s servers for today. Any posts or comments from today could then be migrated using the WXR import/export facility built into WordPress.
So here we are, temporarily back at WPEngine. Tonight I and the Page.ly staff will try the move again.
A few hours ago WPEngine finally traced the problem back the recent comments widget on Skepticlawyer and Catallaxy Files. I’ve disabled those for now. (Amazing how much their service has improved, now that I’m leaving).
Hello everyone, your friendly Ozblogistan Tyrant here.
Ozblogistan was moved to new servers on Monday night by some subcontractors. The ensuing few days have been a mess because the move was only incompletely successful. In particular, bloggers have been unable to log into the wordpress backend to write posts or moderate comments.
This morning a threw a switch and it appears as though we are slowly coming back online. I’m very sorry for the disruption.
Hello all, your friendly Ozblogistan Tyrant here.
This weekend I will be performing one of the biggest changes made to Ozblogistan in several years.
I am moving the site from its current home to a new one.
The new service is a company specialising in WordPress hosting. They will be taking responsibility for all the scut-work of making the site go.
Once the move is finalised, I expect that Ozblogistan will be snappier than it is now. I also won’t have to worry about fixing security problems, doing backups, finding out why plugin X has stopped working, working out why the site has suddenly died and so on. These will now be responsibilities I can delegate to the providers.
This has been in the works for some time. While I was a student I could afford the time it takes to keep Ozblogistan working relatively smoothly. As a professional I no longer have that time. So it makes sense to hire other professionals to do it for me.
The security problem we had a few weeks ago (and which is still causing false positives) was the straw that broke the camel’s back.
Moving providers is never pretty. Things are going to break this weekend. And various things will probably appear broken into the new week. It’s all for the good. Ozblogistan will shine out ever brighter.
Update Saturday 2:17PM WST: Failure. I will probably have to try again on Monday night. I’ve re-enabled comments.
Hello all, your friendly Ozblogistan Tyrant here, abusing my multisite posting powers.
This morning I received two independent reports of trojan warnings being given for two different Ozblogistan websites.
After investigation, I have determined that the server was automatically compromised, presumably by a brand new attack (since we just 2 days ago updated to WordPress 3.4.1), and a trojan inserted into various parts of WordPress.
I have identified and replaced the affected files with clean copies, and you should see no more warnings.
Those who want the gruesome details can learn more.
Hello everyone, it’s your friendly Ozblogistan Tyrant here.
I plan to migrate our servers to a different data centre tonight. There will be some disruption during the move, and it may take time for your DNS records to be updated.
We’re moving from the Fremont data centre, which has turned out to be about as reliable as a Ford Pinto, to the Dallas data centre (I look forward to your big-hats-and-oil jokes).
Hello all, Jacques here.
John’s site is largely migrated. johnquiggin.com is being served from the Ozblogistan servers. You should notice a performance improvement.
Comment in this thread if you see any problems.