Site outage 19th/20th March

[Chief]

Again, our apologies for this outage.

All major catastrophes are usually as a result of multiple points of unlikely failure, and this catastrophe is no exception.

Rest assured there was no mischief, hacking, data breach or anything like that so your data is still safe.

Just one small error being missed at one point. The symptoms of the resulting, seemingly unrelated issue are fixed but the wrong cause is blamed.

That causes another issue. Then finally the whole things closes up shop for the holidays.

I think at one point we had four people working on it until one person said “let me try something”. This was followed by facepalms all around. Then you wait for someone on the other side of the world to read their email, and then you start fixing it so it never happens again.

Will we ever go offline again in the future? Of course. But this particular risk is now being managed to within an inch of its life.

So just sit back and wait for the next highly unlikely series of failures that nobody saw coming.


* And read Taleb on managing risk. It will make you feel better.

 

[Bomberboyokay]

Didn't pay the server bill?

 

[Art Corvelay]

Rest assured there was no mischief, hacking, data breach or anything like that so your data is still safe.

I thought we were in the midst of a hostile takeover by Sooz. Good to hear it was just a regular “feature”

 

[Chief]

Didn't pay the server bill?

Not this time but that’s another thing that could go wrong - bloody bank blocks the transaction every month until we manually approve it. We’ve only been paying the same monthly bill for about 6 years now.

 

[boncer34]

Not this time but that’s another thing that could go wrong - bloody bank blocks the transaction every month until we manually approve it. We’ve only been paying the same monthly bill for about 6 years now.

Which bank?

 

[SHAKESPEARE]

Cheers Chief.

Your work has been noted. Pat on the back

 

[BotsMaster]

I think if the site goes down again we should be redirected to BigCricket.

 

[Chief]

Which bank?

The criminal bank.

 

[OffTheBit]

The Malaysian Airlines of football forums

 

[baz_machine]

My boss thanks you for the outrage Chief.

I was 648% more productive at work today.

 

[Isaac Cumming No 1]

Again, our apologies for this outage.

All major catastrophes are usually as a result of multiple points of unlikely failure, and this catastrophe is no exception.

Rest assured there was no mischief, hacking, data breach or anything like that so your data is still safe.

Just one small error being missed at one point. The symptoms of the resulting, seemingly unrelated issue are fixed but the wrong cause is blamed.

That causes another issue. Then finally the whole things closes up shop for the holidays.

I think at one point we had four people working on it until one person said “let me try something”. This was followed by facepalms all around. Then you wait for someone on the other side of the world to read their email, and then you start fixing it so it never happens again.

Will we ever go offline again in the future? Of course. But this particular risk is now being managed to within an inch of its life.

So just sit back and wait for the next highly unlikely series of failures that nobody saw coming.


* And read Taleb on managing risk. It will make you feel better.

Glad you explained. I was beginning to wonder if this was your "Ivan the Terrible" moment. A taste of life without BF followed by a sharp increase in revenue raising.

Not that I'm complaining $20 is is a pittance for the use I get.

 

[M Malice]

It was the Russians I tell ya, Chief is in collusion.

 

[Biggy_Boy]

I thought Telstra had blocked the site.

 

[TheGreatBarryB]

Kept calling triple-0 and they weren’t very helpful at all...actually quite rude.

 

[Frank Gallagher]

Outage was an outrage!!

 

[TheGhostOfJohnBelushi]

Facebook goes down. Bigfooty says, "Hold my beer."

 

[raskolnikov]

It was the Russians I tell ya, Chief is in collusion.

I had nothing to do with it.

 

[Deaneus!]

The criminal bank.

Can you be more specific?

Was far more productive at work today, and hey no one was hurt probably... and what a time to go on the blink

Well done Chief

 

[NSWCROW]

Again, our apologies for this outage.

All major catastrophes are usually as a result of multiple points of unlikely failure, and this catastrophe is no exception.

Rest assured there was no mischief, hacking, data breach or anything like that so your data is still safe.

Just one small error being missed at one point. The symptoms of the resulting, seemingly unrelated issue are fixed but the wrong cause is blamed.

That causes another issue. Then finally the whole things closes up shop for the holidays.

I think at one point we had four people working on it until one person said “let me try something”. This was followed by facepalms all around. Then you wait for someone on the other side of the world to read their email, and then you start fixing it so it never happens again.

Will we ever go offline again in the future? Of course. But this particular risk is now being managed to within an inch of its life.

So just sit back and wait for the next highly unlikely series of failures that nobody saw coming.


* And read Taleb on managing risk. It will make you feel better.

Thanks for everything you and the crew do for us mate

 

[BG32]

It’s ok frend.

 

[FireKrakouer]

Did you try turning it off, then turning it back on again?

 

[sprockets]

My wife usually tells me before she goes down.

 

[Riviat]

IT sucks, i should know, i work with it on a daily basis.

 

[spudmaster]

If I don't win the 50k in Supercoach this year it's on your head!
Valuable research time something something
P.S Good fix

 

[Kram]

Thanks for everything you and the crew do for us mate

NSWMAUSER, you're the most incredible ass kisser I have ever seen.