Site was down a few minutes ago...

[Evilshy 5,089]

Just wondering why? Or was it scheduled and I missed the memo?

[Electrobolt 2,068]

May have been a crash or something... But I won't swear to it. We'll just have to wait for Feld0's response

[Vicke 3,187]

Crashes happen... And it was back up after like 5-10min?

[Evilshy 5,089]

Yes, I know that crashes happen, I just wanted to know if there was some other reason for it.

[Orien 3]

Sorry, our Twitter has some details...

 

https://twitter.com/#!/cleverkite

[Twilight Sparkle ✨ 8,520]

There was a server problem earlier this morning that had nothing to do with MLP Forums or any of my other sites (other than the fact that it took them offline). Here's a snippet from an email CleverKite sent me about it:

 

Peter,

 

Earlier this morning at 10:20am PDT, your node (bcloud) rebooted itself as part of an unexpected feedback loop. Under normal circumstances, this would have only caused downtime of a minute or two as the node comes back online. Unfortunately instead, the node's filesystem rebooted into read-only mode and decided that there were data inconsistencies after the reboot that needed to be processed through a lengthy fsck (filesystem check), during which all customer clouds would be inaccessible.

 

At this point, we needed to ensure the integrity of all customer data on this node before fully restarting it, which takes time. 26 minutes later at 10:46am PDT, bcloud was brought back online without data loss along with all customer clouds on this node.

 

We are still investigating the feedback loop that caused the reboot in the first place and will make changes to prevent it from happening again. We are deeply sorry for any inconvenience caused by this outage. These sort of thngs shouldn't be happening here and we're not happy about it.

 

Sorry, our Twitter has some details...

 

https://twitter.com/#!/cleverkite

 

Orien, we all know that unexpected problems can happen, and I thank you for coming here and updating my own community on the downtime, as well as communicating about it so transparently.