resolved Constant forum issues, whats the cause?

[Props Valroa 26,021]

This has been going on for days now. What is with the constant lag, and site crashing? Is this site under a DDOS attack or is it the servers? Or something else?

Edited April 14, 2023 by The Wife of Law

[cuteycindyhoney 13,283]

The site just died out from under me a couple hours ago. It seems okay now.

[Deae Rising Shine~ 52,385]

I mean to have readed something about DDOS-Attacks here a couple days ago.

[raykv423 1,689]

It seems to be crashing again a few hours ago but it works okay now....I guess.

 

[tinker 89]

I can confirm that there have been repeated layer 7 DDoS attacks recently, while our DC is mitigating layer 4 attacks.

We are actively working to prevent any additional outages and investigating new technologies to combat layer 7 attacks.

Thank you for your patience and understanding.

[Evil Pink One 12,963]

Its hard to complain since i dont donate and what not.....

[Pentium100 2,056]

They manage to get around cluoudflare protection? Or did they managed to find out your real IP?

[Neoeryn 196]

There was an account called epichacker something that created an account a few hours ago. Not sure if they're related to the incident going on, but just clicking on their profile cause me to lose connection, and we have a really strong internet service. Just wanted everyone to look out for this account just in case!

[Venomous 18,813]

I wanted to give an explanation as to what a DDoS (denial of service attack) is for those of you who may not already know. 

It’s a cyber attack conducted by an individual or multiple individuals that is intended to make a an online resource unavailable to its users temporarily or indefinitely by disrupting traffic to it. Basically, a website is drowned with so many requests to get on that it crashes. The website is just too overwhelmed and can’t handle that influx of requests. 

Here is a more in depth description: https://www.mcafee.com/blogs/consumer/consumer-threat-notices/ddos-attack-work/

 

[Twilight Sparkle ✨ 8,520]

1 hour ago, Pentium100 said:

They manage to get around cluoudflare protection? Or did they managed to find out your real IP?

We don't use Cloudflare's "protection" for several reasons but, above all, because it has historically caused us more downtime than it ever prevented - it's been tried a few times here over the years. I know their marketing is slick and many other sites use them but I've found their free product creates a false sense of security, makes promises it can't keep, and generally creates more problems than it solves. Almost like security theatre.

Our "real IPs" are public but this doesn't worry us because our datacenter is effective at stopping the kinds of attacks that knowledge of a site's "real IP" normally enables. Attack traffic that gets through that, Cloudflare is ineffective against as well - at least on their free tier (I can't comment on their paid tiers as we never had the budget to try them) - so our sysops team would be actively working, as @tinker explained, to mitigate such attacks either way.

[Pentium100 2,056]

4 minutes ago, Twilight Sparkle ✨ said:

Our "real IPs" are public but this doesn't worry us because our datacenter is effective at stopping the kinds of attacks that knowledge of a site's "real IP" normally enables.

Big pipes.

I have used Cloudflare and some other pai tiers (including the enterprise tier of some other company) Seemed to work fine, as long as the real IPs did not leak (the pipes in that case were not very big). IIRC it stopped most Layer7 attacks as well and what got through was not a big problem.

OTOH, I remember cloudflare cutting off service by itself on the free tier if an attack reached a certain level.

I wonder who are that bored to attack this site though. And what they hope to gain.

[Twilight Sparkle ✨ 8,520]

1 hour ago, Neoeryn said:

There was an account called epichacker something that created an account a few hours ago. Not sure if they're related to the incident going on, but just clicking on their profile cause me to lose connection, and we have a really strong internet service. Just wanted everyone to look out for this account just in case!

I appreciate your intent of looking out for your fellow MLPF users. Without commenting on any particular account, including the one you're referring to, I want to say that the connection issue you experienced was almost certainly the result of the DDoS attack itself and not anything specific to any one account.

Coincidental random fluke. Unless you happened to try accessing several profiles repeatedly and consistently observed the connection drop on just one - I really, really want to hear about it if that's what you observed.

[raykv423 1,689]

Is this the first time the MLP Forums got a DDoS attack? 

[Twilight Sparkle ✨ 8,520]

2 hours ago, raykv423 said:

Is this the first time the MLP Forums got a DDoS attack? 

Nope; not even close. MLP Forums has been DDoS-attacked so many times over the years that we long ago stopped keeping count. Due to the defences we have in place, the vast majority of such attacks come and go with no impact other than the attackers' wallets lightening up and those of the criminals who run their botnets becoming a little heavier.

[Twilight Sparkle ✨ 8,520]

19 hours ago, Pentium100 said:

Big pipes.

I have used Cloudflare and some other pai tiers (including the enterprise tier of some other company) Seemed to work fine, as long as the real IPs did not leak (the pipes in that case were not very big). IIRC it stopped most Layer7 attacks as well and what got through was not a big problem.

OTOH, I remember cloudflare cutting off service by itself on the free tier if an attack reached a certain level.

I wonder who are that bored to attack this site though. And what they hope to gain.

Big pipes indeed, yes - measured in terabits. Earth will have bigger problems than MLP Forums being spotty if someone manages to knock our host offline...

Thanks for sharing your experience with Cloudflare. It's nice to hear a success story about it!

Out of curiosity, has all your experience with Cloudflare been on their paid plans? They advertised layer 7 protection as a "Business and up" (>$200 USD/mo) feature last time I looked into this, and the free tier's main option for mitigating such attacks meant placing the site behind irritating interstitial pages with sometimes-barely-working captchas. I hope it would work fairly smoothly, and transparently to a customer site's users, when one pays good money for it.

***

On motivations for DDoS attacks - some general commentary (not specific to this attack):

Usually people DDoS a site for lolz. Little more than personal amusement from the power rush that comes with sending a shadowy anon some Monero and feeling like they get to play god by then pointing a botnet at any site they'd like to disrupt.

Less commonly, they're in it for profit and try to extort a ransom out of a website "in return" for "allowing" it to be online.

And sometimes, they're in it as haters trying to enact some sort of "cleansing" agenda, where the intent may be to damage the reputation of their target and drive users away from it, often by making users get frustrated with trying to use the site and making them resent its staff for being apathetic or incompetent.*

*Note: While I dislike drawing attention to attacks and attackers on principle, and we usually don't, the community has a certain right-to-know when one causes visible disruptions so we can rightfully direct our collective ire toward the malicious actors and even band together to defend our corner of the ponynet. So that y'all know it's not a matter of MLPF "dying," staff giving up on it, or anything along those lines.

We as a staff team aren't always great at communicating this since the sysadmins are usually more preoccupied with fighting problems than announcing them. But it never hurts to tell us or ask questions when something's not working right! The "worst" that'll happen is that your "X is broken for me" report will receive an explanation once the proverbial fire's been put out.

 

Regardless, DDoS attacks are fundamentally antisocial, they're jail-worthy crimes, and they're inevitable in this day and age when one does anything of note on the internet.

In fact, they're almost a testament to a site's notability - that someone thinks a site is a big enough deal that it's worth spending money and risking a criminal record to try and, if not totally take a site down, annoy its users and staff. Receiving one is like a compliment that your site matters.

[Pentium100 2,056]

8 hours ago, Twilight Sparkle ✨ said:

Out of curiosity, has all your experience with Cloudflare been on their paid plans?

As far as I can remember, sites that were at risk of DDoS used paid plans. It was, at least at the time, that you could not even have your own ssl certificate unless it was a paid plan an a cloudflare-provided cert with 50 other domains on it did not look good.

I have also used Incapsula, but only the maximum plan (the one where you negotiate the price).

 

As for the motivations for an attack - I can see why someone would want to DDoS a cryptocurrency exchange or some very popular site (or a government site). I guess someone thought that this site was somehow worth DDoS'ing.

[Props Valroa 26,021]

These days I notice that my notifications seem to 'reset' and display even when I have already read them, and yesterday I noticed some users profile pictures did not display or load.

[Splashee 28,537]

6 minutes ago, Califorum said:

These days I notice that my notifications seem to 'reset' and display even when I have already read them, and yesterday I noticed some users profile pictures did not display or load.

The concern has been brought up to the server staff.

 

Posting new topics, statuses and messages are sometimes delayed, and sometimes show up as duplicate or even triple entries. We know about the issues, and while they are being dealt with, the announcement at the top of the forum page will be displayed.

[Props Valroa 26,021]

4 minutes ago, Splashee said:

The concern has been brought up to the server staff.

 

Posting new topics, statuses and messages are sometimes delayed, and sometimes show up as duplicate or even triple entries. We know about the issues, and while they are being dealt with, the announcement at the top of the forum page will be displayed.

Do you know the cause of it?

[Courageous Thunder Dash 7,809]

7 minutes ago, Califorum said:

Do you know the cause of it?

The staff will provide an update when possible. 

[Jeric 46,834]

On 2021-03-15 at 3:12 PM, Califorum said:

Do you know the cause of it?

 

Some issues are related to third party attempts to impact service. However, there are several issues that will be experienced that are linked to how much MLPF has in our coffers for server maintenance and upgrades. 

 

What can cause temporary server issues? A lack of .... 

 

 

 

 

This original topic was related to DDoS issues. These are resolved so I'll close it out. As always, subscribing or donating to the site helps pay for the upkeep and helps make the site more stable over time. 

[Jeric 46,834]

As the question was asked and answered by either a staffer or user who was correct, we will be closing this question out. Thank you as always.