Mercury

22 hours ago, Ganondorf8 said:

Well... That was quite a roundabout way of introducing a new member of staff.

I liked it.

I definitely like this energy!

It is likely that your browser is enforcing HTTPS or you have an expired session. The homepage of PVL does not support running under HTTPS as several of the stations never converted to SSL streaming. PVL itself is deprecated now unfortunately and most of what is here is just what is still running.

Test!

One of the new chicken sandwiches and probably some fries, but you've really gotta eat them before you get home. Once they're cold they're a no go. (Also testing to see if posts are working again in the new Database schema!)

We're finishing up a migration to a new server stack and the networking configuration is causing some traffic to get redirected incorrectly. This is causing the 503s because the forums trying to connect back to our fastcgi server gets a connection refused error. I am currently working on resolving this, and the situation should improve.

When I ran the troubleshooter last night it disabled most of our plugins. That's all. I was trying to remedy the issue where the reply box and images were not loading and this was just a temporary side-effect. Many apologies.

It's just down ever since some of our DDoS proections we put into place the other day. I'll have tinker look into this some more.

So I wanna preface this with this: I'm thankful that everyone was caring and offered lots of help and showed a lot of concern for the site. 

We weren't hacked, DDoSed or anything like that really. We just experienced a symptom of old age it turns out! To get into a bit of technical stuff, we actually lost both of the main hard drives (SSDs) in the main web hosting server on the same day. To put this succinctly: This kills the server. After spending a day diagnosing the issue, the ops team and I were ready to submit the case to our hosting provider OVH and let them start replacing the parts. 

This is a great time for us to revisit our policies so that we're very sure that something like this wouldn't be a huge loss in the future. We already take daily backups of all of our services and put them offsite as well so that losing a host server like this isn't the end of the world. However due to this extended outage we're going to take our time and verify a few things to make sure that all of the processes we have in place are working as intended just in case something like this happens again in the near future (We just got new hardware, so fingers crossed it doesn't!). This has also given us a lot of insight in the lifetime of our servers and when we actually need to schedule this kind of maintenance with our hosting provider so that in the future we can plan for this and only end up with a few hours of downtime rather than two whole days.

Really though. I want to thank everyone here, new and old for being here and giving the site life. This was a difficult time but a sign that we've been going long enough to wear out the hardware the site was running on and that to me is something to celebrate! Thanks again everyone and if you have any concerns or questions feel free to inbox me or post them here!

@RainbowDashFan124 Unfortunately it's more or less usually because our forum software is locking up several php-fpm threads, it's really only gotten bad on this update that we're on and we're looking at upgrading again here soon and we'll see if that resolves it.

Iceborne can't come to PC soon enough

Happy hunting to all of the console players that get to experience it first!

We're on it. The latest forum update undid several of our fixes unfortunately as IPS hasn't fixed the bugs yet in their mainline release.

All of our custom plugins that drive the economy and class systems.

While I've already responded on Discord, I will go ahead and mirror that here for the users on the site who don't visit the Discord. This server is not being updated past 1.12.2 because currently I'm the only staff working on it and I'm still in the process of getting us ready for 1.13. The newest updates are off the table entirely right now until the main developer has the time to update the custom plugins our server uses.

I should be going, either with Jeric's group or with AppleDash's.

The site's search engine will be a lot less useful for a little while (It's limited to just one day's posts in its search results), since the latest update broke how it handles searching the db. We're working on a potential solution, but it's gotta stay like this until we have the solution ready!

In the meantime, you can still search our site using Google/Your Favorite search engine.

Stuff that's known and being looked into currently: 

- Status updates being blank on initial edit, but edits showing up (We know what's happening but not why)
- FAQ and similar answer posts are dead
 

I moderately agree with this one, this is your only post, so no history would be lost. Have a nice day, this will likely be the last notification you get from here.

First off, I'd like to apologize for having the site unavailable for so long. We encountered an issue with the database locking up last night during one of our nightly backups and were forced to restart one of our host nodes as the process had gone into a non-interruptible state. While I did make an attempt to do a graceful shutdown of all of our services before this (Aside from the process that couldn't be exited) unfortunately I forgot to sync the disks when I did so so we lost our cache configuration for about 10 hours.

In the process, the mail server also failed to start back up properly so notifications and other email services from Poniverse were also down for that period of time, and it went unnoticed as our monitoring software saw that the SMTP port was listening properly. And MLPF itself was up and running from a glance, so we left it like that while Tinker went to work and I slept.

Now that everything has been fixed, with the email issue not being discovered until about 14 hours after the outage occurred I figured I'd address this and let people know what happened and that we'll do our best to check everything more thoroughly in the future now that we know this is a potential problem that crops up.

@tinker @Twilight Sparkle ✨ (Wow Feld pls) and I ended up spending a couple of hours troubleshooting these issues and Feld came up with a few ways to re optimize the forums for our new setup since we hadn't really done that since we moved servers a few years back. Hopefully you will all notice performance improvements as we've moved some of the critical cache operations off of our network attached HDDs and into RAM as well as taken some of the load from our MySQL database and put that to redis and PHP's opcache. From my own testing it's gotten load times down from about 8-10s per page down to 1-2s which should be a lot more manageable. As usual though, I hope everyone keeps an eye out for issues and brings them to our attention!

Yes, everything's fine. We just had a mishap regarding an outage on another part of the node that required a server restart. When it came back up, a couple of important files got caught up in the mess. Everything should be back to normal now, though we're looking at how to minimize various problems in the future and a possible solution to prevent what happened to cause the outage earlier from happening again.

It does not affect us. We're based in Canada, we're not in the jurisdiction of the EU, and just like US copyright law we don't abide by foreign copyright laws (So we ignore DMCA requests, etc too). We do however follow GDPR as a courtesy to EU residents, dealing with GDPR complaints is a bit more complex and has a bit more international weight than Copyright claims. As far as we're concerned, we abide by Canada's Notice and Notice policy where the ISP (Poniverse/MLPF) in this case forward notices to users if action is being taken against them, and remove content once we receive a court order.

We do have stricter rules than are required by Notice and Notice of course, so please refer to our rules as far as breaking copyright goes, but aside from that nothing changes for us.

My first view was that they were weirdos, and that ponies were a weird thing to be into, but then a couple of my friends pushed me into it during a break in one of our classes. I had no idea they were into it beforehand so it took me by a bit of a surprise and so we watched a few episodes before school was let out. I'd already watched a bit of the earlier parts of Season 1 with my sister, but they eventually got me hooked by introducing me to some of the early fanfiction and some of the fandom's songs and I was drawn in shortly after.

I'm honestly not sure how we'd even troubleshoot that, are you sure no browser plugins are causing this? I'd suggest trying to use the forum in a different browser or incognito to test. 

Yep, tap water's spring water here. Delicous stuff  

I'm not sure what I am now. I was raised a baptist christian, the military showed me a lot of things that pushed me away from believing in a god. Now I'm in a sort of limbo where religion hasn't been a part of my life in so long that now that it's odd having bits come back now that I'm visiting my family again.

3 minutes ago, Midnight Solace said:

What's a fairground? 

Wikipedia is your friend! https://en.wikipedia.org/wiki/Fairground

Mine would have to be bumper cars, I had a lot of fun in those in my teens. A close second would be the ferris wheel. It's always fun seeing the crowd below from up there and it's a good break from all of the ruckus.