Jump to content
Banner by ~ Ice Princess Silky

Working on a forum. (This one.)


DDR

Recommended Posts

Hi all! I'm not really sure where to post this, so it sort of falls here as the least-disruptive option. It's not important news by any means, but I figure it might be interesting to see what I've been doing with regards to mlpforums.

2017-10-03: Set up a basic ubuntu VM in virtualbox. Downloaded ipboard (our forum software) from our Github, but couldn't get it running. With Feld0's help, however, I did manage to get the official, vanilla download sort-of running. (ipboard's documentation is really sparse.)

2017-10-07: Installed all the things. Configured nginx, installed mysql and some php extensions. Got the forum 'can it be installed' checker to come up green after some misunderstandings. It seems to want /var/www/html chmod'd to 777, but I gave it 664, which is slightly more secure, and ipboard accepted it. :D

At this point, I realize I should have installed mariadb instead of mysql. It's too late however - after half an hour and gratuitous use of virtualbox's snapshot ability, I determine I can't uninstall mysql enough to install mariadb.

2017-10-11: Some configuration later, and I finally have a running copy of ipboard! Taking a chance, I unzipped the git copy of the forums overtop of it, and it Seems To Work™. Pushing my luck, I install a few extensions for the forum as well, and everything continues to work just fine. Spooky.

2017-10-18: Today, I started looking at installing the iAwards "application" for us. It seemed to be go fine in testing, after I upped my VM's Nginx's HTTP "POST" size limit. I also installed a few other applications, and upgraded my VM's version of IPS Communit Suite. (Which I think is ipboard? I'm not sure.) I'll hopefully have time to finish that up this weekend. :)


Anyway, that's it for now! I hope to post updates here as things get on. Nothing important, just the day-to-day maintenance and sysadminnery that I'm doing behind the scenes. Please, comment, let me know if there's a better way to do what I'm doing, and – thanks for reading!

  • Brohoof 8
Link to comment
Share on other sites

Spent a while figuring out how to set up an award. Nothing is documented as well as it should be, but I -think- it all works now. iAwards bases the award permissions on roles, so it looks like one-offs are either going to be handed out by an admin (probably yours truly) or they'll be handable-out by anyone with the role of the person who should actually hand out the award.

  • Brohoof 3
Link to comment
Share on other sites

  • 2 weeks later...

 

On 2017/10/19 at 11:40 PM, DDR said:

Thanks! Yeah, you can ask anything you want. :) I don't have a very good handle on things yet but I'll answer what I can.

Nice, software experience (or even open-source lol) sharing is always helping. Regardless, I've seen some features very similar to the one I seen in other forum, so what framework you're using?

Also, I have some interests in the forum API stuff, can you do a little...introduction maybe? Just want to know what can that do and how to use them if needed

Link to comment
Share on other sites

Whoo! thank you for all of the great work you're doing!

Why mariadb over mysql? =D Just curious about why you wanted the former because I've never myself used it. I use mysqli extension and have been totally thrilled with it.

Edited by Solar-Wind
Link to comment
Share on other sites

On 10/30/2017 at 8:21 AM, Compeador said:

so what framework you're using?

This forum is a customized installation of IPS Community Suite, and a bunch of plugins we bought.

 

On 10/30/2017 at 8:21 AM, Compeador said:

Also, I have some interests in the forum API stuff

I'm don't know which APIs are exposed by IPS. :/ You'd have to ask one of the more experienced tech ponies, maybe over at https://mlpforums.com/forum/63-site-questions-tech-support/. As for introductions, I'd say try poking around on our Discord server. There's a #talk-with-staff channel which might be helpful. (I'm DDR there too, if you need me.)

 

——————————

 

On 10/30/2017 at 2:46 PM, Solar-Wind said:

Whoo! thank you for all of the great work you're doing!

You're welcome! ^_^

 

On 10/30/2017 at 2:46 PM, Solar-Wind said:

Why mariadb over mysql?

As I understand it - because I haven't been able to install it myself - MariaDB has seen more active development than MySQL. The guy who started MySQL sold it to Oracle, but since MySQL was open-source he forked it as MariaDB and continued working on it. MariaDB is nicer to work with these days, having some nice features MySQL lacks.

I've never worked with MySQL and PHP together, so I can't really speak to mysqli. I suspect it would work just fine with a MariaDB database though, since MariaDB is (supposed to be) a drop-in replacement for MySQL. Myself, I'm primarily a Node developer when I have to do back-end work. I've mostly used mysql2 there, which I'm not entirely pleased with, but it has improved quite a bit since I started using it so it's OK now. :/

  • Brohoof 3
Link to comment
Share on other sites

19 hours ago, DDR said:

This forum is a customized installation of IPS Community Suite, and a bunch of plugins we bought.

 

I'm don't know which APIs are exposed by IPS. :/ You'd have to ask one of the more experienced tech ponies, maybe over at https://mlpforums.com/forum/63-site-questions-tech-support/. As for introductions, I'd say try poking around on our Discord server. There's a #talk-with-staff channel which might be helpful. (I'm DDR there too, if you need me.)

 

——————————

 

You're welcome! ^_^

 

As I understand it - because I haven't been able to install it myself - MariaDB has seen more active development than MySQL. The guy who started MySQL sold it to Oracle, but since MySQL was open-source he forked it as MariaDB and continued working on it. MariaDB is nicer to work with these days, having some nice features MySQL lacks.

I've never worked with MySQL and PHP together, so I can't really speak to mysqli. I suspect it would work just fine with a MariaDB database though, since MariaDB is (supposed to be) a drop-in replacement for MySQL. Myself, I'm primarily a Node developer when I have to do back-end work. I've mostly used mysql2 there, which I'm not entirely pleased with, but it has improved quite a bit since I started using it so it's OK now. :/

Interesting on the MariaDB! I ought to look into that some more, see what features those are. x) 

I'm pretty unfamiliar with Node, need to broaden my horizons at some point. I wish you luck on everything!

Edited by Solar-Wind
Link to comment
Share on other sites

22 hours ago, DDR said:

This forum is a customized installation of IPS Community Suite, and a bunch of plugins we bought.

I'm don't know which APIs are exposed by IPS. :/ You'd have to ask one of the more experienced tech ponies, maybe over at https://mlpforums.com/forum/63-site-questions-tech-support/. As for introductions, I'd say try poking around on our Discord server. There's a #talk-with-staff channel which might be helpful. (I'm DDR there too, if you need me.)

It look like many forums using IPS, thanks for the information!

about the API stuff, I just found this page, maybe I should dig into it and see how it works and how to use it.

Wishing you having no bugs when you working on these stuff :smug:

Link to comment
Share on other sites

I'm much closer to being able to test a potential fix now, but I still haven't been able to actually develop one. Hopefully we'll get to have a go at that next week.

2017-11-01: Installed the rest of the addons/plugins we use. Updated Ubuntu in the testing VM and restarted. This broke my shared folder (which I use to get files to/from the testing VM). I reinstated the folder with some difficulty and marked it non-transient, so it should stay around when I shut down the VM now. :okiedokieloki:

While that was in progress, I started downloading a dump of the forum database.

2017-11-02: Finished downloading the database dump. (All your secrets are mine now, muhahaha! Nothing can stop me now!) Next step: load it into the VM's database so I can roll it back on demand while I figure out why upgrading is taking so long.

 

——————————

 

8 hours ago, Solar-Wind said:

I wish you luck on everything!

4 hours ago, Compeador said:

Wishing you having no bugs when you working on these stuff :smug:

Thanks! I certainly need it these days, everything seems to be falling apart on me. (Except my Factorio game, for once. It's ran without the trains deadlocking for nearly 16 hours now! :o)

 

  • Brohoof 5
Link to comment
Share on other sites

  • 2 weeks later...

The last few days here have been quite frustrating.

2017-11-09: Tried loading the database dump I downloaded on 2107-11-02. It didn't work, I think because it was dumped from a MariaDB database and loaded into a MySQL database. The dump was made to work by removing all occurrences of "PAGE_CHECKSUM=0" and "TRANSACTIONAL=0". This took a while because the dump is 15GB. After that was fixed, the dump was loaded into my testing VM's database in just over four hours. I think the two things I had to remove were optimizations. :okiedokieloki:

Disk space in the VM is a little tight now, so I uninstalled LibreOffice to free up some room. It seems I configured the VM with a 40gb virtual hard drive and the database now takes 30gb of it.

The database loaded, I then blew a few minutes trying to figure out why I couldn't log in. It turns out having loaded the database from this forum I also loaded my password from this forum.

The first thing that happens with the production DB loaded is IPS, our forum software, trying to write somewhere it doesn't have permission to. I made the installation folder, /var/www/, world-writable and world-executable. It works, but I feel all squidgy inside. We seem to be missing quite a few files, such as the theme, so most of the site doesn't work. The admin panel is a little more resilient, thankfully, and seems to be working fine - except for upgrade step 3, which throws a theming-related error. I figure I'll grab the likely files from the production server, there's a bunch in the web directory that seem likely candidates. However, I can't find the SSH details to access production, so I call it quits for the night here.

2017-11-10: Feld politely reminded me where the server was, and I found my notes by searching for the address. :derp: I copied the production files over to my local machine.

Copying the production files over to the development folder didn't work outright, as a 404 error was thrown for the admin panel. I fixed some permissions, and traced the error down to something forcing HTTPS. While quite reasonable, HTTPS doesn't work on my dev machine. I eventually fixed that by coping the first half of the folders from production over using 'merge folders and overwrite existing files', and the second half using 'merge folders and keep existing files'. It's strange, and I don't understand why it works, but it works so yay!  We have admin panel again, and now the theming-related error from yesterday is fixed.

Step 3 out of the way, step 4 promptly fails when it can't download a file. Luckily, it comes with a link, so I download and apply the patch it wants. This fails as a blank page until I fix the permissions again. Now we are on to the real meat of the upgrade!

First, the "converter" application can't be upgraded. After some asking around, it turns out it comes from - not the applications area on Invision's website - but bundled with the download of the forum software. I've decided to backburner installing the app for now, since we don't really use it, but I'll swing around and install it again after the upgrades are complete.

The first step of the upgrade proper is to run a database command. It modifies a large table, so I need to run it manually. I can't run it, however, since the database says I'm out of virtual disk space. (Well, it said something completely unrelated, but I typed the error into Google and Google said 'disk space'.) So, since I can't seem to free up enough space to run the command in, I need to make the virtual disk itself larger. To do that, I need to make a backup of it first in case something goes wrong. To do that, I need more actual disk space to work with - my solid state drive has filled up. I had originally set my computer up so all the virtual disk images were on my larger hard disk drive, but it turns out that the mechanism for that had got broken. I copied over the testing VM's virtual disk to the HDD. I then set VirtualBox to use the new copy on the HDD, and tested that it worked there. I then deleted the old copy on the SSD to free up room, and got on with the process of enlarging the virtual disk. I ran on the HDD:

> cp Poniverse-MLP-Forums.vdi Poniverse-MLP-Forums.vdi.bak
cp: error reading 'Poniverse-MLP-Forums.vdi': Input/output error

:twi:fiddle fiddle fiddle

> rm Poniverse-MLP-Forums.vdi.bak -r
rm: cannot remove 'Poniverse-MLP-Forums.vdi.bak': Read-only file system

:wacko:

 

 

 

… I broke it.

 

2017-11-11: All the files that told the computer how to start were on the HDD, and the HDD is now dead as a doornail.

After 3-odd hours spent trying to install Grub and mark the SSD as "a thing you can boot", I'm finally been able to start the computer again. I have been able to recover a backup of the virtual machine I took at the start of the month, since the current copy died with the HDD. My next step will be to redo the work I did so far this month based on the backup. It should be fairly quick, as I took good notes. But… still. It's been a fair chunk of time I've had to spend getting everything working again. I'll probably be able to take another crack at the upgrade on the 21st or 22nd. I should be able to borrow a replacement HDD before then, too, so that'll be good.

Edited by DDR
nervousness
  • Brohoof 3
Link to comment
Share on other sites

May not be the most helpful advice right now - but you can always just add a much larger second virtual disk to the virtual machine; once that is done, if you shut down the database service, rename the database's file storage folder, then create a new, empty folder with the old name, you can mount that new volume in that location and copy the database files over from the (renamed) original folder.

you can also increase the size of virtual disks, then expand the file system to fit, but I usually find having a separate data disk makes life simpler (and back in the spindle days, meant you could give it a dedicated spindle of its own)

Edited by CypherHoof
Link to comment
Share on other sites

  • 2 weeks later...

Well, after nearly two months of trying, I got my testing forum upgraded at last! This post is a little techier than I'd like because it's been written in haste and anger, but nonetheless — here is what I've been up to:

2017-11-22: Restored from a (thankfully!) recent backup, taken on 2017-10-31. Reapplied the steps I'd taken, which didn't take too long because I know what to do now. I did, however, run out of HDD space before the dump completed this time which was odd because it worked last time with 10gb to spare. Nonetheless, after I deleted my VM snapshots and resized the drive, everything ran begrudgingly to completion. I figured out how to get it to print errors this time too, which was helpful. There were a lot of them. :dry:

2017-11-24: After the DB finished loading into my VM yet again, I took a backup of the VM. On a separate disk drive. I then figured out how to load the production .sql files that make up the forum in such a way that they actually run in my VM. (For the record, I had to copy half the folders over by 'replace existing files', and the other half by 'keep existing files'. :muffins:) At this point, the admin panel started insisting that it wanted to run on https://, as opposed to http://, which my dev environment doesn't support at all. But I can sort of hack around it for now, so it's fine. :)

I think most of my issues with the database have been caused by trying to take data from MariaDB (which mlpf runs on) and put it in to MySQL (which I installed on my dev VM). I can't figure out for the life of me how to get MariaDB running on an Ubuntu instance that's had MySQL installed on it - MySQL somehow poisons the box and prevents MariaDB from running, even after MySQL has been entirely removed. I tried working around that, but I'm the only person with this problem according to my google-fu. I also started setting up a new VM, one with MariaDB on it from the get-go, but I didn't get too far with it as I figured out the problems with the existing VM.

Upon running the forum upgrader in my dev machine, I found that 17 tables had failed to import from production. They used a different database storage engine than I had. I modified the command I'd originally used to grab the production database to only grab those 17 tables. Running this command, I promptly took the forum down for 20 minutes while I downloaded 2.5 gigs of caching data. :blush: I'm so sorry if I caused anyone to loose a post they were working on.

2017-11-25: Carrying on. The data I now had let me run the upgrader a little further. Apparently I was missing two tables, which prevented it from running to completion. I recreated them and left them empty, since they didn't seem important. During the discovery process of what was wrong, I got this gem of a self-contradictory error message out of MySQL:

mysql> CREATE TABLE `mlp_core_search_index` (`index_class` varchar(255));
ERROR 1146 (42S02): Table 'ipboard.mlp_core_search_index' doesn't exist
mysql>

It is complaining that it can't create the table because the table it's trying to create doesn't exist, which makes 0 sense. However, I could and did DROP it, so that solved that mystery. :muffins: On with the upgrade!

And, it runs to completion.

But… were those tables important?

2017-11-26: I grabbed the data of the two missing tables from production, using more or less the same command that killed the forums on the 24th. (I am either brave or stupid.) After applying them, I ran the upgrade again, and again it worked without issue. One amusing part of the upgrade chain is that it asks you for a license key. I don't have one. But… the upgrader URL scheme is mighty predictable, and it turns out you can totally type in the URL of the next page without entering a key. :P

I spent the rest of the day trying to get the admin panel to not require https, since I only have http on the test VM. Despite setting it in three or four places to only use http, it still forces https. It looks like I'm going to have to figure out how to self-sign a certificate so I can have https in the test VM. I'll probably have time to do that later next week, so that'll be good. Then I can continue on the upgrade path on my test VM until I hit the blocker that has prevented mlpf from upgrading. :)

Then I will fix the blocker and upgrade mlpf.

  • Brohoof 2
Link to comment
Share on other sites

I'm so very happy to see the progress on this. Your persistence and attention to detail with this will soon pay off, DDR!

One thing I wanted to comment on was the following:

4 hours ago, DDR said:

I did, however, run out of HDD space before the dump completed this time which was odd because it worked last time with 10gb to spare.

Some versions of MySQL set the innodb_file_per_table setting in my.cnf to false by default. If this is set to false, then all InnoDB tables will write their data into a single "ibdata" file in your MySQL data directory which will forever grow and never shrink. MySQL tries to reclaim space from "deleted" tables but it's not perfect. If this setting is disabled, and you previously imported a database and deleted it, it's possible that you lost some disk space to that pesky ibdata file.

  • Brohoof 1
Link to comment
Share on other sites

8 hours ago, Feld0 said:

If this is set to false, then all InnoDB tables will write their data into a single "ibdata" file in your MySQL data directory which will forever grow and never shrink.

Interesting! I had no idea about this - researching it lead me down an interesting little rabbit hole. (Mostly to the mysql documentation.)


@Nightfall Gloam: Thanks! :)

  • Brohoof 2
Link to comment
Share on other sites

  • 4 weeks later...

~some chaos later~

2017-12-20: Well, the forums are upgraded.

I started the day trying to get my dev environment to display the admin control panel. It had insisted that it was going to, come hell or high water or the three switches I flipped - force an https link. I didn't have https set up in the dev environment, so it didn't work. It turned out to be surprisingly simple though - I uncommented two lines in the Nginx config file, added an exception in Firefox, and the page loaded like a charm. And I was worried! :lol:

@Feld0 renewed our ipBoard license, and a few addon licenses, and I upgraded all the things. This broke some of the things, as you may have read about. The core forum upgrade took about 40 minutes. Other than scaring our ops team when mlpf went down for the upgrade, things went quite smoothly. It took about 40 minutes total, and another 5-6 hours to deal with the aftermath. The forum had some glitches which were quite worrying to see, although I think I've patched things up fairly well now with some horrible, horrible CSS. I was assuming the upgrade would be quieter, so I actually ended up spending a lot of time telling people what was going on while it was still ongoing. One thing that did take us by surprise is that the new ip.board reaction system installed enabled. We had assumed we'd have to check a box somewhere to enable it when we were ready. Work on the reactions system, probably to replace the brohoof system with something more meaty, is ongoing.

I figure that if no major bugs crop up, the next thing to look at are the character pages. This is a bit of a long shot, however, as it's currently broken and I don't understand at all how that part of the system works. But nonetheless, worth a look. :)

  • Brohoof 3
Link to comment
Share on other sites

  • 3 weeks later...

2018-01-11: Did a routine upgrade of the forums. Everything went quickly and smoothly, without issue, both in the VM and here in production. "So this is what it's supposed to be like…"

I also added cron job to the VM, so it should actually do its background processing tasks now. Set the default editor to Vim.

  • Brohoof 3
Link to comment
Share on other sites

  • 3 weeks later...

2018-01-31: Removed the e-commerce integrations from the VM - apparently when I copied over the database, I got all the posts but also the the module that bills people… which was set to bill people. :blush:

Now that the initial shenanigans are over, things seem to actually be quite quiet on the sysadmin front here. Updates come down the pipe from Invision occasionally, and I upload them to MLPF.

I'm starting a new job tomorrow, so hopefully I'll have more time to poke around and fix some of the long-term issues we've been having. My predecessor(s) left me very bad notes, and I'm worried that we've lost a lot of experience of how to modify and fix the forum software. In addition, my dev VM doesn't actually work very well right now. (I guessed, badly, how to get the forum up and running when I set it up.) So while I can view the home page with the dev vm, I can't actually click through to any sub-forums or posts. Only the admin panel seems to work—which is enough to test the updates—but not enough if I'm going to reproduce and fix our standing issues.

I guess the next things I'm going to work on are:
  - figure out what's broken
    - file them as bugs in the internal bug-tracker
  - properly remake the dev vm from the ground up, now that I know what I'm doing

I have made some good progress documenting mlpf in the internal wiki, so the next person to come along won't be starting from scratch. There's still a lot more to go, though. I swear, 80-90% of software is just talking to people or writing things for people to read. :lol:

 

  • Brohoof 2
Link to comment
Share on other sites

  • 4 weeks later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Join the herd!

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...