There is absolutely no point in planning to do something, if you aren't even sure that it will work. That is why today I worked with Moonsy to conduct a feasibility analysis on the plan. Together we looked at our current server specs, and usage statistics, and compared them to what a more modern server would look like, and also what it would cost. It's looking good. Better than good, even. If we make the switch from a managed webhost (who, frankly, don't know enough about this game to know how to manage its server) to a root server where we have full control over everything that goes on, we'd be cutting out a significant chunk of cost already. Add to that the fact that technology has improved and costs have comparatively decreased, and we could easily afford a significantly stronger machine. So then it's a matter of how to structure the project. Currently we just have a single-user machine that's responsible for Apache and MariaDB, which works fine but it isn't very flexible. Moonsy is quite familiar with Docker, which in a nutshell is a way to automatically manage virtual machines and orchestrate their coordination. In this way, we could have one virtual machine whose only job was to run the PHP that powers PFQ, and the database stuff can be an entirely separate virtual machine whose only job is to run MariaDB. Add another VM to act as the "interface to the web" and you end up with a very secure system specifically because each indiviual piece can only do what it is supposed to do. I must admit it's a bit out of my depth but he has helped me get an understanding of it and I like the sound of things. I also have a setup in place where I can set up a development environment on my own computer, which should allow for an easier time working on the build, rather than having to fiddle around with Remote-SSH and FTP. Once it's done, the server can just pull the git repository and ta-da all the files are where they need to be. Restart the server process and updates are live right away. I'll also be drawing on lessons learned during the PFNext project so far. In particular I will be using OpenSwoole as the server framework, rather than Apache. Apache is incredibly powerful, but we only actually use it as a kind of "forwarder", taking requests and spinning up a PHP process to fulfill them. Why not just have a dedicated PHP process using OpenSwoole to handle requests directly? This comes with a number of benefits for this project, such as the ability to pre-load data files (Pokémon species, item data etc.) and hold them in memory once, rather than having to fetch them every single time a user requests something. There will, however, need to be a mechanism by which I can inform the server that these normally unchanging values have actually changed, such as when a new species is added or an edit is made somewhere. It does come with some complications as well, of course. PFQ's current code relies heavily on the fact that any memory I use will be freed at the end of the request anyway, so I don't worry too much about pointers and references. OpenSwoole runs one process (multiple threads) to do everything, so any global memory is kept throughout. Care must be taken to ensure the server doesn't leak memory and run out! That's easy enough to do in theory, it's just a totally different way of working that I need to get used to. Data input and output will be validated much more strongly. PFQ currently just sort of assumes that the data coming in is what is expected, and if it isn't then in most cases it'll just come out with something like "thing not found" since it's looking for an invalid item, but technically it's Undefined Behaviour which is never good to have, especially not on purpose... Improved data validation will ensure that the given input will always be what the code expects it to be, nothing else will do. There will also be a lot more focus on ensuring objects are always valid. Right now the code just assumes that the properties will be where they are expected to be, and all it takes is one typo to bring the whole thing down. This recode will include strong and strict validation of code using PHPStan, which as a requirement needs to be able to understand the code - if it does not, then I either need to fix the code, or "teach" PHPStan how the code makes sense. I will be looking to set up my development environment tomorrow. I'm looking forward to the journey, even if it looks likely to be a quite lengthy one.

Live streamed on Twitch: VOD Today I worked on getting the basics set up in order to work on the project. Step 1: git init Version control is sorely lacking from PFQ's development pipeline. At least we have a second copy for the "dev server" that I can work on before pushing changes to the live server, but even that is shaky! So the first thing I did was to set up the project with version control so that any changes can be tracked and reverted if necessary - built-in backups! Step 2: PHPStan PHPStan is a phenomenal piece of software. As soon as I save a PHP file, it will analyse it and report any issues it finds. It supports many levels of "strictness", from 0 to 9 with increasing levels of pedantry. I will of course be setting the strictness to level 9, to ensure the maximum level of code precision. I almost immediately ran foul of this high level, as I "dared" to write a function without declaring a "void" return type on it! All functions should define their return type, even if they return nothing (void), and I forgot to do so. PHPStan reminded me to be clear in this case. Step 3: Configuration Normally you would define your desired configuration in a php.ini file, but personally I prefer to declare explicitly which configuration settings must have a particular value in my code. To this end, I wrote some code to set the memory limit, default timezone, and how to report errors when they happen. I also set up autoloading of class files, so I can just refer to it and it'll fetch the file as needed. To top off, I added a list of required PHP extensions and ensure they are all present and accounted for before attempting to do anything else. Step 4: Basic server With all the setup in place, I could now write the basic code to get a server running with OpenSwoole. Very simple and straightforward, I just made it so that whenever a request comes in, it responds with "Hello!". Step 5: Managing the server Without anything else in place, I would need to run the command php server.php to launch the server, at which point it would run in my terminal, interactively, and only be able top stop with Ctrl+C, which immediately kills it without any opportunity for it to shutdown properly. I needed a manager script. Having never really done much with Bash before, I got to work with a little research to make a script to perform certain actions on the server. - start: ensure the server isn't already running, then start it in the background and record its process ID in a PID file - stop: read the PID from the file and send "signal 15" (SIGTERM) to that process - this instructs the server to gracefully shut down - reload: read the PID from the file and send "signal 10" (SIGUSR1) to that process - this instructs the server to reload its files (shut down workers and relaunch them) - restart: shorthand for calling stop followed by start, used to hard-relaunch the server And it worked! That's... basically it. Step 6: Testing The purpose of a "reload" command is to enable updates to be made without having to relaunch the whole thing. It works by letting the server's worker processes finish anything they're currently working on before shutting down and restarting individually, which causes them to read the source files anew and use the updated code thereafter. So to test that I added a call to Test::test(), which was defined in an outside file, and just output "Test 1" to the server log. I then changed it to "Test 2" and sure enough it was still logging "Test 1", until I ran ./server reload at which point it switched over to logging "Test 2". Perfect! Step 7: Automating Having to reload or restart the server every time I change a file is not ideal. Not only do I have to remember to do it, but I have to type the command (or find it in command history) and run it for every update. There's a better way! Using the VSCode extension "Run on Save", I configured my workspace such that whenever I update a PHP file, it would run the ./server reload command automatically for me. Now I can just write code, save, and see the result immediately in my browser! Similarly, when updating server.php itself (the core file that manages the whole thing), a reload will not suffice so it instead runs the restart command. All automatically, completely seamless. And that about wraps things up for today! It was a lot of work but I am very satisfied with the setup I have. It will serve me well. Next up, I will be making the server actually respond to requests properly!

Live streamed on Twitch: VOD I started today with a server that did nothing but respond with "Hello!" whenever a request came in. In order to do things, I need to get information about the request coming in, and do something with the response being sent back. However, the raw Request and Response objects given to me by OpenSwoole are rather bloated and awkward to use. For instance, while getting the request method (GET/POST/...) is as simple as $request->getMethod(), getting the requested URL is $request->server['request_uri'] which is anything but simple. So my first step there was to create wrapper objects. My own Request/Response objects take in the raw ones from OpenSwoole and move things around, exposing just the parts I need, and arranging them in a more useful way. Now the request method and URL are $request->method and $request->uri. Furthermore, rather than "method" being just a plain text string, it is now a Method, which I have defined as an enum (list of specific options) to guarantee that the value is exactly what I expect it to be. One important thing to have is an error handler. To demonstrate what happens without one, I added throw new Exception("Hi") into the request handler. When a request comes in, the error is thrown, and the server crashes! It restarts itself, but obviously this is bad - if the server had been handling other requests at the same time, they just died without completing cleanly. And the user who triggered the error would just get a blank response and no idea what happened. So, inside the request handler, I now have a try..catch block, where the "catch" part will collect any error that hasn't already been handled by the handler itself. It can then nicely format the error in the server's error log, and properly return a "500 Internal Server Error" response to the user, indicating that something went wrong. Much better. In order to test all of this, I created a very simple webpage. It lets me select a request method, a path, and optionally a request body to send, sends it to the server, and presents the server's response so I can see what's going on there. Right now it's not a lot, but it will be very important later! I also went ahead and created a wrapper object for the Worker process itself. The server spawns one process for each processor you have on the computer it is being run on, in my case it's an 8-core processor with hyperthreading, so 16 processes are created. Each one has its ID, and that's it. So now I take that ID and create a Worker object with it. This object currently has just one function: handleRequest. This function takes the raw request and response objects, wraps them in the wrapper objects, and ... well, it will handle the request but right now it just ignores that, just logging the fact that it received a request in the server log, to make sure it works. I ended today with a server that did nothing but respond with "Hello!" whenever a request came in... but safer!

A quieter day today! Partly due to IRL goings-on but mostly to avoid working too hard too fast. PFQ consists of "modules". The party page is a module, the lab is a module, the forums are one module with submodules... I like this concept but it needs to be done better. I've also been thinking about how much more efficient the server can be if most of the work is done by the browser instead. For instance, let's say you load the Daycare, there is a fixed structure to the page that doesn't change, and then some parts of the page that shows for instance what Pokémon and eggs are there, what upgrades you have, etc. This can be achieved by defining a static, unchanging HTML file, then having a PHP endpoint that contains data on what should go into the display. JavaScript (also static) can then be responsible for putting the data into the template, while CSS is there to make it look nice. In this way, the rendering of the page is offloaded from the server to your browser, allowing the server more room to breathe. You may see a brief "loading" moment, kind of like you do on other websites, as the data is fetched. But the server will have much less work to do this way. Additionally, all the unchanging files can be cached, although care will need to be taken to ensure that when updates do happen, the cache is dealt with appropriately. I'd rather not have to keep telling people to clear cache when it really should be my responsibility to keep it up to date! I'll think about it and figure something out. All that's left, then, is the dynamic side of things. Sometimes it'll be getting data, such as your Party contents or inventory. Other times it'll be performing actions, such as hatching an Egg or setting up a trade. These can be done in the corresponding endpoint files, which will make use of the API system I've been putting together for PFNext. Come to think of it, I'm going to be using a lot of what I've built for PFNext... We'll have to see exactly how I want to handle it, but the way Pokémon species are handled over there is definitely interesting and may be worth considering migrating PFQ's database to make use of it - this will basically mean "translating" stuff from PFQ into PFQrv2's format, but shouldn't be too difficult in theory! Anyway, while no code was written today, ideas were pieced together! I have others that remain fairly nebulous but there's definite pros and cons to figure out.

Live streamed on Twitch: VOD After moving home, it's time to get back to work! Today I wanted to have the server actually respond differently to different requests. In particular I introduced the concept of "modules" to the code. The party page will be a module, the forums will be a module (with submodules!) and so on. I stalled a little on figuring out what I would want a module to be able to do, so I came at it from another angle: what should the request handler do? Well, first it should make sure the requested module actually exists. So it takes the URL of your request, and looks at the first part. If a corresponding file exists in the modules folder, it loads that file. (The file is the stored in memory so that subsequent requests to that module don't have to load the whole thing again!) The module file defines its class, then returns the name of that class back to the request handler. The handler can then invoke that class, so that each request to the module has its own state - this could be useful later! For instance, the class could receive a database connection to work with, once that's set up. Then, the request handler checks the method of the request, GET or POST. GET requests happen when you load a page, while POST requests will be sent when you try to do things on the page. Consequently the GET request will simply load up a template file and return it. The "template file" consists of two parts. The "container" part includes the generic HTML that makes any page work, and common parts like the navigation bar and the page footer. The second part is specific to the module being loaded, and holds the content of the page - but at this point the page content is static, no dynamic data is loaded in. The request handler will also handle POST requests. To do so, it reads the request body as JSON, ensuring it can decode it. Later I will add validation for the contents of the JSON, but for right now that doesn't really matter. Once parsed, it gets sent to the module's handler, which can use the URL to route the request to a specialised function. For example, a call to "index/login" would go to a login handler, which will then validate that the request body does in fact contain a username and password, and proceed if so. These requests will send a JSON object back to the browser, which the browser can then use to update the contents of the static page template, or provide feedback to the user that an action was successful, or whatever other action may be relevant. That's about it for this time. I'm struggling a little to figure out which angle I want to attack next, but I'll have a think about it and make a decision before too long! That's the big problem with a project like this... But on the plus side, I can pretty much just copy over the validation code from PFNext and use that directly in PFQrv2, so that's a lot of time that'll be saved!

First you write the code, then you rearrange it so that it's "better", for some definition of the word. Of course there's never going to be a time where I'm fully satisfied with the code, but the code I had written so far was not supporting what I needed it to support. To that end I spent today mostly just moving bits of code around, tweaking and adjusting the code path until I could call it acceptable. Modules now have a "handleGetRequest" and "handlePostRequest" handler on them. These will handle verifying any input, doing anything that needs doing, sending some output, and then reporting success to the calling code. These handlers will return a "Status". This will typically be "Status::OK", but may be "Status::NOTFOUND", or perhaps "Status::UNAUTHORISED" for things that require you to be logged in. Or "Status::BADREQUEST" if the given input doesn't match what the module is expecting. The Worker, then, can load up the module, fire off the request at it, and if "Status::OK" isn't returned then it can format up an error message to show instead. In this way, all cases are handled. If they are not handled, the scenario is caught and an error is raised in a way that does not crash the server. Come to think of it, it's actually kind of interesting to have this "never crash the server" requirement. On this version of PFQ, if something is wrong, I can just throw an error and that's the end of it. But with a perpetually running script handling multiple requests, errors must be handled correctly or else the entire thing dies. This is why, every step of the way, I am testing for potential error conditions and handling them appropriately. Anyway, that's all I've got for now. My next step will be to bring over all the Validator classes I wrote for PFNext so that PFQrv2 can make use of the robust system I've already written. At some point I'll also be setting up the database stuff, but one step at a time, we'll get there!