Backup improvement idea & plan

Subsequent to this announcement, I've been investigating and logging things and I have an idea. As some of you have noticed, the backup tends to proceed smoothly until it hits about 65%, at which point it stalls before resuming for a smooth finish. That is the point at which it is backing up Pokémon data. Needless to say, there is a lot of it, but most notably there is a lot of timeline data. In fact, the pokemon.timeline data file accounts for just about 1/3rd of the entire backup, coming in at over 13GB. I have a multi-step plan to help resolve this, and ideally remove the need for backing it up at all - or more accurately, to allow it to be backed up less disruptively. 1. Remove pokemon.timeline from the backup list and see how that impacts the time. 2. Once it's been confirmed that this greatly improves backup speed, prepare the system for using a new storage mechanism for timeline entries. Specifically, files. Rather than putting it in the database, which is overkill for historical data that never changes, put it in files instead. The file system is backed up separately, using a slower method but without requiring the site to be taken offline. 3. There are a few things that need patching as they currently rely on timeline data. Most notably the "Trade Evolution" condition uses the timeline to find the most recent trade. This should be stored separately as a pointer on the Pokémon entry itself. This patching will also include loading the data from the new files in order to display them on the Summary. 4. Gradually migrate existing data into this new file structure. 5. After confirming it all works, delete the old timeline system. Step 1 is being implemented today, and we'll see with tomorrow's backup what the result is. Further updates will be posted as they become relevant. Thank you for your patience as work is done to improve your experience!

Woah okay, I never really mind about the backup unless im doing something important.

sounds good to me

I plan around the backup, so it doesn't bother me. I just hope this doesn't complicate things like hunts. Good luck, I hope.

as i have nooo idea what most of that means, i'm just going to nod my head, give a thumbs up and hope it means good things, yeah? and no, please don't explain it to me... it won't help. it's just not in my wheel house to comprehend things like that. period. no disrespect to those who do get it. i'm glad Somebody does or the net would flummox and deflate like a flan in the cupboard. no, i am thankful folks are on the ball for the background stuff. i just don't have a clue what most of that is these days. and i'm perfectly okay with that. i'm not building web sites any more and am fine with staying out of it. ^_^

I totally know what you're talking about

for anybody who hasnt understood this, im pretty sure its about how pokemon timelines are making daily backups take longer and niet is going to do some moving around of the timeline data to make daily backup faster whilst also backing up timeline data

Sounds good and I hope it works.

I have no idea what you just said but from the bottom of my heart, I greatly appreciate you keeping us in the loop. So many other gaming sites refuse to acknowledge its consumer base or even ask their opinion on upcoming changes/implementations so ya'll are truly a breath of fresh air in this regard. It makes me very happy to see a site that still believes that their consumer base is worthy enough to know what's going on, and have a say in it. You all are so wonderful. Please continue to be amazing. <3 <3

many thanks for the update and the work niet! c: