Major Technical Update, will make Poisons persist through Restarts

Started 27 Sep 2019
by gruenesschaf
in Planned Changes
The first major part of the "GC Work Reduction via Cold Storage" as announced in this thread: https://forum.playphoenix.online/viewtopic.php?f=2&t=10919 will come most likely on Tuesday, the update is currently in an internal testing stage. This will be the first really major technical update since launch and the first to include a noticeable (planned) downtime of about 20 - 45 minutes.

Positive user visible changes:
- faster restart times
- slightly lower GC pause durations
- poisons applied to weapons will persist through restarts

Potential negative changes:
- Logins right after a server restart might fail, in that case some changes will happen a bit later to resolve this

This update has the chance to cause some major issues which is why it'll be deployed Tuesday or Wednesday morning to limit the potential impact.

Technical details for those interested:
Up until this update we're storing item bonuses and item spells in separate tables from the items, each row in those tables represents a single bonus or spell respectively. This means there is an unlimited number of bonuses or spells theoretically possible to be added to items. This update adds fields for 14 bonuses (14 is the highest number of bonuses currently in use for a whooping 2 rogs) and 3 spells to the item table instead, those are then the new hard limits.

This update will then enable us to just load housing / repossession / bg loot chest items on startup and delay loading of account / character specific items to the login time. This delayed loading theoretically makes a ddos somewhat possible which is why after some monitoring some changes here may or may not happen depending on how much load it actually causes in production. It's possible that this extra load is underestimated to the point of actually causing issues under normal circumstances or over estimated to not be a practical ddos vector.
While we could have done the same loading without merging it into a single table, the resulting query and the way it's processed would have required a feature not supported by the underlying database driver (npgsql throws some errors when trying to use the binary copy of arrays that don't have a corresponding table) and changing it over to the conventional query method that would be supported results in practically no performance gain whatsoever. This merged table approach combined with the binary copy query variant reduces the load time from 1.5 minutes to 15 seconds on my dev machine when testing with a slightly older backup containing 9 million+ items all of which were loaded before, after the change only 400k items are loaded on restart.

Since the rvr stats are updated in place instead of recalculated on each reboot this item query is by a huge margin the major contributor to the long restart time (1.5 minutes of 2 minutes) hence a huge reduction in restart time is expected.

Poisons will be persisted after this update as changing / adding a spell to an item, like a poison, is then no more expensive than just reducing the condition of an item due to usage. Before it would have involved adding an entirely new row and later deleting a row while also adding some tracking headache.
Fri 27 Sep 2019 1:11 PM by ExcretusMaximus
I've been wondering this for awhile, what does GC mean?
Fri 27 Sep 2019 1:19 PM by chryso
In programming lingo GC is garbage collection.
It has to do with cleaning up memory.
Fri 27 Sep 2019 2:27 PM by ExcretusMaximus
chryso wrote:
Fri 27 Sep 2019 1:19 PM
In programming lingo GC is garbage collection.
It has to do with cleaning up memory.


Thanks.
Fri 27 Sep 2019 2:38 PM by keen
Never had time to wait 1 min for the restart
Fri 27 Sep 2019 3:54 PM by Roto23
gruenesschaf wrote:
Fri 27 Sep 2019 12:37 PM
Technical details for those interested:
Up until this update we're storing item bonuses and item spells in separate tables from the items, each row in those tables represents a single bonus or spell respectively. This means there is an unlimited number of bonuses or spells theoretically possible to be added to items. This update adds fields for 14 bonuses .

Edgar F. Codd will be turning in his grave when you de-normalize the DB
Fri 27 Sep 2019 4:05 PM by chryso
Nobody runs relational databases any more.
It is all going back to IDMS and walking the set.
Fri 27 Sep 2019 4:08 PM by Roto23
chryso wrote:
Fri 27 Sep 2019 4:05 PM
Nobody runs relational databases any more.
It is all going back to IDMS and walking the set.

millions of companies use relational databases.
Fri 27 Sep 2019 6:33 PM by Keelia
Nerds
Sun 29 Sep 2019 11:58 AM by gruenesschaf
chryso wrote:
Fri 27 Sep 2019 4:05 PM
Nobody runs relational databases any more.
It is all going back to IDMS and walking the set.

Our usage pattern for all "user" generated data would probably benefit from being just flat files due to the access mechanics (loading pretty much everything on startup), however, the guarantees given by mature databases, like postgresql in our case, are valuable even if not a perfect fit for this particular bulk loading scenario and then there is ofc the otherwise unmatched tooling that's available for rdmbs. Or the typical crud use case of our backend where an rdbms is the standard and a good fit.
Anyways, it's actually surprisingly fast and the only part we moved out of the database is the XP / Gold / Item movement tracking as that just made the backup sizes balloon uselessly and also caused the database cache to be filled with useless data displacing other actually used / hot data. While tiny compared to what today is considered a big data set, the raw data is still too large to be sensibly processed without aggregating it first at least to the hour or day level and there isn't much point in doing that on the game server as that would make the db working set no longer fit in ram.
This topic is locked and you can't reply.

Return to Planned Changes or the latest topics