Unfortunately, production level web servers, databases, and worker instances cost money to maintain, which is why I need the community's help in ensuring that Hotstat.us can keep the lights on going into the future.
The technology that makes up Hotstat.us:
- Data Pipeline (Php/C#) catalogs user submitted replay files from a central repository, dynamically requesting AWS spot instances to concurrently download and process Terabytes of replay files as needed.
- Data Pipeline is horizontally scalable: transactions, time-based soft-locking, as well as a variety of database locks ensure that multiple processes can work across the dataset, while being spun up/down at will with no ill-effects, allows for 80% VM cost savings through use of interruptible AWS spot instances.
- Allows users to query statistics through aesthetically pleasing filters, and visualizes results through informative graphs and tables. Filtered data is dynamically loaded in-page. This data is presented to users at far superior speeds to alternative graphical offerings through the use of an extensive cache generation system.
- Dual layer caching of filtered data aggregations through HTTP/Redis greatly reduces primary database load, allows for 60% cost savings on database hardware while still achieving desired worker speeds.
- Extensive Tool Backend (Php/C#/C) that extracts and converts meta-data and textures directly from game files, allowing for quick syncing of description/image changes after patch updates to the base game.
- Custom-built monitoring and analytics for the auto-scaling worker instances that catalog and process matches as they're uploaded.