Database Backups & Redundancy
After our recent data loss, we quickly switched from daily to hourly backups off-site, but now we’ve gone a step further. In addition to encrypted hourly backups being stored off-site, we’re now running master-slave replication. It took longer than we would have liked, but it’s not a change we wanted to take lightly. We’ve invested significantly in improving our infrastructure, and this was our primary concern.
Even under the heaviest loads we’ve ever seen, our application response times have been almost cut in half. Of course, that’s only back-end performance. We still need to make some front-end performance improvements, but this is a step in the right direction. We’re now in a place where traffic spikes will have a much less significant impact on performance.
Application Server Redundancy
In addition to normal performance, Sifter is now running with multiple load-balanced application servers. Recently, we had been running into problems where a neighboring virtual machine was causing performance degradations with Sifter. The load-balanced application servers will minimize the chance of this happening in the future. In addition to insulating Sifter from these kinds of problems, it also makes it much easier for us to scale without downtime.
Dedicated Job & Search Server
In addition to the data redundancy, we’ve also made Sifter more modular. Our background processing as well as search indexing and querying have all been moved to a dedicated virtual machine. Again, this enables us to easily scale to handle the growing importance of these pieces of Sifter as well as freeing the application servers to respond to web and API requests.
This hosting move has consumed almost all of our attention recently, but now we’ve laid a foundation for the future. We’re dying to get back into making user-facing improvements, but this was key to freeing us up to focus on those kinds of things. We still have some additional infrastructure improvements that need to happen before we can get back to focusing 100% on application development, but this was a huge milestone.