Activision’s Journey to Scaling Databases with Vitess
Managing databases can be a challenging task, particularly when dealing with complex applications requiring high scalability and availability. Activision/Blizzard, a prominent video game publisher, faced these challenges while managing its increasingly complex database clusters.
At a recent Kubecon 2023 talk, Greg Smith and Vladimir Kovacik from Activision/Blizzard shared their experience scaling databases using Vitess, a cloud-native database clustering system capable of horizontally scaling MySQL. Here’s what we learned from their journey.
Activision/Blizzard had multiple teams, each with its own database cluster consisting of 100+ shards. Setting up these clusters took over two days and resulted in slow development iterations. All schema migrations required meticulous planning and execution in production.
Selecting the Right Database Solution: Evaluating Key Requirements and Choosing Vitess
The team evaluated more than 100 requirements when seeking a new database solution. Some key requirements included SQL query compatibility, minimal application changes, backend MySQL support, Kubernetes-native functionality, and a Kubernetes operator. After extensive testing and evaluating several options, they chose Vitess.
What is Vitess?
Vitess is a cloud-native database clustering system that scales MySQL horizontally and can be utilized as a standalone service. It offers a Kubernetes operator for easy deployment and management of Vitess clusters within a Kubernetes environment. The operator handles the entire lifecycle of a Vitess cluster, from creation to scale as needed. Vitess is designed to be cloud-agnostic, running on any cloud platform, including on-premise, private, or public clouds. It supports popular cloud platforms like AWS, Google Cloud, and Microsoft Azure.
Why did Activision/Blizzard choose Vitess?
One primary reason was its MySQL compatibility. They had in-house MySQL experts and were aware of successful Vitess implementations at other companies. Additionally, Vitess boasted a large, active community vital for long-term support.
Integrating Vitess into Activision/Blizzard
To begin integrating Vitess, the Activision/Blizzard team started with a small service, the Social service. This allowed them to refine workflows, build production confidence, and align technical expectations with the platform team. It also provided an opportunity to test the service on the Kubernetes cluster and examine how volumes functioned.
After successfully integrating the small service, they moved on to the business-critical Inventory service. They needed to demonstrate scalability and earn company-wide trust. They began with a Minimum Viable Product (MVP) and confirmed the service’s compatibility with Vitess. They also identified business-critical unit tests and resolved blockers.
The Activision team concentrated on production readiness, conducting chaos testing and configuration deployment under load. They transitioned from sharding to a single endpoint, reducing the on-call burden and enabling database setup using the Gitops models.
Lessons Learned from Activision/Blizzard’s Database Scaling Journey
Overall, Activision’s story of adoption and growth is inspiring. With careful planning, a willingness to learn and experiment, and a dedication to teamwork, even the most formidable challenges can be overcome.
In conclusion, the talk by Greg Smith and Vladimir Kovacik was incredibly in-depth and insightful. Their experience in scaling databases at Activision/Blizzard offers valuable lessons for anyone dealing with similar challenges. I highly recommend watching the full lecture once it becomes available, as it provides a comprehensive understanding of the process and the benefits of adopting Vitess in a complex environment.