Streamlining GitHub Runner Deployments with AWS CDK: A Case Study with Cellebrite

Background

develeap is a leading technology company that specializes in providing expert DevOps consulting services. With highly skilled DevOps engineers who are well-versed in the latest technology stack, develeap is ready to help you build, manage and support Kubernetes, Cloud & On-premises, CI/CD processes, Git and GitOps, Infrastructure as Code, Monitoring & Logging, Containers, and more.

Our customer, Cellebrite, is a global leader in digital intelligence solutions for law enforcement, government, and enterprise sectors. The company provides an integrated suite of software solutions, known as the Cellebrite DI Platform, which enables customers to conduct digital investigations, gain data insights, and manage digital intelligence. Cellebrite’s solutions accelerate investigations by enabling the extraction, decoding, analysis, and collaboration of data from a wide range of digital sources, including smartphones, cloud services, computers, IoT devices, and more.

The Need

Cellebrite required an efficient, scalable solution for managing GitHub runner deployments, aiming to enhance their CI/CD pipeline’s performance and reliability. The complexity of managing GitHub runners across various operating systems necessitated a streamlined approach. They needed a system that could automatically configure runners for different environments, ensuring compatibility and optimal performance. Additionally, it was crucial to implement a method for the automatic erasure of runners that were no longer active, preventing resource wastage and maintaining a clean operational environment. This automation would not only reduce manual intervention and potential human errors but also significantly cut down on operational overhead, allowing the development team to focus more on core tasks rather than infrastructure management.

Challenges

• Complex Infrastructure: Integrating GitHub runners with AWS services required meticulous planning and execution.
• Scalability: Ensuring the solution could scale with Cellebrite’s growing needs.
• Security: Maintaining stringent security standards while deploying GitHub runners.
• Cross-Platform Compatibility: Ensuring seamless operation of GitHub runners across multiple operating systems.
• Resource Management: Efficiently managing the lifecycle of GitHub runners to prevent resource wastage.
• Performance Optimization: Continuously monitoring and optimizing runner performance to handle varying workloads.
• Automation Integration: Ensuring smooth integration of automation tools and workflows within the existing CI/CD pipeline.
• Compliance and Governance: Adhering to industry standards and regulatory requirements while implementing the solution.

Strategy

1. Design and Planning: Collaborated with Cellebrite’s AI team to hear what their issues are for designing a solution using AWS CDK, enabling infrastructure as code for seamless deployment and management.
2. Automation: Automated the deployment process of GitHub runners using AWS CDK, reducing manual intervention and potential errors.
3. Launch Templates and Auto Scaling: Implemented several launch templates attached to an auto-scaling group that dynamically adjusts according to context configurations, ensuring optimal runner deployment across different operating systems.
4. Lambda and EventBridge: Utilized a Lambda function triggered by an EventBridge scheduled event to automatically erase “offline” runners once a week, ensuring efficient resource management and reducing clutter.

Implementation

To implement the solution, we used two separate stacks for our configurations. The first was the runner stack, where we created a custom function to generate the appropriate launch template based on the selected OS and environment. This stack managed the deployment and scaling of runners across various operating systems. The second stack was dedicated to the Lambda function, ensuring the separation of concerns and allowing for independent updates and maintenance. The Lambda function, triggered by an EventBridge scheduled event, automatically cleaned up “offline” runners once a week, maintaining a tidy and efficient operational environment. The underlying infrastructure, such as VPCs and subnets, was already in place, and we utilized AWS CDK functionality to integrate them seamlessly into our solution. This approach allowed for a streamlined, automated, and scalable deployment process, significantly enhancing Cellebrite’s CI/CD pipeline.

Results & Solutions

The deployment of GitHub runners using AWS CDK has had a profound impact on Cellebrite’s CI/CD pipeline. By automating the deployment and management of GitHub runners, Cellebrite has significantly improved the efficiency and reliability of their development processes. The implementation of launch templates and auto-scaling groups has ensured that runner instances are optimally configured and managed, providing the necessary scalability to handle varying workloads. The Lambda function, in conjunction with EventBridge, has streamlined the cleanup of inactive runners, ensuring resource utilization remains efficient and cost-effective.

Overall, the solution has reduced operational overhead, allowing Cellebrite’s development team to focus more on core tasks rather than infrastructure management. The integration of AWS CDK has provided a robust, scalable, and secure infrastructure, aligning with Cellebrite’s stringent security standards and compliance requirements. This case study highlights the benefits of using AWS CDK for managing complex infrastructure deployments, demonstrating how automation and infrastructure as code can drive significant improvements in operational efficiency and performance.