Success Story

Monolithic to queue-based architecture migration for a US-based software provider

Download PDF

About the client

The client is a grants management and custom software development provider that works with public and private firms across key industries such as energy, defense, healthcare, emergency management, etc. Established in 2013 in New Orleans, Louisiana, the provider specializes in streamlined data management using powerful tools customized to fit the standards of the specific organization and industry. It is the foremost data management partner for federal government agencies and Fortune 500 publicly traded companies worldwide.

Services used

AWS Solutions DevSecOps Technology Consulting Services

Business Situation

The client found itself grappling with a set of critical issues that significantly impacted the efficiency and scalability of its applications. The existing monolithic architecture, reliant on cron jobs, posed a major hurdle in scaling the application.

Another challenging task was the integration of many AWS services within the application. Tracking security events and managing compliance became increasingly complex for the client, leading to concerns about the overall security posture of their applications.

Managing database connections posed additional challenges for the team. The client was using an internal open-source database proxy, which became difficult to handle. Additionally, as the client expanded, managing several database proxy machines for high availability became expensive.

Further, the deployment process added to the complexity, with multiple stages causing delays. Setting up the CodeBuild containers took too much time, causing delays in updates and hindering software enhancements.

In this complex business scenario, the client turned to Daffodil Software to navigate through these challenges and chart a course towards enhanced scalability, security, and operational efficiency.

Key expectations from Team Daffodil were:

Migrate from the existing monolithic architecture to a queue-based architecture, leveraging containerized technologies.
Implement a robust security solution utilizing various AWS security services.
Optimize database connection management to streamline operations & reduce system downtime, with a simplified and cost-effective approach.
Streamline deployment process to reduce deployment time, enabling faster release cycles and improved operational efficiency.

The Solution

Daffodil Software, in collaboration with the client, embarked on a comprehensive solutioning process to address the challenges impeding their software scalability, security, database management, and deployment efficiency. The process involved strategically utilizing a diverse set of AWS services, orchestrated by Daffodil’s DevSecOps experts.

Migrating to queue-based architecture:
To overcome the limitations of the monolithic architecture, Daffodil leveraged containerized technologies to facilitate a seamless transition to a queue-based architecture. Amazon Container Service and Amazon Redis queue played pivotal roles in segregating application components. This transformation allowed for horizontal scaling of workers, which was particularly beneficial for efficiently handling various job tasks and overcoming the scalability challenges posed by the reliance on cron jobs.

Fortifying security practices with various AWS security services:
Daffodil’s DevSecOps experts orchestrated a robust security solution. This involved the integration of Security Posture Management tools with various AWS security services. The Security Posture Management Tool (SPMT) provided the client with a centralized location to track security events and monitor their progress. AWS Config was employed for ongoing scanning of AWS resources, ensuring compliance with best practices. Additionally, Prowler, GuardDuty, and Inspector were implemented to continuously monitor security events and promptly respond to potential threats.

Database connection management leveraging Amazon RDS Proxy:
Team Daffodil utilized Amazon RDS Proxy to efficiently handle RDS connections within the system, leading to a significant reduction in system downtime and simplification of connection management. Leveraging the capabilities of RDS Proxy streamlined operations, enhancing reliability in database management. Furthermore, the default settings of RDS Proxy ensured a secure and controlled environment for database operations, aligning with heightened security practices while keeping costs in check.

Expedited deployment & improved overall efficiency:
The prolonged deployment process was tackled through the implementation of two key strategies. First, Daffodil employed cache in CodeBuild containers, significantly accelerating the deployment process. This optimization reduced the time required to deploy applications to just a few minutes. Second, AWS CodeDeploy was introduced to automate and standardize deployments. This not only resulted in faster release cycles but also significantly improved overall operational efficiency & cost-effectiveness, addressing the challenges posed by the existence of multiple stages in the deployment pipeline.

The Impact

The implementation of this comprehensive solution profoundly impacted the customer's operations. Migrating from monolithic to queue-based architectures resulted in a huge 30-40% reduction in costs. While the integration of the Security Posture Management Tool significantly slashed security event resolution time by 50%, showcasing high responsiveness to potential threats. Leveraging Amazon RDS Proxy decreased failover times for Aurora and RDS databases by up to 66%, enhancing overall database resilience. Furthermore, the utilization of codebuild containers in the pipeline, leveraging cache, reduced deployment time by 40%. These results solidified Daffodil Software's position as a trusted partner in navigating the complexities of software development, leaving an enduring impact on client outcomes.