Developing an AI-based smart monitoring and anomaly detection system for oil & gas turbomachinery

The project began with our business analysts and software architects outlining the optimal architecture and design for the solution. They refined the functional requirements and developed a complete product vision and its development roadmap. Once the strategic plans were mutually finalized, Daffodil’s team mapped the requirements onto the technology landscape and suggested technologies such as Apache Airflow, AWS EMR, Kubernetes, Node JS, React JS, MongoDB, Data lake, and Pyspark, etc. for the solution. The user experience was one of the project’s cornerstones. We focused on creating intuitive usability to facilitate platform adoption, as well as lucid visualization tools to allow for easy comparisons of complex data. Our engineers designed, developed, and deployed all the infrastructure on AWS Cloud

The Mechademy platform receives data from more than a hundred different sensors integrated with the plant ‘historian’ (software that stores historical data of equipment) as input via a connector agent. The connector agent fetches the data from plant historians to Mechademy data lake. Multiple ETL pipelines (remove missing data, perform data type check, remove outliers, etc.) transforms the plant data in the desired format. The final cleaned data is then fed to various performance modeling algorithms and ML/DL algorithms. 

These algorithms are trained using the historical data to identify anomalies in data and generate prescriptive alerts. These alerts help the end users in making informed decisions about the health and efficiency of the platform. The platform is also capable of plotting graphs for each data point such as raw data, processed data or output data. The dashboard can be customized for each client as per their requirements to give a bird’s eye view or drilled down details about the plant’s performance. Some of the other features that were included in the solution are:

• Authentication with user role group and permissions.
  Creating plant layouts (machine and equipment schematic) in the system.
• Configuring different machine-specific parameters.
• Editing/ Deploying/Undeploying the layouts.
• IOT Data ingestion and Complex ETL based data pipelines creation for catching anomalies and outliers.
• Generating prescriptive alerts from the machine ingestion data.
• Applying machine-specific process algorithms based on config parameters.
• Deploying the customized Machine learning and Deep learning-based models dynamically from the admin frontend.
• Creating customized dashboards dynamically as per client requirements.