Implementing Proof of Concept
of Private Generative AI for Audit Automation
Website: waythru.com
Dedicated team behind the project
Solution Architect
Senior ML Engineer
Senior DevOps Engineer
Senior Data Engineer
The client
WayThru, a platform operating in the U.S. fintech space, approached us with a challenge related to automating security audit processes. As the platform evolved, the need for regular and detailed security audits increased. These audits required access to sensitive information and were handled primarily by senior management, resulting in a time-consuming process that placed additional strain on internal resources.
These audits required sensitive information, handled primarily by senior management, resulting in a time-consuming process that strained their resources.
The challenge
The primary goal of this Proof of Concept (PoC) was to determine the most suitable technological stack and framework for automating the completion of security audit questionnaires. To address this, we conducted extensive research to evaluate the current technological landscape and identify potential solutions.
After selecting the most promising options, we implemented basic functionality to test the feasibility of the project and ensure it could meet WayThru’s requirements. The solution needed to securely handle sensitive data, save time and resources, and be scalable to accommodate future growth while adhering to strict U.S. security regulations.
What was done
To ensure we delivered the best solution, we evaluated three potential setups before recommending Azure OpenAI Studio. Each option was analyzed based on its ability to meet the project’s key requirements of security, scalability, and efficiency:
The project’s key requirements
This setup allowed maximum control and customization, particularly in selecting models and embeddings for data processing. However, it required significant maintenance and was complex to manage, making it more suitable for larger projects with dedicated engineering teams. It was not the right fit for WayThru’s immediate needs due to high maintenance costs.
AWS Bedrock provided a stable and highly reliable environment with minimal bugs and an easy setup process. It allowed us to experiment with vector storage for optimized data retrieval. However, the platform offered a limited selection of models compared to other solutions, which could be a drawback for projects requiring a wide range of options.
Azure OpenAI Studio stood out for its maturity and extensive model availability, including high-performance GPT models from OpenAI and other industry leaders like Microsoft and Meta. The platform’s integration capabilities and user-friendly interface made it a strong candidate, especially for U.S.-based projects where data security and compliance are critical.
Implemented features:
LLM integration
Integrated Azure OpenAI Studio’s Large Language Models (LLMs) to automate responses for security audit questionnaires, ensuring high accuracy and reducing manual effort.
Custom API integration
Developed custom API solutions to seamlessly connect the audit automation tool with WayThru’s existing systems, ensuring smooth data flow and minimal disruption to current workflows.
Data encryption and secure data handling
Implemented strong data encryption protocols to protect sensitive audit information, in line with U.S. regulatory standards, including HIPAA and data privacy laws.
Model fine-tuning and optimization
Fine tuned GPT models within Azure to align with WayThru’s specific audit requirements, optimizing for both speed and accuracy.
Implementation of local LLM on different infrastructures
We implemented a local version of the LLM to give WayThru full control over their data processing environments. This deployment was configured across multiple infrastructures, including on-premises servers and cloud platforms.
Results:
Potential for 50% time reduction in audit processes
The PoC showed that by automating the completion of security audit questionnaires, WayThru could reduce the time senior management spends on audits by up to 50%, potentially saving around $18,000 annually.
30-40% potential reduction in operational costs
The PoC indicated that switching to a serverless architecture could reduce operational costs by 30-40%, primarily by eliminating the need for extensive infrastructure maintenance and dedicated engineering teams
Full compliance with U.S. data security regulations
The PoC verified that Azure’s platform is fully capable of meeting U.S. data security and compliance standards, including HIPAA, ensuring secure handling of sensitive audit data
Scalable solution for future growth
The PoC confirmed that Azure OpenAI Studio’s serverless design can support WayThru’s future growth without significant changes to infrastructure, ensuring scalability for evolving business needs.
Tech stack
Other сases
Team size: 5 developers
Team size: 7 developers
Team size: 7 developers
Team size: 3 developers
Team size: 6 developers