Reducing Deployment Time by 80%: Veriff’s Success with Amazon SageMaker Multi-Model Endpoints
Introduction:
Veriff, an identity verification platform, has partnered with various organizations in the financial services, FinTech, gaming, and other industries. They offer advanced technology combining AI-powered automation, human feedback, and expertise. Veriff ensures users’ identities and personal attributes are trusted throughout the customer journey, and they are trusted by companies such as Bolt, Deel, Trustpilot, and Wise. Veriff’s AI-powered solution requires running machine learning models in a cost-effective way, and they have standardized their model deployment workflow using Amazon SageMaker, reducing costs and development time.
Full News:
Veriff, an identity verification platform partner for various industries, has standardized its model deployment workflow using Amazon SageMaker, resulting in reduced costs and development time.
Veriff’s backend architecture relies on a microservices pattern, with services running on different Kubernetes clusters hosted on AWS infrastructure. This approach includes running expensive computer vision machine learning (ML) models on GPU instances. To save costs, Veriff developed a custom solution on Kubernetes to share GPU resources between different service replicas. However, this solution required data scientists to determine beforehand how much GPU memory their models would require and resulted in manual provisioning of GPU instances by DevOps, leading to operational overhead and overprovisioning.
To address these challenges, Veriff’s data science platform team opted to use Amazon SageMaker multi-model endpoints (MMEs). MMEs support NVIDIA’s Triton Inference Server, which allows data scientists to build REST APIs from models without writing code and provides compatibility with major AI frameworks. Triton also enables the deployment of model ensembles, which are groups of models chained together. Veriff utilizes this feature to deploy preprocessing and postprocessing logic with each ML model, ensuring seamless integration and accurate output in production.
SageMaker MMEs offer a scalable and cost-effective solution for real-time inference by using a shared serving container and a fleet of resources. This approach maximizes endpoint utilization compared to single-model endpoints and reduces deployment overhead. SageMaker also offers built-in capabilities for managing and monitoring models, including shadow variants, auto scaling, and integration with Amazon CloudWatch.
Veriff’s solution involves creating Triton model repositories, which include preprocessing and postprocessing code, trained model weights, ensemble model definitions, and dependencies. These repositories are compressed, stored in an Amazon S3 bucket connected to the MME, and detected and served by SageMaker.
To streamline the deployment process, Veriff built a monorepo that houses all models to be deployed to MMEs. Data scientists collaborate in a Gitflow-like approach within this monorepo, and continuous integration tools automate the necessary steps to deploy models, including code quality checks, downloading model weights, building Conda environments, and spinning up Triton servers.
With Amazon SageMaker, Veriff has achieved a standardized model deployment workflow that simplifies the transition from research to production, reduces operational costs, and optimizes GPU instance provisioning. This solution allows Veriff to scale its offerings and provide innovative, hyper-personalized solutions to its customers.
Conclusion:
Veriff, an identity verification platform, has partnered with various organizations in finance, fintech, gaming, and more. They offer advanced technology that combines AI automation with human feedback to ensure trust in user identities. Veriff has standardized its model deployment workflow using Amazon SageMaker, reducing costs and development time while providing a hyper-personalized solution for customers.
Frequently Asked Questions:
1. How did Veriff manage to decrease deployment time by 80%?
Veriff achieved a significant reduction in deployment time by implementing Amazon SageMaker multi-model endpoints. This advanced technology allows Veriff to deploy models at a much faster pace compared to traditional methods, resulting in a remarkable 80% reduction in deployment time.
2. What is Amazon SageMaker multi-model endpoint?
Amazon SageMaker multi-model endpoint is a powerful feature that enables organizations like Veriff to host multiple machine learning models within a single endpoint. This eliminates the need to create separate endpoints for each model, saving time and simplifying deployment processes.
3. How does Amazon SageMaker multi-model endpoint contribute to faster deployments?
With Amazon SageMaker multi-model endpoints, Veriff can upload and manage multiple models simultaneously, reducing the time required to deploy each model individually. This streamlined approach dramatically improves efficiency and speeds up the deployment process by up to 80%.
4. What are the benefits of using Amazon SageMaker multi-model endpoints?
Using Amazon SageMaker multi-model endpoints offers several benefits for organizations like Veriff. Apart from faster deployment times, it allows for cost optimization by consolidating models in a single endpoint. Additionally, it simplifies maintenance and reduces complexity in managing multiple endpoints.
5. Is Amazon SageMaker multi-model endpoint suitable for all types of machine learning models?
Yes, Amazon SageMaker multi-model endpoint supports a wide range of machine learning models, including traditional statistical models, deep learning models, and custom models. Veriff can leverage this capability to host and serve various types of models using a centralized endpoint.
6. What challenges did Veriff face before implementing Amazon SageMaker multi-model endpoints?
Prior to implementing Amazon SageMaker multi-model endpoints, Veriff had to create separate endpoints for each machine learning model, which increased complexity and deployment time. This approach was not scalable and hindered efficient development and deployment processes.
7. How does Veriff benefit from the reduction in deployment time?
Veriff’s reduction in deployment time by 80% enables them to quickly iterate and improve their machine learning models. They can swiftly respond to customer needs and deliver more timely updates and enhancements, ultimately enhancing the overall user experience.
8. Can Veriff easily manage multiple machine learning models with Amazon SageMaker multi-model endpoints?
Absolutely. Amazon SageMaker multi-model endpoints provide Veriff with a user-friendly interface to upload, deploy, and manage multiple machine learning models efficiently. This simplifies the overall management process and allows Veriff to focus more on improving their models.
9. Does Veriff need extensive technical expertise to utilize Amazon SageMaker multi-model endpoints?
While some technical knowledge is beneficial, Veriff does not necessarily need extensive expertise to leverage Amazon SageMaker multi-model endpoints. Amazon provides comprehensive documentation and resources to guide users through the deployment and management processes.
10. Can other organizations benefit from Amazon SageMaker multi-model endpoints like Veriff did?
Definitely. Organizations that rely on machine learning models can benefit greatly from the usage of Amazon SageMaker multi-model endpoints. By adopting this technology, they can significantly reduce deployment time, enhance efficiency, and accelerate the development and deployment of their models.
