Achieve Enhanced Data Security using AWS Lake Formation within Amazon SageMaker Data Wrangler

Introduction:Amazon SageMaker Data Wrangler is a powerful tool that reduces the time and effort required to collect and prepare data for machine learning. With its user-friendly interface, you can streamline the process of feature engineering and data preparation, all within a single visual platform. Additionally, SageMaker Data Wrangler supports fine-grained data access control using AWS Lake Formation and Amazon Athena connections. This allows you to implement secure and tailored access to your data. In this post, we’ll show you how to leverage the capabilities of Amazon EMR and Lake Formation to enable fine-grained data access control with SageMaker Data Wrangler.

Full Article: Achieve Enhanced Data Security using AWS Lake Formation within Amazon SageMaker Data Wrangler

Immerse yourself in the world of Amazon’s latest innovation, SageMaker Data Wrangler. This powerful tool is revolutionizing the way data is collected and prepared for machine learning. With Data Wrangler, what used to take weeks can now be done in a matter of minutes!

Imagine being able to streamline the entire process of feature engineering and data preparation within a single visual interface. From data selection to purification, exploration, visualization, and processing at scale, SageMaker Data Wrangler has got you covered.

But that’s not all. With Data Wrangler, you also have the ability to implement fine-grained access control using AWS Lake Formation. This means you can control who has access to your data with a simple grant or revoke procedure. Data security has never been easier!

And now, we have some exciting news to share. SageMaker Data Wrangler now supports using Lake Formation with Amazon EMR. This integration allows data scientists to harness the power of Apache Spark, Hive, and Presto for fast data preparation. No more steep learning curves or complex processes. Everything can be done with just a few clicks.

Let’s dive deeper into this game-changing solution. We’ll walk you through an end-to-end use case using a sample dataset called the TPC data model. This dataset includes transaction data for products, customer demographics, inventory, web sales, and promotions. It’s the perfect example to showcase the capabilities of SageMaker Data Wrangler and Lake Formation.

To demonstrate the fine-grained data access permissions, let’s introduce two data scientists: David and Tina. David is part of the marketing team and his task is to build a model on customer segmentation. He only has access to non-sensitive customer data. On the other hand, Tina is on the sales team. She’s responsible for building the sales forecast model and needs access to sales data for a specific region. Additionally, she also needs access to product data for the product team’s innovation efforts.

So, how does all of this work? The architecture is set up as follows:

1. Lake Formation manages the data lake, with the raw data stored in Amazon S3 buckets.
2. Amazon EMR is used to query the data and perform data preparation using Spark.
3. AWS Identity and Access Management (IAM) roles are used to manage data access with Lake Formation.
4. SageMaker Data Wrangler serves as the single visual interface for interactively querying and preparing the data.

This powerful combination of technologies ensures efficient and secure data processing. But before you can get started, there are a few prerequisites. First, make sure you have an AWS account and an IAM user with administrator access. You’ll also need an S3 bucket and the necessary IAM roles for accessing the data.

To simplify the setup process, we provide a CloudFormation template that deploys all the required resources. This includes the data lake bucket, the EMR cluster with runtime roles, IAM roles for data access, a SageMaker Studio domain with user profiles, and a Lake Formation database pre-populated with the TPC data. We also take care of the networking resources, such as VPC, subnets, and security groups.

For added security, we recommend encrypting the data in transit. You can do this by creating PEM certificates and uploading them to an S3 bucket. We provide detailed instructions on how to generate and upload these certificates.

Once everything is set up, you can launch the CloudFormation template and let it create all the necessary resources for you. This usually takes around 10 to 15 minutes. Once the stack is created, you’ll need to update the External Data Filtering settings on Lake Formation to allow Amazon EMR to query the data.

Now comes the exciting part – testing the data access permissions. You can verify that David, who shouldn’t have access to any private customer information, is indeed restricted to non-sensitive data. And Tina, who needs access to sales and product data, can easily retrieve the information she needs. This level of granular control ensures that your data remains secure and only accessible to those who need it.

In conclusion, SageMaker Data Wrangler combined with Lake Formation and Amazon EMR is a game-changer for data professionals. The ability to collect, prepare, and analyze data with ease and efficiency is now a reality. Say goodbye to weeks of tedious work and hello to the future of machine learning data preparation. Amazon has once again raised the bar, setting new standards for the industry.

Summary: Achieve Enhanced Data Security using AWS Lake Formation within Amazon SageMaker Data Wrangler

Amazon SageMaker Data Wrangler is a tool that reduces the time it takes to collect and prepare data for machine learning. It streamlines the process of feature engineering and data preparation by providing a visual interface for data selection, purification, exploration, visualization, and processing. With the integration of AWS Lake Formation and Amazon EMR, users can implement fine-grained access control and run ad hoc SQL queries on Hive or Presto for data preparation. This article provides a detailed solution overview and instructions for setting up the architecture using AWS CloudFormation.