Loader AWS S3
AWS S3, or Amazon Simple Storage Service, is a foundational cloud storage service provided by Amazon Web Services (AWS). It delivers a scalable and secure solution for storing and retrieving data over the internet, designed to provide durability, availability, and low-latency access to stored objects. This makes it a fundamental component for managing digital assets, backups, and data archives in the cloud.
Business value in CDP
The AWS S3 Loader within Meiro Integration enhances data management by streamlining the file upload process, ensuring a seamless data transition from CDP to AWS S3. Beyond efficient data management, businesses also gain from the secure cloud storage capabilities of AWS S3, providing a reliable and compliant environment for their data.
Setting up the loader in MI
For setting AWS S3 as a destination within Meiro Integration, use the AWS S3 loader component.
Requirements
For setting up a configuration of the AWS S3 connector, you need an AWS account with a created S3 bucket in it. You can create your account here. If you have not used AWS before, we recommend you to check these articles:
- How to create and activate AWS account
- How to create an S3 Bucket
- How do I create an AWS Access Key?
- Where is My Secret Access Key?
- Best Practices for Managing AWS Access Keys
Data In/Data Out
| Data In | The files that need to be uploaded should be in the folder in/files/. Be aware that if your files are located in the in/tables they won’t be written by AWS S3 loader to your AWS S3 bucket. |
| Data Out | A folder in the S3 bucket for uploading the files is defined in a prefix field. According to the Amazon S3 structure, the names of the files will be changed to “prefix + “/” + name of the file”. |
Learn more: how to move the files using a CLI processor please refer to this article.
Parameters
|
Access Key ID (required) |
The AWS Access Key ID, looks like AKIA**** and you need to create it in the Credential section of your AWS S3 account: My_AWS -> My Security Credentials -> Access keys (access key ID and secret access key) -> Create New Access Key -> Download Key File |
| Secret Access Key (required) |
The AWS Secret Access Key is provided by the AWS when you create a new AWS Access Key: My_AWS -> My Security Credentials -> Access keys (access key ID and secret access key) -> Create New Access Key -> Download Key File. |
| Bucket (required) | Provide an AWS S3 bucket name which is a globally unique identifier and the region will be autodetected. |
|
Prefix (optional) |
Provide a prefix to key names of the files For example, if you wish to upload data in the directory |
Learn more: on how to create your AWS S3 Access Key can be found here.
Learn more: on how to create your AWS S3 Secret Access Key can be found here.
No Comments