> ## Documentation Index
> Fetch the complete documentation index at: https://docs.datazone.co/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS S3 CSV

> AWS S3 is a scalable object storage service that can be used to store and retrieve files.

# Overview

Amazon Simple Storage Service (S3) is an object storage service offering industry-leading scalability, availability, and durability. Datazone provides native integration with AWS S3 to read data files directly from your S3 buckets.

## Connection Parameters

Set on the source when you create it. Names match the API and CLI payload.

<ParamField path="name" type="string" required>
  **Name** — a unique identifier for your AWS S3 source.
</ParamField>

<ParamField path="bucket_name" type="string" required>
  **Bucket Name** — the S3 bucket containing your data files.
</ParamField>

<ParamField path="aws_access_key_id" type="string" required>
  **AWS Access Key ID** — the access key ID from your AWS credentials.
</ParamField>

<ParamField path="aws_secret_access_key" type="string" required>
  **AWS Secret Access Key** — the secret access key from your AWS credentials. Stored encrypted.
</ParamField>

<ParamField path="aws_region" type="string" default="us-east-1">
  **AWS Region** — the region the bucket lives in, for example `eu-west-2`.
</ParamField>

## Extract Parameters

Set per extract, in `source_parameters`. See [Shared Extract Parameters](/reference/sources/overview#shared-extract-parameters) for `mode`, `replication_key`, and `schema_mapping`.

<ParamField path="search_pattern" type="string" required>
  **Search Pattern** — a regular expression matched against object keys in the bucket. Every matching object is read as one dataset, so all matches must share a schema. For example `^exports/orders/.*\.csv$`.
</ParamField>

<ParamField path="search_prefix" type="string">
  **Search Prefix** — an S3 key prefix that limits which objects are listed before the pattern is applied, for example `exports/orders/`. Always set this on a large bucket: without it, every key is listed on every run.
</ParamField>

<ParamField path="delimiter" type="string" default=",">
  Field separator in the files. Use `\t` for tab-separated exports.
</ParamField>

<ParamField path="encoding" type="string" default="utf-8">
  Character encoding of the files.
</ParamField>

<ParamField path="batch_size" type="integer" default="1000000">
  Rows read and written per batch.
</ParamField>

<ParamField path="select_clause" type="string" default="*">
  Columns to read, as a comma-separated list.
</ParamField>

<ParamField path="filter_clause" type="string">
  Row filter applied to the parsed rows.
</ParamField>

<Note>
  In `append` mode the extract tracks the last-modified timestamp of the objects it has read, so a later run picks up newly written files rather than re-reading the whole prefix.
</Note>

## Required Permissions

The AWS IAM user account needs the following permissions on the specified S3 bucket:

* `s3:GetObject` - For reading files from the bucket
* `s3:ListBucket` - For listing contents of the bucket
* `s3:GetBucketLocation` - For determining the bucket's region

## Limitations

Be aware of the following limitations when working with AWS S3 CSV sources:

* CSV, TXT, Parquet, JSON files are supported
* UTF-8 encoding is recommended
* Individual file size limits apply based on your AWS S3 configuration
* The S3 bucket and Datazone instance should ideally be in the same region for optimal performance
* Cross-region access may incur additional AWS charges

## Next Steps

After configuring your AWS S3 source:

1. Create extracts to specify which CSV files to ingest
2. Configure scheduling for recurring extracts
3. Integrate the source into your data pipelines

For more information about working with extracts and pipelines, refer to their respective documentation sections.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.