> ## Documentation Index > Fetch the complete documentation index at: https://docs.datazone.co/llms.txt > Use this file to discover all available pages before exploring further. # Data Ingestion > Data ingestion is the first step in Datazone's data journey. # Overview Data Ingestion is where data from various sources is imported into the platform for processing, analysis, and storage. This process is managed through two key concepts: [Sources](/key-concepts#source) and [Extracts](/key-concepts#extract). A **Source** holds the connection to an external system — credentials, host, project. You create it once and reuse it. An **Extract** selects what to pull from that source — a table, a collection, a file pattern — and how to pull it. One source can back many extracts. That split is also how configuration is split: | Layer | Where it is set | What it holds | | - | - | - | | **Connection parameters** | On the Source, at creation | Host, port, credentials, database, bucket, project ID | | **Extract parameters** | On the Extract, in `source_parameters` | Table or collection name, file pattern, batching, filtering | Each source page documents both. Parameter names match the API and CLI payloads. ## Getting Started To configure a new source: 1. Select your source type from the available connectors 2. Provide the required connection details 3. Start creating extracts from your source For detailed configuration steps, visit the specific documentation page for your chosen source type. ## Shared Extract Parameters These parameters apply to every extract regardless of source type. Source-specific parameters are documented on each connector's page. **Extract Mode** — `overwrite` replaces the dataset on every run (full table). `append` reads only new rows since the last run and appends them, using `replication_key` to track position. **Replication Key** — the column Datazone compares against the last extracted value to find new rows. Required when `mode` is `append`. Use a monotonically increasing column such as an auto-increment id or an `updated_at` timestamp. **Schema Mapping** — per-field selection, renaming, and type casting applied before the data is written. Each entry accepts `source_field`, `source_type`, `target_field` (rename, `null` keeps the original), `target_type` (cast, `null` keeps the inferred type), and `include` (set `false` to drop the field). **Custom Reader Config** — an escape hatch. Keys in this object are merged over the resolved reader configuration for every execution, overriding anything computed from the source and extract. Use it only for parameters not exposed elsewhere. The following go in the extract's `source_parameters` and are supported by all connectors that use the reader framework: Rows read and written per batch. Higher values reduce round trips but raise memory use. The default varies per connector — see each source page. Columns to read, as a comma-separated list (for example `id, name, created_at`). Defaults to every column. A row filter applied at the source, written without the `WHERE` keyword — for example `status = 'active' AND region = 'EU'`. Pushing filters down here is far cheaper than filtering later in a pipeline. `reset_state` is set per execution, not on the extract. It re-reads all data while still capturing the latest replication state, and is only valid for an `append` extract that already has state. ## Connectors Direct connection to MySQL and MySQL-compatible databases Direct connection to PostgreSQL databases Direct connection to SQL Server instances Direct connection to Oracle Database Document collections from MongoDB Tables and views from BigQuery datasets Delimited files from S3 buckets Files from Azure Blob containers Direct SQL access to SAP HANA SAP ERP tables via the CloudFeed connector SAP S/4HANA tables and CDS views via CloudFeed SAP BW InfoProviders via CloudFeed CRM objects from HubSpot This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.