> ## Documentation Index
> Fetch the complete documentation index at: https://docs.datazone.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Project Repository

> Organize and deploy your pipelines, actions, apps, and endpoints in a single project structure.

## Overview

A Datazone project is a **Git repository** containing your data pipelines, actions, intelligent apps, and endpoints. All project components are defined in a central **`config.yml`** file.

## Project Structure

<Tree>
  <Tree.Folder name="my-project" defaultOpen>
    <Tree.File name="config.yml" />

    <Tree.Folder name="pipelines" defaultOpen>
      <Tree.File name="hello_world.py" />

      <Tree.File name="data_processing.py" />
    </Tree.Folder>

    <Tree.Folder name="actions">
      <Tree.File name="send_notification.py" />
    </Tree.Folder>

    <Tree.Folder name="apps">
      <Tree.File name="my_app.py" />
    </Tree.Folder>

    <Tree.File name="utils.py" />
  </Tree.Folder>
</Tree>

<Note>
  * Each project **must have** a `config.yml` file
  * Each pipeline should have a **unique alias** and be defined in separate files
  * You can organize your code with utility files for shared logic
</Note>

## Configuration File

The `config.yml` file defines all project resources:

```yaml config.yml theme={null}
project_name: my-pretty-project
project_id: 67280ba2f4a0960d02159675

pipelines:
  - alias: hello_world
    path: pipelines/hello_world.py
    compute: LARGE
    spark_config:
      deploy_mode: client
      executor_instances: 3
    python_dependencies:
      - name: pandas
        version: 1.3.3

actions:
  - path: actions/send_notification.py

apps:
  - path: apps/my_app.py

endpoints:
  - path: endpoints/api_config.yml
```

## Configuration Reference

### Project Fields

<ParamField path="project_name" type="string" required>
  Name of your project. Used for display and identification.

  ```yaml theme={null}
  project_name: data-processing-platform
  ```
</ParamField>

<ParamField path="project_id" type="string" required>
  Unique identifier for your project. Generated when you create a project.

  ```yaml theme={null}
  project_id: 67280ba2f4a0960d02159675
  ```
</ParamField>

<ParamField path="pipelines" type="array">
  List of data pipeline definitions. Each pipeline processes and transforms data.

  ```yaml theme={null}
  pipelines:
    - alias: etl_pipeline
      path: pipelines/etl.py
  ```
</ParamField>

<ParamField path="actions" type="array">
  List of serverless action functions. Actions can be triggered by endpoints or used by AI agents.

  ```yaml theme={null}
  actions:
    - path: actions/send_email.py
  ```

  Learn more in the [Actions documentation](/reference/development/actions).
</ParamField>

<ParamField path="apps" type="array">
  List of intelligent AI applications.

  ```yaml theme={null}
  apps:
    - path: apps/customer_support.py
  ```
</ParamField>

<ParamField path="endpoints" type="array">
  List of API endpoint configurations.

  ```yaml theme={null}
  endpoints:
    - path: endpoints/webhooks.yml
  ```

  Learn more in the [Endpoints documentation](/reference/integration/endpoints).
</ParamField>

### Pipeline Configuration

<ParamField path="alias" type="string" required>
  Short, unique identifier for the pipeline. Used in CLI commands and UI.

  ```yaml theme={null}
  alias: daily_etl
  ```
</ParamField>

<ParamField path="path" type="string" required>
  Relative path to the pipeline Python file from project root.

  ```yaml theme={null}
  path: pipelines/etl_pipeline.py
  ```
</ParamField>

<ParamField path="compute" type="string" default="SMALL">
  Compute instance size for pipeline execution.

  **Available sizes:**

  * `XSMALL` - 2 vCPU, 8 GB RAM
  * `SMALL` - 4 vCPU, 16 GB RAM
  * `MEDIUM` - 8 vCPU, 32 GB RAM
  * `LARGE` - 16 vCPU, 64 GB RAM
  * `XLARGE` - 32 vCPU, 128 GB RAM (Enterprise only)

  ```yaml theme={null}
  compute: LARGE
  ```
</ParamField>

<ParamField path="spark_config.deploy_mode" type="string" default="local">
  Spark deployment mode for distributed processing.

  **Options:**

  * `local` - Runs on a single machine (default)
  * `client` - Driver runs in the same process, executors run separately
  * `cluster` - Both driver and executors run in separate processes (Enterprise only)

  ```yaml theme={null}
  spark_config:
    deploy_mode: client
  ```
</ParamField>

<ParamField path="spark_config.executor_instances" type="integer">
  Number of Spark executors for parallel processing. Only applies when `deploy_mode` is `client` or `cluster`.

  ```yaml theme={null}
  spark_config:
    deploy_mode: client
    executor_instances: 5
  ```
</ParamField>

<ParamField path="spark_config.extra_spark_config" type="object">
  Additional Spark configuration properties. Pass any custom Spark configuration key-value pairs.

  ```yaml theme={null}
  spark_config:
    deploy_mode: client
    executor_instances: 3
    extra_spark_config:
      spark.sql.shuffle.partitions: "200"
      spark.default.parallelism: "100"
      spark.sql.adaptive.enabled: "true"
  ```
</ParamField>

<ParamField path="python_dependencies" type="array">
  Python packages required by the pipeline. Installed before execution.

  ```yaml theme={null}
  python_dependencies:
    - name: pandas
      version: 2.0.0
    - name: requests
      version: 2.31.0
      index_url: https://pypi.org/simple
  ```
</ParamField>

### Python Dependency Fields

<ParamField path="name" type="string" required>
  Python package name from PyPI or custom index.

  ```yaml theme={null}
  - name: numpy
  ```
</ParamField>

<ParamField path="version" type="string">
  Specific package version. If omitted, installs the latest version.

  ```yaml theme={null}
  - name: pandas
    version: 2.0.0
  ```
</ParamField>

<ParamField path="index_url" type="string">
  Custom Python package index URL. Useful for private packages or mirrors.

  ```yaml theme={null}
  - name: my-private-package
    version: 1.2.3
    index_url: https://pypi.mycompany.com/simple
  ```
</ParamField>

### Action Configuration

<ParamField path="path" type="string" required>
  Relative path to the action Python file containing an `@action` decorated function.

  ```yaml theme={null}
  actions:
    - path: actions/send_notification.py
    - path: workflows/data_validator.py
  ```

  Each file should contain **one action function**. Learn more in the [Actions documentation](/reference/development/actions).
</ParamField>

### App Configuration

<ParamField path="path" type="string" required>
  Relative path to the intelligent app Python file.

  ```yaml theme={null}
  apps:
    - path: apps/customer_assistant.py
  ```
</ParamField>

### Endpoint Configuration

<ParamField path="path" type="string" required>
  Relative path to the endpoint configuration YAML file.

  ```yaml theme={null}
  endpoints:
    - path: endpoints/webhooks.yml
    - path: endpoints/api_routes.yml
  ```

  Learn more in the [Endpoints documentation](/reference/integration/endpoints).
</ParamField>

## Example Projects

### Data Processing Pipeline

```yaml theme={null}
project_name: sales-analytics
project_id: 67280ba2f4a0960d02159675

pipelines:
  - alias: daily_sales_etl
    path: pipelines/sales_pipeline.py
    compute: MEDIUM
    spark_config:
      deploy_mode: client
      executor_instances: 3
    python_dependencies:
      - name: pandas
        version: 2.0.0
      - name: sqlalchemy
        version: 2.0.0
```

### Multi-Component Project

```yaml theme={null}
project_name: customer-platform
project_id: 67280ba2f4a0960d02159675

pipelines:
  - alias: customer_data_sync
    path: pipelines/sync.py
    compute: SMALL
    python_dependencies:
      - name: requests
      - name: pydantic
        version: 2.0.0

actions:
  - path: actions/send_welcome_email.py
  - path: actions/validate_customer.py

apps:
  - path: apps/support_chatbot.py

endpoints:
  - path: endpoints/customer_api.yml
```

## Next Steps

* [Learn about Pipelines](/reference/development/pipeline)
* [Create Actions](/reference/development/actions)
* [Build Intelligent Apps](/reference/agents/apps)
* [Set up Endpoints](/reference/integration/endpoints)