Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
✨ Introducing Embedded Agents! Drop your Datazone agents into any website, app, or third-party tool with a single line of code. See Embedding Agents for details.
Each pipeline in Datazone has a context object that provides access to resources and configuration settings.
@transform
def my_transform(context):
context.state.write(key="my_key", value="my_value")
state attribute that allows you to read and write key-value pairs.
The state is stored in database and can be accessed in any transform function in the pipeline.
Also you can access the state in the next executions of the pipeline.
from datazone import transform
@transform
def my_transform(context):
context.state.write(key="my_key", value="my_value")
@transform(depends=[my_transform])
def my_transform(context):
value = context.state.read(key="my_key")
print(value)
success, failure, now.
from datazone import transform
@transform
def my_transform(context):
context.state.write(key="my_key", value="my_value", on="success")
raise Exception("Error")
pyspark attribute of the resources object.
from datazone import transform
@transform
def my_transform(context):
spark = context.resources["pyspark"].spark
# Use the PySpark session
df = spark.createDataFrame({
"name": ["Alice", "Bob"],
"age": [25, 30]
})
return df