Airflow API models

Option

Description

CLUSTER_VIEW_INVALID

Invalid cluster view value.

CLUSTER_VIEW_BASIC

Output basic information about the cluster.

CLUSTER_VIEW_FULL

Output full information about the cluster including allocated resources and Apache Airflow® settings.

Field

Description

version_id

string Version id of Apache Airflow®.

git_sync

doublecloud.airflow.v1.SyncConfig Git sync config

custom_image_digest

string Custom image digest, Apache Airflow® version must match current version_id.

managed_requirements_txt

string Target requirements.txt content to construct and apply a custom image in a managed way.

user_service_account_id

string Defines a service account that is avaialble for DC API auth from workers code.

env_vars

list Env variables that passing to your cluster

env_vars[]

doublecloud.airflow.v1.AirflowEnvVariable

Field

Description

name

string Name of environment variable

value

string Value of environment variable

Field

Description

id

string ID of the Apache Airflow® cluster.

project_id

string ID of the project that the Apache Airflow® cluster belongs to.

cloud_type

string Type of the cloud where instances should be hosted.

region_id

string ID of the region to place instances.

create_time

google.protobuf.Timestamp Creation timestamp.

name

string Name of the Apache Airflow® cluster. The name is unique within the project.

description

string Description of the Apache Airflow® cluster.

status

doublecloud.v1.ClusterStatus Current state of the cluster.

resources

doublecloud.airflow.v1.ClusterResources Resources allocated to Airflow cluster

connection_info

doublecloud.airflow.v1.ConnectionInfo Connection information.

access

doublecloud.v1.Access External network access settings

config

doublecloud.airflow.v1.AirflowConfig Configuration of the Apache Airflow® cluster.

network_id

string ID of the network that the Apache Airflow® cluster belongs to.

maintenance_window

doublecloud.v1.MaintenanceWindow Window of maintenance operations.

cr_connection_info

doublecloud.airflow.v1.CrConnectionInfo Custom images remote image path and credentials.

managed_image_status

doublecloud.airflow.v1.ManagedImageStatus Status of managed image.

Field

Description

airflow

doublecloud.airflow.v1.ClusterResources.Airflow Airflow specific cluster resource config

Field

Description

max_worker_count

google.protobuf.Int64Value Max number of Airflow workers

environment_flavor

string Flavor of Airflow cluster environment

min_worker_count

google.protobuf.Int64Value Min number of Airflow workers

worker_concurrency

google.protobuf.Int64Value Concurrency of Airflow workers

worker_disk_size

google.protobuf.Int64Value Disk size of Airflow workers (GiB)

worker_preset

string Preset of Airflow cluster workers

Field

Description

host

string Host to connect.

user

string Airflow admin user.

password

string Password for Airflow admin user.

Field

Description

remote_image_path

string Path to upload custom images.

user

string Container registry user.

password

string Container registry password.

Field

Description

username

string Username for repository auth

password

string Password for repository auth

Field

Description

build_errors

string If not empty, describes the latest managed image build errors.

applied_requirements_txt

string Currently applied requirements.txt.

Field

Description

repo_url

string Git repository for Airflow DAGs

branch

string Git branch

revision

string Git revision

dags_path

string Path of directory with DAGs

credentials

one of: api_credentials

api_credentials

doublecloud.airflow.v1.GitApiCredentials

Field

Description

cluster_id

string Required. ID of the creating Airflow cluster.

Field

Description

project_id

string ID of the project to create the Apache Airflow® cluster in.

cloud_type

string Type of the cloud where instances should be hosted.

region_id

string ID of the region to place instances.

name

string Name of the Apache Airflow® cluster. The name must be unique within the project.

description

string Description of the Apache Airflow® cluster.

resources

doublecloud.airflow.v1.ClusterResources Resources allocated to the Apache Airflow® cluster.

access

doublecloud.v1.Access External network access settings

config

doublecloud.airflow.v1.CreateClusterRequest.AirflowConfig

network_id

string ID of the network that the Apache Airflow® cluster belongs to.

maintenance_window

doublecloud.v1.MaintenanceWindow Window of maintenance operations.

Field

Description

version_id

string Version id of Apache Airflow®.

git_sync

doublecloud.airflow.v1.SyncConfig Git sync config

requirements_txt

string Requirements.txt content that will be used to construct and apply custom image.

user_service_account_id

string If supplied, defines a service account that would be available for DC API auth from workers code.

env_vars

list Env variables that passing to your cluster.

env_vars[]

doublecloud.airflow.v1.AirflowEnvVariable

Field

Description

digest

string Image digest.

airflow_version

string Version of underlying Apache Airflow®.

tags

list Custom image user tags.

tags[]

string

created_time

google.protobuf.Timestamp Custom image upload timestamp.

Field

Description

cluster_id

string Required. ID of the deleting Airflow cluster.

Field

Description

cluster_id

string ID of the Apache Airflow® cluster to delete. To get the Apache Airflow® cluster ID, use a [ClusterService.List] request.

Field

Description

cluster_id

string ID of the Apache Airflow® cluster resource to return.

sensitive

bool Whether to include sensitive data in the response. If unspecified, defaults to false.

Field

Description

cluster_id

string Required. ID of the cluster.

paging

doublecloud.v1.Paging Paging information of the request

Field

Description

operations

list

operations[]

doublecloud.v1.Operation

next_page

doublecloud.v1.NextPage Pagination information of the response

Field

Description

project_id

string ID of the project to list Apache Airflow® clusters in.

paging

doublecloud.v1.Paging Paging information of the request

view

doublecloud.airflow.v1.ClusterView The level of detail to be returned. If unspecified, defaults to CLUSTER_VIEW_BASIC.

Field

Description

clusters

list List of Apache Airflow® cluster resources.

clusters[]

doublecloud.airflow.v1.Cluster

next_page

doublecloud.v1.NextPage Pagination information of the response

Field

Description

cluster_id

string ID of the Apache Airflow® cluster.

Field

Description

custom_images

list List of custom images associated with the cluster.

custom_images[]

doublecloud.airflow.v1.CustomImage

Field

Description

cluster_id

string Required. ID of the Airflow cluster to maintenance reschedule.

delayed_until_time

google.protobuf.Timestamp The time for SPECIFIC_TIME reschedule. Limited by two weeks since first time scheduled.

Field

Description

cluster_id

string Required. ID of the Airflow cluster to maintenance reschedule.

reschedule_type

doublecloud.v1.RescheduleType Required. The type of reschedule request.

delayed_until_time

google.protobuf.Timestamp The time for SPECIFIC_TIME reschedule. Limited by two weeks since first time scheduled.

Field

Description

cluster_id

string ID of the Apache Airflow® Cluster resource to update. To get the Apache Airflow® cluster ID, use a [ClusterService.List] request.

description

google.protobuf.StringValue New description of the Apache Airflow® cluster.

resources

doublecloud.airflow.v1.UpdateClusterRequest.UpdateClusterResources New resources allocated to the Apache Airflow® cluster.

access

doublecloud.v1.Access Access settings

maintenance_window

doublecloud.v1.MaintenanceWindow Window of maintenance operations.

config

doublecloud.airflow.v1.UpdateClusterRequest.UpdateAirflowConfig Configuration and resources for hosts that should be created for the Apache Airflow® cluster.

name

google.protobuf.StringValue New name of the Apache Airflow® cluster.

Field

Description

git_sync

doublecloud.airflow.v1.UpdateClusterRequest.UpdateAirflowConfig.UpdateGitSyncConfig Git sync config

custom_image

one of: custom_image_digest / requirements_txt Custom image digest, Apache Airflow® version must match current version_id.

custom_image_digest

google.protobuf.StringValue Custom image digest, Apache Airflow® version must match current version_id.

requirements_txt

google.protobuf.StringValue Requirements.txt content that will be used to construct and apply custom image.

user_service_account_id

google.protobuf.StringValue Defines a service account that would be avaialble for DC API auth from workers code.

env_vars

list Env variables that passing to your cluster.

env_vars[]

doublecloud.airflow.v1.AirflowEnvVariable

Field

Description

git_sync

doublecloud.airflow.v1.SyncConfig Git sync config

Field

Description

airflow

doublecloud.airflow.v1.UpdateClusterRequest.UpdateClusterResources.Airflow Airflow specific cluster resource config

Field

Description

max_worker_count

google.protobuf.Int64Value Max number of Airflow workers

min_worker_count

google.protobuf.Int64Value Min number of Airflow workers

worker_concurrency

google.protobuf.Int64Value Concurrency of Airflow workers

worker_disk_size

google.protobuf.Int64Value Disk size of Airflow workers

worker_preset

google.protobuf.StringValue Preset of Airflow cluster workers

Field

Description

operation_id

string Required. ID of the operation to return.

Field

Description

project_id

string ID of the project to list clusters operations in.

paging

doublecloud.v1.Paging Paging information of the request

Field

Description

operations

list List of operations.

operations[]

doublecloud.v1.Operation

next_page

doublecloud.v1.NextPage Pagination information of the response