> ## Documentation Index > Fetch the complete documentation index at: https://private-7c7dfe99-detect-table-modification.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Update ClickPipe > Update the specified ClickPipe. Update the specified ClickPipe. ## OpenAPI ````yaml /_specs/cloud-openapi.json patch /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId} openapi: 3.1.2 info: title: OpenAPI spec for ClickHouse Cloud version: '1.0' contact: name: ClickHouse Support url: >- https://clickhouse.com/docs/en/cloud/manage/openapi?referrer=openapi-910487 email: support@clickhouse.com servers: - url: https://api.clickhouse.cloud security: - basicAuth: [] tags: - name: Organization - name: User management - name: Billing - name: Role Management - name: Service - name: Backup - name: OpenAPI - name: Prometheus - name: ClickPipes - name: ClickStack - name: Postgres paths: /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}: patch: tags: - ClickPipes summary: Update ClickPipe description: Update the specified ClickPipe. operationId: clickPipeUpdate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service to create the ClickPipe for. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the requested ClickPipe. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipePatchRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipe' '400': description: >- The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: >- An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid components: schemas: ClickPipePatchRequest: properties: name: description: Name of the ClickPipe. type: - string - 'null' source: oneOf: - $ref: '#/components/schemas/ClickPipePatchSource' - type: 'null' destination: oneOf: - $ref: '#/components/schemas/ClickPipePatchDestination' - type: 'null' fieldMappings: type: array description: >- Field mappings of the ClickPipe. This will not update the table schema, only the ClickPipe configuration. items: $ref: '#/components/schemas/ClickPipeFieldMapping' settings: oneOf: - $ref: '#/components/schemas/ClickPipeSettings' - type: 'null' ClickPipe: properties: id: description: Unique ClickPipe ID. type: string format: uuid serviceId: description: ID of the service this ClickPipe belongs to. type: string format: uuid name: description: Name of the ClickPipe. type: string example: my_postgres_pipe maxLength: 255 minLength: 1 state: description: >- Current lifecycle state of the ClickPipe. For database pipes: "Provisioning" (initial setup), "Setup" (configuring replication), "Snapshot" (initial data load), "Running" (actively replicating), "Pausing" (transitioning to paused state), "Paused" (temporarily paused), "Modifying" (applying configuration updates), "Resync" (swapping resync tables with original tables), "Failed" (error occurred), "Unknown". For streaming/object storage pipes (Kafka, Kinesis, S3): "Unknown" (initial state), "Provisioning" (setting up resources), "Running" (actively ingesting data), "Stopping" (transitioning to stopped state), "Stopped" (manually stopped, can be restarted), "Completed" (batch ingestion finished for object storage), "Failed" (error occurred, pipe stopped), "InternalError" (internal system error). type: string enum: - Unknown - Provisioning - Running - Stopping - Stopped - Failed - Completed - InternalError - Setup - Snapshot - Paused - Pausing - Modifying - Resync example: Running scaling: $ref: '#/components/schemas/ClickPipeScaling' source: $ref: '#/components/schemas/ClickPipeSource' destination: $ref: '#/components/schemas/ClickPipeDestination' fieldMappings: type: array description: >- Field mappings of the ClickPipe. Note that all destination columns must be included in the mappings. items: $ref: '#/components/schemas/ClickPipeFieldMapping' settings: $ref: '#/components/schemas/ClickPipeSettings' createdAt: description: Creation timestamp of the ClickPipe in ISO 8601 format. type: string format: date-time updatedAt: description: Last update timestamp of the ClickPipe in ISO 8601 format. type: string format: date-time ClickPipePatchSource: properties: kafka: oneOf: - $ref: '#/components/schemas/ClickPipePatchKafkaSource' - type: 'null' objectStorage: oneOf: - $ref: '#/components/schemas/ClickPipePatchObjectStorageSource' - type: 'null' kinesis: oneOf: - $ref: '#/components/schemas/ClickPipePatchKinesisSource' - type: 'null' pubsub: oneOf: - $ref: '#/components/schemas/ClickPipePatchPubSubSource' - type: 'null' postgres: $ref: '#/components/schemas/ClickPipePatchPostgresSource' mysql: oneOf: - $ref: '#/components/schemas/ClickPipePatchMySQLSource' - type: 'null' mongodb: oneOf: - $ref: '#/components/schemas/ClickPipePatchMongoDBSource' - type: 'null' validateSamples: description: >- Validate data samples received from data source. It will validate the connection and data availability and correctness. If not enabled, only connection will be validated. This has no effect on Postgres or MySQL pipes, they always only validate the connection and table definitions. This is experimental and can be removed in the future. type: boolean ClickPipePatchDestination: properties: columns: type: array description: >- Columns of the destination table. This will not update the table schema, only the ClickPipe configuration. items: $ref: '#/components/schemas/ClickPipeDestinationColumn' ClickPipeFieldMapping: properties: sourceField: description: Source field name. type: string destinationField: description: Destination field name. type: string ClickPipeSettings: properties: streaming_max_insert_wait_ms: description: >- Streaming max insert wait time. Configures the max wait period before inserting data into the ClickHouse. type: - integer - 'null' minimum: 500 maximum: 60000 example: 5000 object_storage_concurrency: description: >- Object storage concurrency. Number of concurrent file processing threads type: - integer - 'null' minimum: 1 maximum: 35 example: 1 object_storage_polling_interval_ms: description: >- Object storage polling interval. Configures the refresh interval for querying continuous ingest for new object storage data type: - integer - 'null' minimum: 100 maximum: 3600000 example: 30000 object_storage_max_insert_bytes: description: >- Max insert bytes. Number of bytes to process in a single insert batch type: - integer - 'null' minimum: 10485760 maximum: 53687091200 example: 10737418240 object_storage_max_file_count: description: >- Max file count. Maximum number of files to process in a single insert batch type: - integer - 'null' minimum: 1 maximum: 10000 example: 100 clickhouse_max_threads: description: >- Max threads. Maximum number of concurrent threads for file processing type: - integer - 'null' minimum: 0 maximum: 64 example: 8 clickhouse_max_insert_threads: description: Max insert threads. Maximum number of concurrent insert threads type: - integer - 'null' minimum: 0 maximum: 16 example: 1 clickhouse_min_insert_block_size_bytes: description: >- Min insert block size bytes. Minimum size of data block for insert (in bytes) type: - integer - 'null' minimum: 0 maximum: 10737418240 example: 1073741824 clickhouse_max_download_threads: description: Max download threads. Maximum number of concurrent download threads type: - integer - 'null' minimum: 0 maximum: 32 example: 4 clickhouse_parallel_distributed_insert_select: description: >- Parallel distributed insert select. Parallel distributed insert select setting type: - integer - 'null' minimum: 0 maximum: 2 example: 2 object_storage_use_cluster_function: description: >- use cluster function. Whether to use ClickHouse cluster function for distributed processing type: - boolean - 'null' example: true clickhouse_parallel_view_processing: description: >- parallel view processing. Whether to enable pushing to attached views concurrently instead of sequentially type: - boolean - 'null' example: false ClickPipeScaling: properties: replicas: description: Desired number of replicas. Only for scalable pipes. type: integer minimum: 1 maximum: 40 concurrency: description: >- Desired number of concurrency. Only for S3 pipes. If set to 0, concurrency is auto-scaled based on the cluster memory. type: integer deprecated: true replicaCpuMillicores: description: CPU in millicores for each replica. Only for streaming pipes. type: integer minimum: 125 maximum: 2000 replicaMemoryGb: description: Memory in GB for each replica. Only for streaming pipes. type: number minimum: 0.5 maximum: 8 ClickPipeSource: properties: kafka: oneOf: - $ref: '#/components/schemas/ClickPipeKafkaSource' - type: 'null' objectStorage: oneOf: - $ref: '#/components/schemas/ClickPipeObjectStorageSource' - type: 'null' kinesis: oneOf: - $ref: '#/components/schemas/ClickPipeKinesisSource' - type: 'null' pubsub: oneOf: - $ref: '#/components/schemas/ClickPipePubSubSource' - type: 'null' postgres: oneOf: - $ref: '#/components/schemas/ClickPipePostgresSource' - type: 'null' mysql: oneOf: - $ref: '#/components/schemas/ClickPipeMySQLSource' - type: 'null' bigquery: oneOf: - $ref: '#/components/schemas/ClickPipeBigQuerySource' - type: 'null' mongodb: oneOf: - $ref: '#/components/schemas/ClickPipeMongoDBSource' - type: 'null' ClickPipeDestination: properties: database: description: Destination database. type: string table: description: >- Destination table. Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). type: string managedTable: description: >- Is the table managed by ClickPipes? Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). type: boolean tableDefinition: $ref: '#/components/schemas/ClickPipeDestinationTableDefinition' columns: type: array description: >- Columns of the destination table. Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). items: $ref: '#/components/schemas/ClickPipeDestinationColumn' ClickPipePatchKafkaSource: properties: authentication: description: >- Authentication method of the Kafka source. Supported authentication methods: kafka: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, msk: SCRAM-SHA-512, IAM_ROLE, IAM_USER, MUTUAL_TLS, gcmk: PLAIN, MUTUAL_TLS, confluent: PLAIN, MUTUAL_TLS, warpstream: PLAIN, azureeventhub: PLAIN, redpanda: SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, dokafka: SCRAM-SHA-256, MUTUAL_TLS type: - string - 'null' enum: - PLAIN - SCRAM-SHA-256 - SCRAM-SHA-512 - IAM_ROLE - IAM_USER - MUTUAL_TLS iamRole: description: >- IAM role for the Kafka source. Use with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/kafka#iam type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole caCertificate: description: PEM encoded CA certificates to validate the broker's certificate. type: - string - 'null' reversePrivateEndpointIds: type: array description: >- Reverse private endpoint UUIDs used for a secure private connection to the Kafka source. items: type: string credentials: description: >- Credentials for Kafka source. Choose one that is supported by the authentication method. oneOf: - $ref: '#/components/schemas/PLAIN' - $ref: '#/components/schemas/MskIamUser' - $ref: '#/components/schemas/AzureEventHub' - $ref: '#/components/schemas/MutualTLS' ClickPipePatchObjectStorageSource: properties: skipInitialLoad: description: >- If set to true, skips the initial load and only ingests files delivered by queue notifications. Only applicable when queueUrl is provided. type: - boolean - 'null' startAfter: description: >- Start continuous ingestion after this object key. Cannot be provided when skipInitialLoad is true. type: - string - 'null' example: events/2026-06-01/ authentication: description: >- Authentication method. IAM_USER is for S3, GCS, and DigitalOcean Spaces. IAM_ROLE is for S3 only. SERVICE_ACCOUNT is for GCS only. CONNECTION_STRING is for Azure Blob Storage. PUBLIC uses no authentication. type: - string - 'null' enum: - IAM_ROLE - IAM_USER - CONNECTION_STRING - SERVICE_ACCOUNT iamRole: description: >- IAM role to be used with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#authentication type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole connectionString: description: >- Connection string for Azure Blob Storage authentication. Required when authentication is CONNECTION_STRING. type: - string - 'null' example: >- DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;EndpointSuffix=core.windows.net path: description: >- Path to the file(s) within the Azure container. Used for Azure Blob Storage sources. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations type: - string - 'null' example: data/logs/*.json azureContainerName: description: >- Container name for Azure Blob Storage. Required when type is azureblobstorage. type: - string - 'null' example: mycontainer accessKey: oneOf: - $ref: '#/components/schemas/MskIamUser' - type: 'null' serviceAccountKey: description: >- Base64-encoded GCP service account JSON key. Required when authentication is SERVICE_ACCOUNT. type: - string - 'null' ClickPipePatchKinesisSource: properties: authentication: description: Authentication method to use with the Kinesis stream. type: - string - 'null' enum: - IAM_ROLE - IAM_USER iamRole: description: IAM role to use for authentication. Required if IAM_ROLE is used. type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole accessKey: oneOf: - $ref: '#/components/schemas/MskIamUser' - type: 'null' ClickPipePatchPubSubSource: properties: authentication: description: >- Authentication method to use with GCP Pub/Sub. Currently only SERVICE_ACCOUNT is supported. type: - string - 'null' enum: - SERVICE_ACCOUNT example: SERVICE_ACCOUNT ackDeadline: description: >- Acknowledgement deadline for messages, in seconds. Must be between 10 and 600. type: - integer - 'null' minimum: 10 maximum: 600 serviceAccountKey: oneOf: - $ref: '#/components/schemas/ServiceAccount' - type: 'null' required: - authentication - serviceAccountKey ClickPipePatchPostgresSource: properties: credentials: $ref: '#/components/schemas/PLAIN' host: description: >- PostgreSQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: - string - 'null' format: hostname example: my-postgres-server.example.com port: description: PostgreSQL server port. type: - integer - 'null' minimum: 1 maximum: 65535 example: 5432 database: description: PostgreSQL database name to replicate from. type: - string - 'null' example: production_db tlsHost: description: TLS/SSL host for secure connections. type: - string - 'null' example: my-postgres-server.example.com caCertificate: description: >- PEM encoded CA certificate to validate the Postgres server certificate. type: - string - 'null' example: |- -----BEGIN CERTIFICATE----- ... disableTls: description: >- Disable TLS for the Postgres connection. Use with caution in production environments. type: - boolean - 'null' example: false skipCertVerification: description: >- Skip TLS certificate verification for the Postgres connection. Use with caution in production environments. type: - boolean - 'null' example: false settings: $ref: '#/components/schemas/ClickPipePatchPostgresPipeSettings' tableMappingsToAdd: type: array description: >- Table mappings to add to the pipe. Can be an empty array if no tables are being added. items: $ref: '#/components/schemas/ClickPipePostgresPipeTableMapping' minItems: 0 tableMappingsToRemove: type: array description: >- Table mappings to remove from the pipe. Only sourceSchemaName, sourceTable, and targetTable are required for removal. items: $ref: '#/components/schemas/ClickPipePatchPostgresPipeRemoveTableMapping' minItems: 0 ClickPipePatchMySQLSource: properties: credentials: $ref: '#/components/schemas/PLAIN' host: description: >- MySQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: - string - 'null' format: hostname example: my-mysql-server.example.com port: description: MySQL server port. type: - integer - 'null' minimum: 1 maximum: 65535 example: 3306 authentication: description: Authentication method for MySQL connection. type: - string - 'null' enum: - basic - IAM_ROLE example: basic iamRole: description: >- IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: - string - 'null' example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: - string - 'null' example: my-mysql-server.example.com caCertificate: description: PEM encoded CA certificate to validate the MySQL server certificate. type: - string - 'null' example: |- -----BEGIN CERTIFICATE----- ... disableTls: description: >- Disable TLS for the MySQL connection. Use with caution in production environments. type: - boolean - 'null' example: false skipCertVerification: description: >- Skip TLS certificate verification for the MySQL connection. Use with caution in production environments. type: - boolean - 'null' example: false serverId: description: >- Optional MySQL server_id the pipe declares itself as in the MySQL replication topology. Must be unique across replicas connected to the source. If omitted, one is assigned automatically. type: - integer - 'null' minimum: 1 maximum: 4294967295 example: 4242 settings: $ref: '#/components/schemas/ClickPipePatchMySQLPipeSettings' tableMappingsToAdd: type: array description: >- Table mappings to add to the pipe. Can be an empty array if no tables are being added. items: $ref: '#/components/schemas/ClickPipeMySQLPipeTableMapping' minItems: 0 tableMappingsToRemove: type: array description: >- Table mappings to remove from the pipe. Only sourceSchemaName, sourceTable, and targetTable are required for removal. items: $ref: '#/components/schemas/ClickPipePatchMySQLPipeRemoveTableMapping' minItems: 0 required: - host - port ClickPipePatchMongoDBSource: properties: credentials: $ref: '#/components/schemas/PLAIN' uri: description: >- MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...). type: - string - 'null' example: mongodb+srv://cluster0.example.mongodb.net/mydb readPreference: description: MongoDB read preference for replica set reads. type: - string - 'null' enum: - primary - primaryPreferred - secondary - secondaryPreferred - nearest example: secondaryPreferred tlsHost: description: TLS/SSL host for secure connections. type: - string - 'null' example: cluster0.example.mongodb.net disableTls: description: >- Disable TLS for the MongoDB connection. Defaults to false (TLS enabled). type: - boolean - 'null' example: false skipCertVerification: description: >- Skip TLS certificate verification for the MongoDB connection. Use with caution in production environments. type: - boolean - 'null' example: false caCertificate: description: >- PEM encoded CA certificate to validate the MongoDB server certificate. type: - string - 'null' example: |- -----BEGIN CERTIFICATE----- ... settings: $ref: '#/components/schemas/ClickPipePatchMongoDBPipeSettings' tableMappingsToAdd: type: array description: >- Collection mappings to add to the pipe. Can be an empty array if no collections are being added. items: $ref: '#/components/schemas/ClickPipeMongoDBPipeTableMapping' minItems: 0 tableMappingsToRemove: type: array description: >- Collection mappings to remove from the pipe. Only sourceDatabaseName, sourceCollection, and targetTable are required for removal. items: $ref: '#/components/schemas/ClickPipePatchMongoDBPipeRemoveTableMapping' minItems: 0 required: - uri - readPreference ClickPipeDestinationColumn: properties: name: description: Name of the column. type: string type: description: Type of the column. type: string ClickPipeKafkaSource: properties: type: description: Type of the Kafka source. type: string enum: - kafka - redpanda - msk - gcmk - confluent - warpstream - azureeventhub - dokafka format: description: Format of the Kafka source. type: string enum: - JSONEachRow - Avro - AvroConfluent - Protobuf brokers: description: Brokers of the Kafka source. type: string topics: description: Topics of the Kafka source. type: string consumerGroup: description: >- Consumer group of the Kafka source. If not provided "clickpipes-<>" will be used. type: - string - 'null' example: my-clickpipe-consumer-group authentication: description: >- Authentication method of the Kafka source. Supported authentication methods: kafka: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, msk: SCRAM-SHA-512, IAM_ROLE, IAM_USER, MUTUAL_TLS, gcmk: PLAIN, MUTUAL_TLS, confluent: PLAIN, MUTUAL_TLS, warpstream: PLAIN, azureeventhub: PLAIN, redpanda: SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, dokafka: SCRAM-SHA-256, MUTUAL_TLS type: string enum: - PLAIN - SCRAM-SHA-256 - SCRAM-SHA-512 - IAM_ROLE - IAM_USER - MUTUAL_TLS iamRole: description: >- IAM role for the Kafka source. Use with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/kafka#iam type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole offset: oneOf: - $ref: '#/components/schemas/ClickPipeKafkaOffset' - type: 'null' schemaRegistry: oneOf: - $ref: '#/components/schemas/ClickPipeKafkaSchemaRegistry' - type: 'null' caCertificate: description: PEM encoded CA certificates to validate the broker's certificate. type: - string - 'null' reversePrivateEndpointIds: type: array description: >- Reverse private endpoint UUIDs used for a secure private connection to the Kafka source. items: type: string exactlyOnce: description: >- Enable exactly-once delivery. Guarantees every Kafka record is inserted exactly once across restarts and rebalances. Can only be set at pipe creation. type: - boolean - 'null' ClickPipeObjectStorageSource: properties: type: description: Type of the ObjectStorage source. type: string enum: - s3 - gcs - dospaces - azureblobstorage - cloudflarer2 - ovhobjectstorage format: description: Format of the files. type: string enum: - JSONEachRow - JSONAsObject - CSV - CSVWithNames - TabSeparated - TabSeparatedWithNames - Parquet - Avro url: description: >- Provide a path to the file(s) you want to ingest. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations type: string example: >- https://datasets-documentation.s3.eu-west-3.amazonaws.com/http/**.ndjson.gz delimiter: description: Delimiter used in the files. type: - string - 'null' example: ',' compression: description: Compression algorithm used for the files. type: - string - 'null' enum: - none - gzip - gz - brotli - br - xz - LZMA - zstd - auto example: auto isContinuous: description: >- If set to true, the pipe will continuously read new files from the source. If set to false, the pipe will read the files only once. New files have to be uploaded lexically order. type: - boolean - 'null' queueUrl: description: >- Queue URL for event-based continuous ingestion. For S3, provide an SQS queue URL. For GCS, provide a Pub/Sub subscription (e.g. projects/{project}/subscriptions/{name}). When provided, files are ingested based on event notifications rather than lexicographical order. Only applicable when isContinuous is true and authentication is not public. type: - string - 'null' example: https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue skipInitialLoad: description: >- If set to true, skips the initial load and only ingests files delivered by queue notifications. Only applicable when queueUrl is provided. type: - boolean - 'null' startAfter: description: >- Start continuous ingestion after this object key. Cannot be provided when skipInitialLoad is true. type: - string - 'null' example: events/2026-06-01/ authentication: description: >- Authentication method. IAM_USER is for S3, GCS, and DigitalOcean Spaces. IAM_ROLE is for S3 only. SERVICE_ACCOUNT is for GCS only. CONNECTION_STRING is for Azure Blob Storage. PUBLIC uses no authentication. type: - string - 'null' enum: - IAM_ROLE - IAM_USER - CONNECTION_STRING - SERVICE_ACCOUNT iamRole: description: >- IAM role to be used with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#authentication type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole connectionString: description: >- Connection string for Azure Blob Storage authentication. Required when authentication is CONNECTION_STRING. type: - string - 'null' example: >- DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;EndpointSuffix=core.windows.net path: description: >- Path to the file(s) within the Azure container. Used for Azure Blob Storage sources. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations type: - string - 'null' example: data/logs/*.json azureContainerName: description: >- Container name for Azure Blob Storage. Required when type is azureblobstorage. type: - string - 'null' example: mycontainer ClickPipeKinesisSource: properties: format: description: Format of the Kinesis stream. type: string enum: - JSONEachRow - Avro - AvroConfluent streamName: description: Name of the Kinesis stream. type: string example: my-stream region: description: AWS region of the Kinesis stream. type: string example: us-east-1 useEnhancedFanOut: description: Use enhanced fan-out for the Kinesis stream. type: - boolean - 'null' iteratorType: description: >- Type of iterator to use when reading from the Kinesis stream. If AT_TIMESTAMP is used, the timestamp field must be provided. type: string enum: - TRIM_HORIZON - LATEST - AT_TIMESTAMP timestamp: description: >- UNIX timestamp to start reading from the Kinesis stream. Required if iteratorType is AT_TIMESTAMP. type: - integer - 'null' example: 1615766400 authentication: description: Authentication method to use with the Kinesis stream. type: string enum: - IAM_ROLE - IAM_USER iamRole: description: IAM role to use for authentication. Required if IAM_ROLE is used. type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole ClickPipePubSubSource: properties: format: description: >- Format of messages in the Pub/Sub topic. GCP Pub/Sub ClickPipes are in limited preview — contact support to enable this feature for your organization. type: string enum: - JSONEachRow - Avro - Protobuf example: JSONEachRow projectId: description: GCP project ID that owns the Pub/Sub topic. type: string example: my-gcp-project topic: description: Pub/Sub topic name (not the fully-qualified path). type: string example: my-topic authentication: description: >- Authentication method to use with GCP Pub/Sub. Currently only SERVICE_ACCOUNT is supported. type: string enum: - SERVICE_ACCOUNT example: SERVICE_ACCOUNT seekType: description: >- Starting position strategy for consuming the subscription. The seekTimestamp companion is required only when seekType is "timestamp"; setting it for a mismatched seek type is rejected. type: string enum: - latest - earliest - timestamp example: earliest seekTimestamp: description: >- RFC 3339 / ISO 8601 timestamp to seek to. Required when seekType is "timestamp"; must be omitted otherwise. type: - string - 'null' format: date-time example: '2026-04-10T12:00:00Z' filter: description: >- Optional Pub/Sub subscription filter expression (CEL). Maximum 256 characters. type: - string - 'null' maxLength: 256 enableOrdering: description: >- Whether to enable ordered delivery of messages (requires messages to be published with ordering keys). type: - boolean - 'null' ackDeadline: description: >- Acknowledgement deadline for messages, in seconds. Must be between 10 and 600. type: - integer - 'null' minimum: 10 maximum: 600 required: - format - projectId - topic - authentication - seekType ClickPipePostgresSource: properties: type: description: >- Type of the Postgres source. Defaults to "postgres" if not specified. type: - string - 'null' enum: - postgres - supabase - neon - alloydb - planetscale - rdspostgres - aurorapostgres - cloudsqlpostgres - azurepostgres - crunchybridge - tigerdata host: description: >- PostgreSQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: string format: hostname example: my-postgres-server.example.com port: description: PostgreSQL server port. type: integer minimum: 1 maximum: 65535 example: 5432 database: description: PostgreSQL database name to replicate from. type: string example: production_db authentication: description: Authentication method for Postgres connection. type: string enum: - basic - IAM_ROLE example: IAM_ROLE iamRole: description: >- IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: string example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: string example: my-postgres-server.example.com caCertificate: description: >- PEM encoded CA certificate to validate the Postgres server certificate. type: string example: |- -----BEGIN CERTIFICATE----- ... disableTls: description: >- Disable TLS for the Postgres connection. Use with caution in production environments. type: boolean example: false skipCertVerification: description: >- Skip TLS certificate verification for the Postgres connection. Use with caution in production environments. type: boolean example: false settings: $ref: '#/components/schemas/ClickPipePostgresPipeSettings' tableMappings: type: array description: >- List of table mappings defining which PostgreSQL tables to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipePostgresPipeTableMapping' ClickPipeMySQLSource: properties: type: description: Type of the MySQL source. Defaults to "mysql" if not specified. type: - string - 'null' enum: - mysql - rdsmysql - auroramysql - mariadb - rdsmariadb host: description: >- MySQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: string format: hostname example: my-mysql-server.example.com port: description: MySQL server port. type: integer minimum: 1 maximum: 65535 example: 3306 authentication: description: Authentication method for MySQL connection. type: string enum: - basic - IAM_ROLE example: basic iamRole: description: >- IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: string example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: string example: my-mysql-server.example.com caCertificate: description: PEM encoded CA certificate to validate the MySQL server certificate. type: string example: |- -----BEGIN CERTIFICATE----- ... disableTls: description: >- Disable TLS for the MySQL connection. Use with caution in production environments. type: boolean example: false skipCertVerification: description: >- Skip TLS certificate verification for the MySQL connection. Use with caution in production environments. type: boolean example: false serverId: description: >- Optional MySQL server_id the pipe declares itself as in the MySQL replication topology. Must be unique across replicas connected to the source. If omitted, one is assigned automatically. type: integer minimum: 1 maximum: 4294967295 example: 4242 settings: $ref: '#/components/schemas/ClickPipeMySQLPipeSettings' tableMappings: type: array description: >- List of table mappings defining which MySQL tables to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipeMySQLPipeTableMapping' required: - host - port - settings - tableMappings ClickPipeBigQuerySource: properties: snapshotStagingPath: description: >- GCS bucket path for staging snapshot data (e.g., gs://my-bucket/staging/). Data will be automatically cleaned up after initial load. type: string settings: $ref: '#/components/schemas/ClickPipeBigQueryPipeSettings' tableMappings: type: array description: Table mappings for BigQuery pipe. items: $ref: '#/components/schemas/ClickPipeBigQueryPipeTableMapping' ClickPipeMongoDBSource: properties: uri: description: >- MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...). type: string example: mongodb+srv://cluster0.example.mongodb.net/mydb readPreference: description: MongoDB read preference for replica set reads. type: string enum: - primary - primaryPreferred - secondary - secondaryPreferred - nearest example: secondaryPreferred tlsHost: description: TLS/SSL host for secure connections. type: string example: cluster0.example.mongodb.net disableTls: description: >- Disable TLS for the MongoDB connection. Defaults to false (TLS enabled). type: boolean example: false skipCertVerification: description: >- Skip TLS certificate verification for the MongoDB connection. Use with caution in production environments. type: boolean example: false caCertificate: description: >- PEM encoded CA certificate to validate the MongoDB server certificate. type: string example: |- -----BEGIN CERTIFICATE----- ... settings: $ref: '#/components/schemas/ClickPipeMongoDBPipeSettings' tableMappings: type: array description: >- List of collection mappings defining which MongoDB collections to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipeMongoDBPipeTableMapping' required: - uri - readPreference ClickPipeDestinationTableDefinition: properties: engine: $ref: '#/components/schemas/ClickPipeDestinationTableEngine' sortingKey: type: array description: Sorting key of the destination table. List of columns. items: type: string partitionBy: description: Partition key SQL expression. type: string primaryKey: description: Primary key of SQL expression. type: string PLAIN: properties: username: description: Database username. type: string example: postgres_user password: description: Database password. type: string format: password example: your_secure_password MskIamUser: properties: accessKeyId: description: IAM access key ID. type: string secretKey: description: IAM secret key. type: string AzureEventHub: properties: connectionString: description: Connection string for Azure EventHub source. type: string MutualTLS: properties: certificate: description: PEM encoded client certificate for mTLS authentication. type: string privateKey: description: PEM encoded client private key for mTLS authentication. type: string format: password ServiceAccount: properties: serviceAccountFile: description: Google Cloud service account JSON key file content, base64 encoded. type: string ClickPipePatchPostgresPipeSettings: properties: syncIntervalSeconds: description: >- Interval in seconds to sync data from Postgres during CDC replication. type: - integer - 'null' minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: - integer - 'null' minimum: 1 example: 1000 ClickPipePostgresPipeTableMapping: properties: sourceSchemaName: description: PostgreSQL source schema name. type: string example: public sourceTable: description: PostgreSQL source table name. type: string example: users targetTable: description: >- ClickHouse target table name, optionally prefixed with schema name (e.g., "my_schema_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: string example: public_users excludedColumns: type: array description: >- List of column names to exclude from replication. Column names must be unique within this list. items: type: string example: - internal_id - temp_data uniqueItems: true useCustomSortingKey: description: >- Whether to use a custom sorting key. If true, sortingKeys must be provided. If false or omitted, the default sorting key is the PostgreSQL primary key. type: boolean example: false sortingKeys: type: array description: >- Ordered list of column names to use as the sorting (ORDER BY) key in ClickHouse. Only used when useCustomSortingKey is true. Column names must be unique within this list. items: type: string example: - created_at_date - event_id uniqueItems: true tableEngine: description: >- ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it). type: string enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: >- Custom partitioning column used for parallel snapshotting. Only beneficial for PostgreSQL 13 (no benefit for PG14+, which supports indexed ctid scans). Must be an indexed column of type: `smallint`, `integer`, `bigint`, `timestamp without time zone`, or `timestamp with time zone`. Unrelated to ClickHouse partitioning. type: string example: id ClickPipePatchPostgresPipeRemoveTableMapping: properties: sourceSchemaName: description: PostgreSQL source schema name. type: - string - 'null' example: public sourceTable: description: PostgreSQL source table name. type: - string - 'null' example: users targetTable: description: >- ClickHouse target table name, optionally prefixed with schema name (e.g., "my_schema_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: - string - 'null' example: public_users tableEngine: description: >- ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it). type: - string - 'null' enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: >- Custom partitioning column used for parallel snapshotting. Only beneficial for PostgreSQL 13 (no benefit for PG14+, which supports indexed ctid scans). Must be an indexed column of type: `smallint`, `integer`, `bigint`, `timestamp without time zone`, or `timestamp with time zone`. Unrelated to ClickHouse partitioning. type: - string - 'null' example: id ClickPipePatchMySQLPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from MySQL during CDC replication. type: - integer - 'null' minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: - integer - 'null' minimum: 1 example: 1000 useCompression: description: Enable compression for the MySQL connection. type: - boolean - 'null' example: false ClickPipeMySQLPipeTableMapping: properties: sourceSchemaName: description: MySQL source database name. type: string example: my_database sourceTable: description: MySQL source table name. type: string example: users targetTable: description: >- ClickHouse target table name, optionally prefixed with schema name (e.g., "my_database_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: string example: my_database_users excludedColumns: type: array description: >- List of column names to exclude from replication. Column names must be unique within this list. items: type: string example: - internal_id - temp_data uniqueItems: true useCustomSortingKey: description: >- Whether to use a custom sorting key. If true, sortingKeys must be provided. If false or omitted, the default sorting key is the MySQL primary key. type: boolean example: false sortingKeys: type: array description: >- Ordered list of column names to use as the sorting (ORDER BY) key in ClickHouse. Only used when useCustomSortingKey is true. Column names must be unique within this list. items: type: string example: - created_at_date - event_id uniqueItems: true tableEngine: description: >- ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it). type: string enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: >- Custom partitioning column used for parallel snapshotting. Must be an indexed column of an integer, date, datetime or timestamp type. Unrelated to ClickHouse partitioning. type: string example: id required: - sourceSchemaName - sourceTable - targetTable ClickPipePatchMySQLPipeRemoveTableMapping: properties: sourceSchemaName: description: MySQL source database name. type: - string - 'null' example: my_database sourceTable: description: MySQL source table name. type: - string - 'null' example: users targetTable: description: >- ClickHouse target table name, optionally prefixed with schema name (e.g., "my_database_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: - string - 'null' example: my_database_users tableEngine: description: >- ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it). type: - string - 'null' enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: >- Custom partitioning column used for parallel snapshotting. Must be an indexed column of an integer, date, datetime or timestamp type. Unrelated to ClickHouse partitioning. type: - string - 'null' example: id required: - sourceSchemaName - sourceTable - targetTable ClickPipePatchMongoDBPipeSettings: properties: syncIntervalSeconds: description: >- Interval in seconds to sync data from MongoDB during CDC replication. type: - integer - 'null' minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: - integer - 'null' minimum: 1 example: 100000 ClickPipeMongoDBPipeTableMapping: properties: sourceDatabaseName: description: MongoDB source database name. type: string example: mydb sourceCollection: description: MongoDB source collection name. type: string example: users targetTable: description: >- ClickHouse target table name. The table will be created automatically if it does not exist. type: string example: mydb_users tableEngine: description: >- ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it). type: string enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree required: - sourceDatabaseName - sourceCollection - targetTable ClickPipePatchMongoDBPipeRemoveTableMapping: properties: sourceDatabaseName: description: MongoDB source database name. type: - string - 'null' example: mydb sourceCollection: description: MongoDB source collection name. type: - string - 'null' example: users targetTable: description: >- ClickHouse target table name. The table will be created automatically if it does not exist. type: - string - 'null' example: mydb_users tableEngine: description: >- ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it). type: - string - 'null' enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree required: - sourceDatabaseName - sourceCollection - targetTable ClickPipeKafkaOffset: properties: strategy: description: Offset strategy. type: string enum: - from_beginning - from_latest - from_timestamp timestamp: description: >- A minute precision UTC timestamp to start from. Required for "from_timestamp" strategy. type: - string - 'null' example: 2021-01-01T00:00 ClickPipeKafkaSchemaRegistry: properties: url: description: Schema URL. HTTPS required. type: string example: https://psrc-aa00.us-east-2.aws.confluent.cloud/schemas/ids/100004 authentication: description: Authentication type of the schema registry. type: string enum: - PLAIN caCertificate: description: >- PEM encoded CA certificates to validate the schema registry's certificate. type: - string - 'null' ClickPipePostgresPipeSettings: properties: syncIntervalSeconds: description: >- Interval in seconds to sync data from Postgres during CDC replication. type: integer minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: integer minimum: 1 example: 1000 publicationName: description: >- PostgreSQL publication name to use for CDC replication. If not provided, ClickPipes will create one automatically. type: string example: clickpipes_publication replicationMode: description: >- Replication mode: "cdc" (change data capture with initial snapshot), "snapshot" (one-time snapshot only), or "cdc_only" (CDC without initial snapshot). type: string enum: - cdc - snapshot - cdc_only example: cdc replicationSlotName: description: >- PostgreSQL replication slot name. Only valid for "cdc_only" mode. For "cdc" mode, ClickPipes creates the slot automatically. type: string example: clickpipes_slot allowNullableColumns: description: >- Preserve nullability from Postgres in the destination ClickHouse table. When true, columns without NOT NULL constraints are created as Nullable(...). When false, all columns are non-nullable and NULL values are replaced with the default value for the type. Note: Nullable types have performance overhead in ClickHouse. type: boolean example: false initialLoadParallelism: description: >- Number of parallel workers to use per table in the initial snapshot phase. type: integer minimum: 1 example: 1 snapshotNumRowsPerPartition: description: Number of rows per partition during the snapshot phase. type: integer minimum: 1000 example: 100000 snapshotNumberOfParallelTables: description: >- Number of tables to snapshot in parallel during the initial load phase. type: integer minimum: 1 example: 1 enableFailoverSlots: description: >- Enable failover support for the replication slot on PG17 and newer. Only applicable when ClickPipes creates the replication slot (i.e., replicationSlotName is NOT provided). type: boolean example: false deleteOnMerge: description: >- Enable hard delete behavior in ReplacingMergeTree for PostgreSQL DELETE operations. type: boolean example: false ClickPipeMySQLPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from MySQL during CDC replication. type: integer minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: integer minimum: 1 example: 1000 replicationMode: description: >- Replication mode: "cdc" (change data capture with initial snapshot), "snapshot" (one-time snapshot only), or "cdc_only" (CDC without initial snapshot). type: string enum: - cdc - snapshot - cdc_only example: cdc replicationMechanism: description: >- MySQL replication mechanism: "GTID" (Global Transaction Identifier) or "FILE_POS" (binary log file and position). Defaults to "GTID" if not specified. MariaDB supports "GTID" only. For "FILE_POS" on MySQL, contact support. type: string enum: - GTID - FILE_POS example: GTID useCompression: description: Enable compression for the MySQL connection. type: boolean example: false allowNullableColumns: description: >- Preserve nullability from MySQL in the destination ClickHouse table. When true, columns without NOT NULL constraints are created as Nullable(...). When false, all columns are non-nullable and NULL values are replaced with the default value for the type. Note: Nullable types have performance overhead in ClickHouse. type: boolean example: false initialLoadParallelism: description: >- Number of parallel workers to use per table in the initial snapshot phase. type: integer minimum: 1 example: 1 snapshotNumRowsPerPartition: description: Number of rows per partition during the snapshot phase. type: integer minimum: 1000 example: 100000 snapshotNumberOfParallelTables: description: >- Number of tables to snapshot in parallel during the initial load phase. type: integer minimum: 1 example: 1 deleteOnMerge: description: >- Enable hard delete behavior in ReplacingMergeTree for MySQL DELETE operations. type: boolean example: false required: - replicationMode ClickPipeBigQueryPipeSettings: properties: replicationMode: description: Replication mode. BigQuery only supports snapshot mode. type: string enum: - snapshot allowNullableColumns: description: Allow nullable columns in the destination table. type: boolean initialLoadParallelism: description: Number of parallel workers during initial load. type: number snapshotNumRowsPerPartition: description: Number of rows to snapshot per partition. type: number snapshotNumberOfParallelTables: description: Number of parallel tables to snapshot. type: number ClickPipeBigQueryPipeTableMapping: properties: sourceDatasetName: description: Source BigQuery dataset name. type: string sourceTable: description: Source table name. type: string targetTable: description: Target ClickHouse table name. type: string excludedColumns: type: array description: Columns to exclude from the target table. items: type: string useCustomSortingKey: description: Whether to use a custom sorting key for the target table. type: boolean sortingKeys: type: array description: Ordered list of columns to use as sorting key for the target table. items: type: string tableEngine: description: Table engine to use for the target table. type: string enum: - MergeTree - ReplacingMergeTree - 'Null' ClickPipeMongoDBPipeSettings: properties: syncIntervalSeconds: description: >- Interval in seconds to sync data from MongoDB during CDC replication. type: integer minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: integer minimum: 1 example: 100000 replicationMode: description: >- Replication mode: "cdc" (change data capture with initial snapshot), "snapshot" (one-time snapshot only), or "cdc_only" (CDC without initial snapshot). type: string enum: - cdc - snapshot - cdc_only example: cdc snapshotNumRowsPerPartition: description: Number of rows per partition during the snapshot phase. type: integer minimum: 1000 example: 100000 snapshotNumberOfParallelTables: description: >- Number of collections to snapshot in parallel during the initial load phase. type: integer minimum: 1 example: 1 deleteOnMerge: description: >- Enable hard delete behavior in ReplacingMergeTree for MongoDB DELETE operations. type: boolean example: false useJsonNativeFormat: description: >- Store JSON values in native ClickHouse JSON format. When disabled, JSON data is stored as String. type: boolean example: true required: - replicationMode ClickPipeDestinationTableEngine: properties: type: description: Engine type of the destination table. type: string enum: - MergeTree - ReplacingMergeTree - SummingMergeTree - 'Null' versionColumnId: description: Column name to use as version for ReplacingMergeTree engine. type: - string - 'null' columnIds: type: array description: Column names to sum for SummingMergeTree engine. items: type: string securitySchemes: basicAuth: type: http scheme: basic description: >- Use key ID and key secret obtained in ClickHouse Cloud console: https://clickhouse.com/docs/cloud/manage/openapi ````