GCP Private Service Connect for ClickPipes

You can use GCP Private Service Connect (PSC) to establish a secure connection between ClickPipes and a data source hosted on GCP. ClickPipes creates a reverse private endpoint (RPE) in its VPC and points it at a private endpoint service published for your data source, so traffic is never exposed to the public internet.

ClickPipes supports cross-region PSC connectivity via global access. Global access is required for endpoints located in a different region than the region where the ClickPipes infrastructure runs. Check the ClickPipes GCP region documentation for more details.

Supported patterns

Pattern	When it applies	Service attachment
Native PSC	Managed services that automatically publish a PSC service attachment (e.g. Google Cloud SQL, AlloyDB, Google Cloud Managed Service for Apache Kafka).	Automatically created when you enable PSC on the instance.
Producer-owned PSC	Data sources that don’t publish a service attachment, such as Postgres on Compute Engine or a managed service reachable only by private IP.	Created and managed by you.

Prerequisites

A ClickHouse Cloud service hosted on GCP. ClickPipes runs in the same region as your ClickHouse Cloud service (with a few exceptions).
IAM rights to enable PSC on your source and manage PSC service attachments (roles/compute.networkAdmin).
The ClickPipes consumer project allowed to connect to your service attachment (clickpipes-production).
If you’re provisioning ClickPipes programmatically: a ClickHouse Cloud API key for the organization that owns the service.

Create the reverse private endpoint

To create a RPE, you need the following information for your data source:

Service attachment URI: projects/<PROJECT>/regions/<REGION>/serviceAttachments/<NAME>, the PSC endpoint published in front of your data source.
Private DNS name: the hostname your pipe connects to. A PSC endpoint gets a static internal IP with no DNS name attached, so ClickPipes resolves this hostname to that IP for you.

ClickPipes UI
Terraform
API

In ClickHouse Cloud, open your service and go to Data Sources > ClickPipes.
Select the data source you want to ingest from.
Under Setup your ClickPipe connection, toggle on Use secure connection, then click + Reverse private endpoint and pick GCP PSC service attachment.
Fill in the Service attachment URI, the Private DNS name, and a Description.
Click Create. The endpoint moves through Provisioning → Ready. (Native PSC auto-accepts, so you will not see PendingAcceptance.)

Custom private DNS mappings are not an attribute of clickhouse_clickpipes_reverse_private_endpoint — they are managed by the separate clickhouse_clickpipes_reverse_private_endpoint_custom_private_dns resource, which takes a full replacement list of mappings and references the endpoint by ID:

resource "clickhouse_clickpipes_reverse_private_endpoint" "gcp_psc" {
  service_id             = var.clickhouse_service_id
  description            = "GCP PSC endpoint"
  type                   = "GCP_PSC_SERVICE_ATTACHMENT"
  gcp_service_attachment = "projects/<PROJECT>/regions/<REGION>/serviceAttachments/<NAME>"
}

resource "clickhouse_clickpipes_reverse_private_endpoint_custom_private_dns" "gcp_psc" {
  service_id                  = var.clickhouse_service_id
  reverse_private_endpoint_id = clickhouse_clickpipes_reverse_private_endpoint.gcp_psc.id

  mapping = [
    { private_dns_name = "<PRIVATE_DNS_NAME>" }
  ]
}

curl --silent --user $KEY_ID:$KEY_SECRET \
  -X POST -H "Content-Type: application/json" \
  https://api.clickhouse.cloud/v1/organizations/<ORG_ID>/services/<SERVICE_ID>/clickpipesReversePrivateEndpoints \
  -d '{
    "type": "GCP_PSC_SERVICE_ATTACHMENT",
    "description": "GCP PSC endpoint",
    "gcpServiceAttachment": "projects/<PROJECT>/regions/<REGION>/serviceAttachments/<NAME>",
    "customPrivateDnsMappings": [
      { "privateDnsName": "<PRIVATE_DNS_NAME>" }
    ]
  }'

Custom private DNS names

Because GCP PSC does not propagate DNS, a custom private DNS mapping is required for every GCP PSC endpoint so ClickPipes can resolve it to the endpoint’s static internal IP. The standard ClickPipes custom private DNS naming rules apply: wildcard support, uniqueness across a service’s reverse private endpoints, and reserved-suffix restrictions are enforced identically for every provider. Wildcards are especially useful, e.g. a single mapping such as *.<cluster>.<region>.managedkafka.<project>.cloud.goog fronts every broker of a Managed Kafka cluster. For GCP PSC, only the mapping’s privateDnsName is used. The internalDNSName field is not supported, because a PSC endpoint exposes a static IP with no DNS name to pin to — mappings always resolve to that IP.

Approving producer-owned connections

For producer-owned PSC, append the ClickPipes consumer project (clickpipes-production) to the auto-accept list of the service attachment, so the connection is accepted automatically. If you skip that, the RPE sits in PendingAcceptance until you approve it manually. To accept the connection, the consumer accept list of the service attachment needs the numeric ID-based URI of the endpoint (projects/<PROJECT>/regions/<REGION>/forwardingRules/<NUMERIC_ID>). To get the endpointWithId value for the pending connection on the service attachment, run:

gcloud beta compute service-attachments describe <SERVICE_ATTACHMENT_NAME> \
  --region=<REGION> --project=<YOUR_PROJECT_ID> \
  --format='json(connectedEndpoints)'

Take the endpointWithId of the entry whose status is PENDING, strip the https://www.googleapis.com/compute/beta/ prefix, and pass the result to --consumer-accept-list:

--consumer-accept-list overwrites the accept list rather than appending the new consumer project to it. If the service attachment already accepts other projects or endpoints, pass all of them in the same command, or the ones you leave out lose access.

gcloud beta compute service-attachments update <SERVICE_ATTACHMENT_NAME> \
  --region=<REGION> --project=<YOUR_PROJECT_ID> \
  --consumer-accept-list=projects/<PROJECT>/regions/<REGION>/forwardingRules/<NUMERIC_ID> \
  --reconcile-connections

Do not use the endpoint_id from the ClickPipes API/Terraform response here; that value is the forwarding-rule name, not the numeric ID GCP requires, so the accept list will not match. Always take the numeric ID from the pending endpointWithId of the service attachment .

Managing reverse private endpoints

You can manage existing reverse private endpoints in the ClickHouse Cloud service settings:

In the ClickHouse Cloud console sidebar, navigate to Settings.

Under ClickPipe reverse private endpoints , click on Reverse private endpoints. Reverse private endpoint extended information is shown in the flyout. From this view, it’s possible to delete RPEs: click the × on an endpoint and confirm to delete it. Deletion is **permanent** and will break any ClickPipe currently using that endpoint. To change an endpoint’s custom private DNS mappings, use the Cloud API or Terraform. The update replaces the full list of mappings, rather than adding to it. If an endpoint shows Rejected or Failed, read the message under its status: it distinguishes a connection the producer rejected from one that was closed or that needs attention. Both usually mean the accept list of the service attachment doesn’t include the ClickPipes consumer project. To see the status GCP reports for the endpoint, describe the connectedEndpoints for the service attachment.

Known limitations

A PSC service attachment can only be claimed by one ClickHouse Cloud service at a time; it cannot be reused across multiple services. To move a service attachment to a different service, Contact ClickHouse support to release the existing claim.

ClickPipes for Object Storage

ClickPipes for Postgres

ClickPipes for Kafka

ClickPipes for Kinesis

ClickPipes for GCP Pub/Sub

ClickPipes for MySQL

ClickPipes for MongoDB

ClickPipes for BigQuery

Programmatic access

Networking

Other

GCP Private Service Connect for ClickPipes

Supported patterns

Prerequisites

Create the reverse private endpoint

Custom private DNS names

Approving producer-owned connections

Managing reverse private endpoints

Known limitations

​Supported patterns

​Prerequisites

​Create the reverse private endpoint

​Custom private DNS names

​Approving producer-owned connections

​Managing reverse private endpoints

​Known limitations

Supported patterns

Prerequisites

Create the reverse private endpoint

Custom private DNS names

Approving producer-owned connections

Managing reverse private endpoints

Known limitations