Access DoubleCloud resources with resources in your AWS VPC

This tutorial shows how to peer your DoubleCloud and AWS networks. Most sections describe AWS-specific actions such as the configuration of security groups, route tables, and the EC2 instance. For demonstration purposes, we will use a default VPC. The DoubleCloud network will contain a Managed ClickHouse® cluster and the AWS one will have an EC2 instance. At the final stage, we'll send a query from the EC2 virtual machine to the Managed ClickHouse® cluster to test the connection.

scenario-diagram

  1. Create a Managed ClickHouse® cluster in a DoubleCloud network.

  2. Inspect the AWS console.

  3. Bind your AWS and DoubleCloud networks.

    1. Confirm peering on the AWS side.
  4. Configure a route table.

  5. Add the AWS VPC CIDR to the Managed ClickHouse® allow list.

  6. Use EC2 to connect the Managed ClickHouse® cluster in DoubleCloud.

    1. Run an Ubuntu instance in EC2.

    2. Connect to the instance.

    3. Access the Managed ClickHouse® cluster.

Create a Managed ClickHouse® cluster and a DoubleCloud network

A network in DoubleCloud appears automatically when you create a resource. It can be one of the cluster type - ClickHouse® or Apache Kafka®. In this tutorial, we will create a Managed ClickHouse® cluster but won't perform any actions with it - we need it only for the network in a specific region.

  1. Go to the Clusters overview page in console.

  2. Click Create cluster in the upper-right corner of the page.

  3. Select ClickHouse®.

  4. Choose AWS as your provider.

  5. Select your cluster's region. This is an important point - you can only peer networks that belong to the same region. In this tutorial, we create a cluster in N. Virginia: us-east-1 region.

  6. Under Resources:

    • Select the s2-c2-m4 preset for CPU, RAM capacity, and storage space to create a cluster with minimal configuration.

    • Choose the number of replicas. Let's keep it as is with a single replica.

    • Select the number of shards. Keep a single shard.

  7. Under Basic settings:

    • Enter the cluster Name, in this tutorial - first-cluster.

    • From the Version drop-down list, select the ClickHouse® version the Managed ClickHouse® cluster will use. For most clusters, we recommend using the latest version.

  8. Under NetworkingVPC, select a VPC in DoubleCloud in which to locate your cluster. Make sure that you have the default value - default in us-east-1.

  9. Click Submit.

Your cluster will appear with the Creating status on the Clusters page in the console. Setting everything up may take some time. When the cluster is ready, it changes its state to Alive.

cluster-created

Now you have a cluster in the us-east-1. Let's proceed to create an AWS VPC in the same region.

Inspect the AWS console

  1. Open the AWS console .

  2. Select the US East (N. Virginia) us-east-1 in the header:

    select a region

  3. Go to the AWS VPC service page .

  4. From the list of options on the left, click Your VPCs under Virtual private cloud. This section lists the default VPC that we will use.

A default VPC has already attached and basically configured subnets, a security group, an internet gateway, and a route table. In this scenario, we will need only to configure the route table, and we leave other resources with their default configuration options.

Bind your AWS and DoubleCloud networks

  1. Go to the Connections tab on the VPC page in the console.

  2. Specify the AWS account ID you want to connect to. You can find this numeric ID in the AWS console in the drop-down menu with your account information on the top panel on the right:

    aws-account-id

  3. Specify the AWS VPC ID. In the AWS console, find your default VPC and copy its ID. It looks as follows: vpc-xxxxxx.

  4. Provide your AWS IPv4 CIDR. You can find it on the right from the VPC ID in the AWS console.

  5. Select your VPC Region - US East (N. Virginia) us-east-1.

  6. From the drop-down menu, select your DoubleCloud Network to peer with.

    connection-prepared

  7. Click Submit.

After you have created a connection, enable it on the AWS side. Note that it will take some time before the request appears on the AWS side.

Confirm peering on the AWS side

  1. Go to the AWS VPC service page .

  2. Go to the Virtual private cloudPeering connections section in the menu on the left:

    select-peering-connections

  3. Click your VPC Peering connection ID and choose ActionsAccept Request.

  4. In the confirmation dialog, choose Accept request.

At this point, you have successfully peered your AWS and DoubleCloud networks. Now it's time to add a route to the peered VPC.

Configure a route table

Route tables contain the rules called routes that define where to direct your network traffic.

We will use the default route table and add a route to the DoubleCloud VPC.

  1. Go to the AWS VPC service page .

  2. Open the Virtual private cloudRoute Tables section from the menu on the left:

    route-tables-section

  3. Select a route table associated with the subnet you want to access clusters from.

  4. In the Routes tab, click Edit routes.

  5. Click Add route and specify its properties:

    • Destination to your DoubleCloud connection as a Managed IPv4 address in the CIDR notation:

      connection-created

    • Target is the ID of a peering connection in DoubleCloud. Click on this field, select Peering connection and then select your connection ID.

  6. Click Save changes.

Add the AWS VPC CIDR to the Managed ClickHouse® allow list

  1. Go to the Clusters overview page in console.

  2. Select the first-cluster cluster.

  3. Click the Allow List tab.

  4. Enter the CIDR of your AWS network from the Connections page in the DoubleCloud console:

    add-cidr

  5. Click Add.

Now you have fully connected your AWS and DoubleCloud networks. The traffic flow is also configured and you can proceed to creating a virtual machine in the AWS network to reach a resource in your DoubleCloud network.

Use EC2 to connect the Managed ClickHouse® cluster in DoubleCloud

Run an Ubuntu instance in EC2

For demonstration purposes, we will create a free-tier virtual machine with Ubuntu. It will be a server that we'll later use to reach a Managed ClickHouse® cluster in DoubleCloud.

  1. Go to the AWS EC2 service page . Make sure that you are still in the same region - N. Virginia: us-east-1

  2. Click Launch instanceLaunch instance:

    launch-instance

  3. Give a name to your server, for example first-server.

  4. Select Ubuntu® with any available free tier.

  5. Keep the Instance type default.

  6. Use your Key pair or generate a new one. If you create a new key pair, follow the steps below:

    1. Under Key pair name, specify the name of these keys - tutorial-key-pair.

    2. Under Key pair type, select RSA.

    3. Select .pem under Private key file format.

      This scenario implies that you have a CLI ssh client, You can create a .ppk key if you prefer such tools as PuTTY .

    4. Click Create key pair. The file with your keys will be downloaded automatically. You will use this file later to establish an SSH connection to the instance.

  7. Under Network settings, click Edit to specify the settings you have previously configured:

    1. VPC

      Select your VPC ID - <vpc-xxxxxx>.

    2. Firewall (security group)

      Switch to Select existing security group and select the default security group.

  8. You aren't going to need other configuration options for this tutorial, so you can click Launch instance.

    The result should be the following:

    instance-launched

As soon as your instance comes alive, you'll be able to connect to it and reach the Managed ClickHouse® cluster. Click on the instance ID and go to its page.

Connect to the instance

We have everything set up and running. The connection between the networks is established, the Managed ClickHouse® cluster and the EC2 instance are running. Now you should connect to the instance to send a query to the Managed ClickHouse® cluster. You can choose any kind of connection provided by AWS depending on your account roles, but here we show the SSH way.

  1. On the instance page, click Connect.

    instance-connect

  2. Select the SSH client tab.

  3. Follow the instructions from this tab to check if your key is publicly viewable and get ready to run your SSH client.

    Tip

    Use the chmod command to change the access mode to your SSH key if the key is too open.

  4. Navigate to the folder with your PEM key and run the following command with the information suggested on this page:

    ssh -i "tutorial-key-pair.pem" ubuntu@<ec2-machine-address>.amazonaws.com
    
  5. Agree to establish the connection in the CLI and wait until you see something like this:

    ubuntu@ip-172-31-20-0:~$
    

Access the Managed ClickHouse® cluster

Warning

When you connect to a cluster via a peering connection from VPC, you need to use a private address instead of the normally used public address.

To obtain a cluster's private connection string, go to the cluster overview page. Under Connection strings, switch to the Private tab:

connection strings tabs

  1. In your terminal with the SSH session, combine a query similar to the one below. You can find the ID of your cluster on the cluster's overview page. Make sure to add private to the resulting address:

    telnet rw.<cluster ID>.private.at.double.cloud 9440
    
  2. If everything was configured correctly, you should see the following output:

    Connected to <host>.<id>.private.at.double.cloud.
    Escape character is '^]'.
    

This step concludes this scenario. In real use-cases, you are likely to use a similar flow with other resources, accessing them with Transfer or other data tools. Keep exploring!

See also