When talking to potential customers, we sometimes get to the unfortunate point where they really like our services and offerings, but still have to decline due to a variety of reasons.
The data collected by said customer might be highly confidential and they’re not allowed to use any cloud providers at all for their production data.
The amount of data and the machines they’re already using might be way too expensive to handle within a cloud environment or there might be many other reasons they can’t expound on.
I’d been using ClickHouse® for over five years in an OnPrem scenario at a previous company and I’ve worked with a managed ClickHouse solution whilst at DoubleCloud for many customers.
If you’ve a good team and are managing your own servers (which I fortunately had, greetings to Francois and Atanas), then there’s absolutely nothing stopping you managing your own Clickhouse cluster OnPrem!
But still, the world isn’t black and white, it’s filled with different shades of gray and a whole host of other colors to choose between as well.
Why limit yourself to the mindset of just using OnPrem or just using CloudServices?
Hybrid systems are all over the place, in storage solutions as well as in the automotive industry.
Why not choose a hybrid solution for your ClickHouse services as well?
Development and testing
Even if you’ve a good team managing your ClickHouse clusters, and also know your way around upgrading stuff and creating new ones yourself, one of the most time consuming things in your day-to-day business will always be setting up new infrastructure.
Let’s say your developers want to test the newest version of ClickHouse; it still takes various steps to do so: you’ll have to create a ticket for your operations team, redefine the ticket to specify exactly what ops need to know, download the specified version of ClickHouse into your internal repository for infrastructure, set up the cluster itself, make entries to DNS so developers can reach the new cluster, copy the data from your other dev system into the new cluster…
And on and on it goes!
All those steps take up valuable time that could be better used for other things. And in the end it might not have even been worth it, as you decide the new version you’ve tested has a bug or is already too old or you want to wait for LTS or whatever.
So the solution is quite simple.
Just split your development and testing clusters from your production ones. All you need to have is an account on DoubleCloud and give your developers access to it.
They can then spin up a new dev cluster in a few minutes, choose the version they want to test, play around with it, and then shut it down with another two clicks after their test is finished.
You can also create a new cluster out of an existing backup with only a few clicks, so there’s no need to copy test data to a new cluster as well. Just create it from backups and upgrade the version to the one you want to test.
Your operations team will still be responsible for all production systems but they’ll have more time and energy to do so if all these small little things around development and test systems for developers can be handled by the developers themselves.
Separate clusters by logic
The previous example might have been an obvious one, but it’s already shown where you can split between OnPrem and Cloud quite easily.
But there are other potential layers, where a combination between OnPrem solutions and Cloud solutions can make sense.
Especially when using OnPrem solutions, monitoring is a must.
You have to make sure everything runs as it should and you’ll want to keep metrics and logs as long as possible to look for degradations, impacts, incidents etc.
ClickHouse is one of the best solutions out there for storing that kind of data. But no matter if you’ve no experience with ClickHouse at all or you are already using ClickHouse for other parts of your application, it makes sense to bring your monitoring to a managed ClickHouse in DoubleCloud.
Monitoring systems tend to grow over time, especially when you’re OnPrem and constantly adding new VMs, Containers and Systems etc.
But you’ll still want to use monitoring, without investing more time managing the monitoring itself.
Also, you’ve heard the term: “Who watches the watcher?”
Imagine you’re already using ClickHouse for production and you get some problems with your ClickHouse clusters.
Do you really want to have your monitoring within the same logic?
If your ClickHouse clusters break because of an incorrect configuration you’ve rolled out, your monitoring might break as well.
Using managed ClickHouse on DoubleCloud however will avoid this problem as you’ve separated your monitoring solution from your production logic, but can still use the same technology you’re used to.
It’s also easy to use DoubleCloud’s visualization to build your monitoring, without an extra cost.