ClickHouse Overview

ClickHouse is a fast, open-source columnar database designed for analytical processing. It uses compression and encoding techniques to store data efficiently and speed up query execution. Here's what sets it apart:

Scalability

ClickHouse uses vectorized execution and parallel processing to use all CPU cores on a single machine.

Vectorized Execution involves using modern CPU vector instructions to process multiple data points simultaneously via SIMD.

For example, instead of adding numbers from two arrays one pair at a time as in a traditional loop, vectorized execution allows a CPU to add multiple pairs of numbers concurrently, depending on the vector instructions it supports.

Parallel Processing denotes the simultaneous execution of computations, either on different processors or cores within a CPU or across separate machines. It's common to combine both vectorized execution and parallel processing.

For instance, in a multi-threaded application, each thread might conduct vectorized operations on a data segment, taking advantage of both the data-level parallelism from vectorization and the task-level parallelism from multi-threading or multi-processing.

Speed

Inserts are instant, and ClickHouse handles billions of rows with sub-second query response times.

Compression and Encoding Techniques

ClickHouse gives you two ways to shrink and reshape data, and the difference between them is small. Encoding converts data into a form that stores efficiently. Compression cuts the size of the data, which saves space and speeds up transmission. Some popular techniques include:

• LowCardinality: A special label in ClickHouse, ideal for columns with few unique values. Using LowCardinality can save space and speed up searches. Instead of constantly repeating values, this method assigns each unique value a number. When that value is needed, ClickHouse just refers back to the number's corresponding value.
• Delta: A method where, instead of noting every number, the change from the previous one is highlighted. This approach is a space-saver, especially for datasets where the values don't change dramatically. Delta encoding is particularly well-suited for series of numbers that exhibit slow or consistent growth.
• DoubleDelta: Rather than marking the change from one number to the next, the difference between those changes is recorded. It's especially handy when there's a consistent rate of growth, making it a top choice for datasets like stock prices which often grow at a steady pace.
• Gorilla: A data storage technique developed by Facebook, is particularly adept at managing data where numbers show minimal change. When two numbers are alike, Gorilla identifies the difference and further refines it. It's highly efficient for datasets where numbers remain relatively consistent, such as temperature readings taken every minute.
• T64: Represents a unique way in ClickHouse to handle groups of numbers. Utilizing a speedy method known as TurboPFor, it compresses numbers to a more compact size. This is optimal for columns that contain extensive sequences of numbers.

In many situations, ClickHouse first squeezes the data to make it smaller and then changes its language, so it works best for its purpose. For example, think about watching a video online; the video is first made smaller and then put in a format that's best for watching on the internet.

Replication

ClickHouse uses a multi-master replication system, which keeps data consistent across nodes. Multiple master databases synchronize with each other, and the setup is fault tolerant.

So if one or more nodes hit problems or fail, the system keeps running. Disruption stays small and the data stays intact.

Integration

ClickHouse integrates with many platforms, including Kafka, JDBC, HDFS, RDBMS, and Object Storage/S3.
For a full overview and further details, please visit the official documentation.

Table Capabilities in ClickHouse

• Enum: A special data type that matches strings to numbers, saving space when columns have only a few possible string values.
• Partitioning: Tables are divided into parts based on set rules (like time). This makes it easier to manage data and helps queries run faster.
• Arrays: This column type holds multiple related values, such as tags, all in one spot. It also comes with special functions tailored for arrays.
• TTL Tables (Time to Live): This sets how long data will stay in tables. Once this time is up, the data is either deleted or moved, all on its own.
• Nested Columns: Think of this like columns within columns. It's useful for storing structured data, similar to how JSON objects work. Plus, you can search within it using dot notation.
• Materialized Views: These are saved query results. Instead of calculating every time, it uses these saved results for quick answers. They're especially helpful for data that's been added up or changed in advance to make things run faster.

See the official documentation for details.

Query Language in ClickHouse

ClickHouse employs its unique query language, built upon the foundations of SQL. This design choice ensures that individuals already acquainted with SQL find it relatively straightforward to operate within ClickHouse.
For a deeper understanding and specifics, kindly refer to the official documentation.

Table Engine in ClickHouse

A table engine in ClickHouse decides how data is stored, read, and written on the disk. It also decides how operations like indexing and replication run on that data.

The engine sits under the table's structure and behavior, so it shapes the storage format, the query performance, and the features you can use. Different engines are tuned for different use cases. So the engine you pick affects both what your data operations can do and how fast they run. See the official documentation for details.

When to Choose ClickHouse