Chproxy

Chproxy is an open-source HTTP proxy and load balancer specifically designed for the ClickHouse database. It is a community-driven open-source project that provides powerful capabilities for traffic management and security when working with ClickHouse clusters.

Chproxy represents a reliable and feature-rich solution for organizing secure and efficient access to ClickHouse clusters in production environments.

Key features and capabilities of Chproxy:

Routing and Load Balancing:

• Proxying requests to multiple distinct ClickHouse clusters depending on the input user
• Even distribution of requests among replicas and nodes using least loaded + round robin technique
• Node health monitoring and prevention of sending requests to unhealthy nodes

User Management and Security:

• Mapping input users to per-cluster users, preventing exposure of real usernames and passwords used in ClickHouse clusters
• Ability to map multiple distinct input users to a single ClickHouse user
• HTTP and HTTPS access limitation by IP/IP-mask lists
• Per-user access limitation by IP/IP-mask lists

Resource Control and Performance:

• Per-user query duration limitation with forcible termination via KILL QUERY for timed out or canceled queries
• Per-user request rate limitation
• Per-user concurrent request number limitation
• Request execution delay until it fits per-user limits
• Independent configuration of all limits for each input user and each per-cluster user

Caching and Optimization:

• Configurable per-user response caching
• Built-in protection against thundering herd problem (dogpile effect) in response caches

Connection Security:

• Support for incoming requests via HTTP and HTTPS
• Automatic HTTPS certificate issuing and renewal via Let’s Encrypt
• Request proxying to each configured cluster via either HTTP or HTTPS

Monitoring and Management:

• Exposure of various useful metrics in Prometheus text format
• Prepending User-Agent request header with remote/local address and in/out usernames for querying from system.query_log.http_user_agent
• Configuration updates without restart using SIGHUP signal
• Easy management and operation — just pass config file path to a single chproxy binary

Deployment Instructions

For production environments, do not use address 0.0.0.0 or hack_me_please

1. Create a virtual machine with the ch-proxy product (PROXY-VM) from Yandex Cloud Marketplace using this instruction and with a public IP address.

2. After starting the virtual machine, connect to the PROXY-VM console via SSH (opening access to the VM through port 22, as well as port 8080 for later connection from TEST-VM, and port 8123 via security groups).

3. On the PROXY-VM, configure the /etc/chproxy/chproxy.yml file using sudo nano:

hack_me_please: true

server:
  http:
      listen_addr: ":8080"

users:
  - name: "default"
    to_cluster: "default"
    to_user: "default"

# by default each cluster has `default` user which can be overridden by section `users`
clusters:
- name: "default"
  nodes: ["TEST-VM_ip:8123"]

4. Restart chproxy: sudo systemctl status chproxy, sudo systemctl start chproxy

5. Create another virtual machine (TEST-VM) for testing clickhouse-keeper operation and connect to it via SSH as well

6. Install clickhouse server, client and keeper on TEST-VM (https://clickhouse.com/docs/en/install/debian_ubuntu) (previously opening access for this VM to incoming traffic through port 80 TCP and security groups)

7. Execute a request from TEST-VM: curl "http://PROXY-VM_ip:8123/?query=SELECT%201". If the request executes without errors or hangs, congratulations — ClickHouse is working with ClickHouse-proxy!

• Load balancing between multiple ClickHouse clusters
• Limiting user access to specific clusters by IP addresses
• Response caching for improved query performance
• ClickHouse node health monitoring and automatic traffic switching
• Query rate and execution time control per user
• Automatic SSL certificate acquisition and renewal via Let’s Encrypt
• Hiding real ClickHouse credentials from client applications

Yandex Cloud technical support is available 24/7 to respond to requests. The types of requests available and their response time depend on your pricing plan. You can enable paid support in the management console. Learn more about requesting technical support.

Yandex Cloud does not provide technical support for this product. If you have any issues, please refer to the developer’s information resources.